Skip to content

bug(cohere): emit OpenAI-shaped tool_calls in CohereChatCompleteStreamChunkTransform - #1906

Open
definitelynotguru wants to merge 1 commit into
Portkey-AI:mainfrom
definitelynotguru:fix/cohere-stream-tool-calls
Open

definitelynotguru wants to merge 1 commit into
Portkey-AI:mainfrom
definitelynotguru:fix/cohere-stream-tool-calls

Conversation

@definitelynotguru

Copy link
Copy Markdown

Why

Cohere v2 streams each tool call as tool-call-start / tool-call-delta events where delta.message.tool_calls is a single object and the top-level index is the tool-call index (Cohere chat-stream docs). CohereChatCompleteStreamChunkTransform forwarded that object unchanged and put the tool index into choices[].index. OpenAI-compatible clients expect delta.tool_calls to be an array of deltas that each carry their own index, all inside choice 0, so SDKs failed to accumulate streamed tool calls (and the second tool call appeared as a second choice).

What changed

  • src/providers/cohere/chatComplete.ts: for tool-call-start / tool-call-delta chunks, emit tool_calls: [{ index, ...toolCall }]; keep choices[].index at 0 for content, tool and the terminal message-end chunk.
  • src/providers/cohere/chatComplete.test.ts (new): regression tests for two streamed tool calls, plain text deltas, and malformed JSON.

Scope

Cohere streaming chat completions only. Non-streaming and other providers are untouched.

Blast Radius

Clients that read choices[0].delta.tool_calls from Cohere streams now receive the OpenAI shape. Cohere only returns one generation, so pinning the choice index to 0 matches the non-streaming response.

Verification

  • npx jest src/providers/cohere/chatComplete.test.ts: new tool-call test fails on main (tool_calls is an object, no index), all 4 pass with the fix.
  • npx prettier --check on changed files passes; husky pre-commit (format:check) and pre-push (npm run build && node start-test.js) pass.
  • Type of change: bug fix (non-breaking).

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant