Repository navigation
bug(cohere): emit OpenAI-shaped tool_calls in CohereChatCompleteStreamChunkTransform - #1906
Open
definitelynotguru wants to merge 1 commit into
Open
definitelynotguru wants to merge 1 commit into
definitelynotguru wants to merge 1 commit into
Conversation
…teStreamChunkTransform
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Cohere v2 streams each tool call as
tool-call-start/tool-call-deltaevents wheredelta.message.tool_callsis a single object and the top-levelindexis the tool-call index (Cohere chat-stream docs).CohereChatCompleteStreamChunkTransformforwarded that object unchanged and put the tool index intochoices[].index. OpenAI-compatible clients expectdelta.tool_callsto be an array of deltas that each carry their ownindex, all inside choice 0, so SDKs failed to accumulate streamed tool calls (and the second tool call appeared as a second choice).What changed
src/providers/cohere/chatComplete.ts: fortool-call-start/tool-call-deltachunks, emittool_calls: [{ index, ...toolCall }]; keepchoices[].indexat0for content, tool and the terminalmessage-endchunk.src/providers/cohere/chatComplete.test.ts(new): regression tests for two streamed tool calls, plain text deltas, and malformed JSON.Scope
Cohere streaming chat completions only. Non-streaming and other providers are untouched.
Blast Radius
Clients that read
choices[0].delta.tool_callsfrom Cohere streams now receive the OpenAI shape. Cohere only returns one generation, so pinning the choice index to 0 matches the non-streaming response.Verification
npx jest src/providers/cohere/chatComplete.test.ts: new tool-call test fails onmain(tool_callsis an object, noindex), all 4 pass with the fix.npx prettier --checkon changed files passes; husky pre-commit (format:check) and pre-push (npm run build && node start-test.js) pass.