Skip to content

Commit 49d93f2

Browse files
authored
Handle ollama's deviation from the OpenAI tool streaming spec (#1512)
The canonical sequence of tool streaming chunks according to OpenAI documentation looks like this: ```json [{"index": 0, "id": "call_DdmO9pD3xa9XTPNJ32zg2hcA", "function": {"arguments": "", "name": "get_weather"}, "type": "function"}] [{"index": 0, "id": null, "function": {"arguments": "{\"", "name": null}, "type": null}] [{"index": 0, "id": null, "function": {"arguments": "location", "name": null}, "type": null}] [{"index": 0, "id": null, "function": {"arguments": "\":\"", "name": null}, "type": null}] [{"index": 0, "id": null, "function": {"arguments": "Paris", "name": null}, "type": null}] [{"index": 0, "id": null, "function": {"arguments": ",", "name": null}, "type": null}] [{"index": 0, "id": null, "function": {"arguments": " France", "name": null}, "type": null}] [{"index": 0, "id": null, "function": {"arguments": "\"}", "name": null}, "type": null}] null ``` In ollama's case (tested on `llama3.2:3b` at least) the sequence of chunks is different, the first chunk can already contain the `arguments` string. ``` <Tool call_uiutvfya> task_complete {} ``` This should close #1502
1 parent edf1051 commit 49d93f2

File tree

2 files changed

+12
-7
lines changed

2 files changed

+12
-7
lines changed

packages/mcp-client/package.json

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -34,7 +34,7 @@
3434
"prepare": "pnpm run build",
3535
"test": "vitest run",
3636
"check": "tsc",
37-
"agent": "tsx cli.ts"
37+
"cli": "tsx cli.ts"
3838
},
3939
"files": [
4040
"src",

packages/mcp-client/src/McpClient.ts

Lines changed: 11 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -160,13 +160,18 @@ export class McpClient {
160160
for (const toolCall of delta.tool_calls ?? []) {
161161
// aggregating chunks into an encoded arguments JSON object
162162
if (!finalToolCalls[toolCall.index]) {
163+
/// first chunk of the tool call
163164
finalToolCalls[toolCall.index] = toolCall;
164-
}
165-
if (finalToolCalls[toolCall.index].function.arguments === undefined) {
166-
finalToolCalls[toolCall.index].function.arguments = "";
167-
}
168-
if (toolCall.function.arguments) {
169-
finalToolCalls[toolCall.index].function.arguments += toolCall.function.arguments;
165+
166+
/// ensure .function.arguments is always a string
167+
if (finalToolCalls[toolCall.index].function.arguments === undefined) {
168+
finalToolCalls[toolCall.index].function.arguments = "";
169+
}
170+
} else {
171+
/// any subsequent chunk to the same tool call
172+
if (toolCall.function.arguments) {
173+
finalToolCalls[toolCall.index].function.arguments += toolCall.function.arguments;
174+
}
170175
}
171176
}
172177
if (opts.exitIfFirstChunkNoTool && numOfChunks <= 2 && Object.keys(finalToolCalls).length === 0) {

0 commit comments

Comments
 (0)