Feeding tool result back into the messages

[msjoshlopez]

[elitan]

EDIT: It's supported here: https://sdk.vercel.ai/examples/node/tools/call-tools-with-automatic-roundtrips

OLD ANSWER:

Here's what I'm experimenting with right now:

async function main() {

  let messages: CoreMessage[] = [
    { role: "user", content: "Hi!" },
    { role: "assistant", content: "Hello, how can I help?" },
    {
      role: "user",
      content: "What's the weather in New York?",
    },
  ];

  let textResultIsPresent = false;
  let textResult = "";

  while (!textResultIsPresent) {
    await sleep(2000);
    const { text, toolCalls, toolResults } = await generateText({
      model,
      tools: {
        weather: tool({
          description: "Get the weather in a location",
          parameters: z.object({
            location: z
              .string()
              .describe("The location to get the weather for"),
          }),
          execute: async ({ location }) => ({
            location,
            temperature: 72 + Math.floor(Math.random() * 21) - 10,
          }),
        }),
      },
      messages,
    });

    if (toolResults.length > 0 && toolCalls.length > 0) {
      messages.push({
        role: "assistant" as const,
        content: toolCalls,
      });

      messages.push({
        role: "tool" as const,
        content: toolResults,
      });
    } else {
      textResult = text;
      textResultIsPresent = true;
    }

    console.log(JSON.stringify(messages, null, 2));
  }

  console.log(textResult);
}

Output:

[
  {
    "role": "user",
    "content": "Hi!"
  },
  {
    "role": "assistant",
    "content": "Hello, how can I help?"
  },
  {
    "role": "user",
    "content": "What's the weather in New York?"
  },
  {
    "role": "assistant",
    "content": [
      {
        "type": "tool-call",
        "toolCallId": "call_mlnWc8xQxqlQNOrCwVA2SJTw",
        "toolName": "weather",
        "args": {
          "location": "New York"
        }
      }
    ]
  },
  {
    "role": "tool",
    "content": [
      {
        "toolCallId": "call_mlnWc8xQxqlQNOrCwVA2SJTw",
        "toolName": "weather",
        "args": {
          "location": "New York"
        },
        "result": {
          "location": "New York",
          "temperature": 66
        }
      }
    ]
  }
]
[
  {
    "role": "user",
    "content": "Hi!"
  },
  {
    "role": "assistant",
    "content": "Hello, how can I help?"
  },
  {
    "role": "user",
    "content": "What's the weather in New York?"
  },
  {
    "role": "assistant",
    "content": [
      {
        "type": "tool-call",
        "toolCallId": "call_mlnWc8xQxqlQNOrCwVA2SJTw",
        "toolName": "weather",
        "args": {
          "location": "New York"
        }
      }
    ]
  },
  {
    "role": "tool",
    "content": [
      {
        "toolCallId": "call_mlnWc8xQxqlQNOrCwVA2SJTw",
        "toolName": "weather",
        "args": {
          "location": "New York"
        },
        "result": {
          "location": "New York",
          "temperature": 66
        }
      }
    ]
  }
]

The current temperature in New York is 66°F.

[zineanteoh]

Is there anyway to stream the final output?

The current temperature in New York is 66°F.

[benjaminalgreen]

Have been exploring how to do exactly this, thanks!

Just curious, how good/bad is the AI at being able to find the right info from previous tool responses when appropriate? E.g. Let's say you ask it to find the weather for New York. Then you ask it later how the weather in NY compares to SF. Does it know to search for SF and use the previous data from NY if prompted well?

[lgrammel]

With useChat, you can now do automatic roundtrips: https://sdk.vercel.ai/docs/ai-sdk-ui/chatbot-with-tool-calling

Server-side roundtrip support in streamText and generateText is planned

[benjaminalgreen]

lgrammel, does streamText with maxToolRoundtrips on the server support sending toolResults back to the frontend?

[lgrammel]

yes

[castortech]

yes

First of, thanks so much for having this in place and on schedule. Really appreciated.

Right now I am getting an empty array for toolCalls and toolResults is returning a never[] | undefined

I am on the latest versions from 1 day ago.

[hminle]

how to use this one maxToolRoundtrips?

[lgrammel]

https://sdk.vercel.ai/docs/ai-sdk-core/tools-and-tool-calling#multi-step-calls

[Wh1isper]

I want to save the tool-result during consumption of fullStream . But it seems that only tool-call appears in the stream.

const { usage, fullStream, text, response } = await streamText({
    model,
    messages,
    abortSignal: this.abortController?.signal,
    onError: (error) => {
        console.error('Error', JSON.stringify(error, null, 2));
        throw error;
    },
    maxSteps: 20,
    tools: tools,
    maxTokens: 64000,
    headers: {
        'anthropic-beta': 'output-128k-2025-02-19',
    },
});
for await (const partialStream of fullStream) {
	emitMessagePart(partialStream)
	// ...
}

[lgrammel]

Do your tools have an execute method? Have any errors been thrown? (onError or in the stream)

[Wh1isper]

@lgrammel Tool works well with no error.

I found this workaround and works well: use onStepFinish in streamText

onStepFinish: ({toolResults}) => {
    for (const toolResult of toolResults) {
        this.emitMessagePart(toolResult);
    }
}

[lgrammel]

Weird - we have tests that check that full stream contains the tool result parts, and no changes were made in that area recently.

[Wh1isper]

I rechecked the code and it might have something to do with an error, I'm still debugging it.

Let me reconfirm, is it also possible that the tool-return message is missing if the error occurs but the tool is called successfully?

[lgrammel]

Potentially - please check for error parts or use onError on streamText.

[github-actions[bot]]

This discussion was automatically locked because it has not been updated in over 30 days. If you still have questions about this topic, please ask us at community.vercel.com/ai-sdk