Are "Multi-modal Tool Results" only supported for Anthropic "provider-defined" tools?

[steve-marmalade]

I have defined a tool type: 'function' and added the experimental_toToolResultContent so that it gets converted from result to content, but the messages I'm seeing in useChat are the standard result format.

Here's a trivial example for illustration purposes:

export const customTool = tool({
  description: "TKTK",
  parameters: z.object({}),
  execute: async () => {
    return "test";
  },
  experimental_toToolResultContent(result) {
    return [
      { type: "text", text: result },
    ];
});

The Multi-modal Tool Results docs state that only Anthropic is supported, which is the model I'm using. The question is whether user-defined tools like the one above are allowed, or only provider-defined tools like anthropic.tools.computer_20241022

[lgrammel]

Yes - this is currently only for Anthropic (and experimental; might change at any point). Should apply to all tools in the anthropic provider; potentially only used by the Computer Use tool on their end tho.

[steve-marmalade]

Hi @lgrammel , thanks for the response! Sorry to be pedantic: I am using model: anthropic("claude-3-7-sonnet-20250219") so when I read "only for Anthropic" my understanding was that any tool (whether provider-defined or one that I wrote via execute) would support this feature. But you are saying it only works for the built-in tools, right?