fix(anthropic): preserve MCP tool call context in spans - #844
Kexin Chen (March-7) wants to merge 2 commits into
Conversation
| # Supplement the recorded provider response with deterministic ordering and | ||
| # incomplete-pair cases that cannot be requested reliably from a live model. |
There was a problem hiding this comment.
I don't think this is true! Please try using the vcr tests https://github.com/braintrustdata/braintrust-sdk-python/blob/main/docs/vcr-testing.md
There was a problem hiding this comment.
You're right — I hadn't established that those cases couldn't be recorded. I've removed that claim and the synthetic test_anthropic_mcp_tool_spans_pair_by_id cases.
Following docs/vcr-testing.md, the sync/async regression reuses the existing MCP cassette unchanged. Its child-span expectations now come from the raw recorded provider response, including the call/result ID association, tool name, input, and result text. The HTTP request and cassette payload are unchanged. Both cases fail against the pre-fix production code and pass with this fix using --vcr-record=none.
The separate streaming-input test remains a constructed SDK-event unit test, not a recording of a live MCP SSE exchange. I haven't made new provider calls or generated a cassette from synthetic data.
Validation on Python 3.10.20 and 3.14.5: Anthropic 1.8.0 has 61 passing tests; 0.48.0 has 44 passing tests and 17 skips on each Python version. Pre-commit and pylint on the changed test file also pass.
Anthropic MCP connector results currently produce generic
mcptool spans without the corresponding tool name, input, or call type, even though the parent LLM span retains the original response. Recognizemcp_tool_usealongside built-in server tool calls and pair it with results by ID. For streamed beta messages, use the SDK's beta accumulator so MCP input JSON deltas reach the final span; accommodate the accumulator signatures used by both supported SDK versions.Extend the existing MCP HTTP cassette test to check child spans for sync and async clients against the raw recorded provider response. Keep a separate typed SDK-event unit test for streamed MCP input split across JSON deltas. That unit test is not a recorded MCP SSE exchange. Existing HTTP requests and cassettes are unchanged.
Fixes #797.
Validation:
AI-assisted implementation and testing with Codex.