Python: MCPStreamableHTTPTool.connect() never sends a header_provider's headers, so any headers_env/header_provider-authenticated server 401s on connect

[ReubenHawley]

Summary

MCPStreamableHTTPTool's request-header injection only ever runs for call_tool(). The initial connect() handshake (and any later reconnect) never carries the header_provider's headers, so a server that requires the header to authenticate the connect handshake itself gets a 401 before a single tool call is ever made.

Where this lives (checked against PyPI 1.0.1 and 1.15.0; 1.15.0 improves but does not fix this)

In agent-framework-core 1.0.1, get_mcp_client()'s httpx request hook reads a module-level contextvars.ContextVar (_mcp_call_headers) that is only ever .set() inside call_tool():

# get_mcp_client()
async def _inject_headers(request: Request) -> None:
    headers = _mcp_call_headers.get({})
    for key, value in headers.items():
        request.headers[key] = value
...

# call_tool()
if self._header_provider is not None:
    headers = self._header_provider(kwargs)
    token = _mcp_call_headers.set(headers)
    try:
        return await super().call_tool(tool_name, **kwargs)
    finally:
        _mcp_call_headers.reset(token)

connect() drives get_mcp_client() directly and never sets _mcp_call_headers, so the hook always sees {} during the initial handshake.

1.15.0 adds an "ambient request" fallback for when no call is in flight, which calls self._header_provider({}) (empty kwargs) on a request outside call_tool(). This helps a static header_provider (one that ignores its kwargs argument and always returns the same headers), but does nothing for a header_provider that legitimately depends on its kwargs argument (e.g. a provider that expects the caller to inject a per-turn secret via FunctionInvocationContext.kwargs, as our agent-runtime integration does): calling it with {} silently returns no headers, and the request still goes out unauthenticated. Worth noting explicitly in case this is read as already fixed.

Even a correctly-scoped fix at the call site would not be reliable, because connect() does not run inline: it dispatches through a lifecycle-owner task created via asyncio.create_task() (_ensure_lifecycle_owner, _run_lifecycle_owner). A contextvars.ContextVar set by the caller around connect() is only visible to that new task via the context snapshot taken at task-creation time; it is not visible to any later reconnect (a new owner task, created after the old one's finally clears _lifecycle_owner_task), and .reset() on the caller's own context cannot affect the task's copy anyway. So the real fix likely belongs inside get_mcp_client()/_connect_on_owner() itself, resolving and applying the header_provider's headers (with real kwargs where available, or a documented static-provider convention) at the point the client is actually built or the request is actually sent, not via a ContextVar set at an outer call site.

Reproduction

Live-reproduced against a real, deployed MCP server configured with header_provider-based bearer-token auth: a bare curl (no Authorization header) against the server's /mcp endpoint returns HTTP/2 401 with www-authenticate: Bearer error="invalid_token". connect() against that same server via MCPStreamableHTTPTool fails/hangs the same way, because it sends the same unauthenticated request. call_tool() against the same server, once connected another way, sends the header correctly.

Suggested fix

For header-provider-authenticated servers, apply the resolved headers to the client construction path itself (get_mcp_client()/_connect_on_owner()), not only around call_tool(). If a header_provider is present but the caller has not supplied kwargs (i.e. connect/discovery/reconnect), consider either:

• documenting that header_provider must tolerate being called with {} and return a best-effort static credential in that case, or
• providing a distinct, explicit hook for "headers needed at connect time" separate from the per-call kwargs-dependent provider, so a caller with genuinely per-call-dependent auth and genuinely connect-time-static auth are not forced into the same callable signature.

Workaround in use

We worked around this downstream (not in the vendored library) by giving header-provider-authenticated servers their own httpx.AsyncClient with the resolved headers baked in as client defaults (httpx.AsyncClient(headers=...)), rather than relying on header_provider/_mcp_call_headers for these servers at all. This sidesteps the ContextVar/task-lifecycle issue entirely, since default client headers apply to every request the client sends regardless of which task sends it.

Versions

• agent-framework-core / agent-framework-openai: reproduced on 1.0.1 (pinned in our project); confirmed not fixed (for kwargs-dependent providers) on 1.15.0 (latest at time of writing) by downloading the wheel and diffing _mcp.py.

[lukiod]

Checked this against main (not just the PyPI 1.0.1/1.15.0 builds in the report) and the core architectural concern here already looks resolved.

The cross-task ContextVar problem described in this issue (a value set inside call_tool never being visible to the request task spawned earlier at connect() time) is exactly what #7305 addressed, landed 2026-07-30. MCPStreamableHTTPTool now keeps self._active_call_headers as an instance level fallback alongside the ContextVar (_mcp.py, constructor around line 3123), which is visible across tasks since it is not task local. _inject_headers in get_mcp_client() reads the ContextVar first, then falls back to _active_call_headers, then to invoking header_provider({}) for a genuinely ambient request (initialize, discovery, pings).

There is real, passing test coverage for exactly this, not just the code path:

• test_mcp_streamable_http_tool_header_provider_applies_across_transport_tasks drives the real transport against a mock server and asserts the per-call header reaches the tools/call HTTP request specifically, the cross-task case this issue is about.
• test_mcp_streamable_http_tool_header_provider_injects_on_ambient_request and _ambient_kwarg_error_is_benign cover the initialize/ambient path.

Ran the full header_provider test group on a clean main checkout, 17 passed:

python -m pytest tests/core/test_mcp.py -k "header_provider" -q
.................  [100%]

One real, remaining nuance worth being explicit about, since it is the part of this issue's report that is still technically accurate: for a header_provider that only produces headers from real call-time kwargs (like the example in this issue), get_mcp_client()'s ambient fallback calls header_provider({}), and a KeyError from that call is deliberately swallowed so the initialize handshake proceeds unauthenticated rather than crashing, see test_mcp_streamable_http_tool_header_provider_ambient_request_tolerates_kwargs_provider's own docstring. That is a real, tested, intentional choice, not an oversight, but it does mean a server that requires auth on the connect handshake itself will still see an unauthenticated initialize request from a kwargs-dependent provider, and will presumably reject it at the HTTP level (a real error from the server, not a silent hang).

If that is still the actual gap you want addressed, it reads more like the "distinct, explicit hook for headers needed at connect time" feature request from this issue's own suggested fix than a bug in the current header delivery path, since the delivery mechanism itself is now correct and tested. Happy to take that on if so, just want to confirm the scope before starting given how much has already landed here since this was filed.