Skip to content

[bot] Amazon Bedrock Agents (bedrock-agent-runtime invoke_agent) not instrumented #568

Description

@braintrust-bot

Summary

This repo instruments boto3's bedrock-runtime service (model invoke/converse) via py/src/braintrust/integrations/bedrock_runtime/, but the separate bedrock-agent-runtime service — which drives Amazon Bedrock Agents' invoke_agent API (multi-step agent orchestration: action-group/tool calls, knowledge-base retrieval, and a returned execution trace) — is explicitly excluded, not just unimplemented.

py/src/braintrust/integrations/bedrock_runtime/patchers.py gates instrumentation with _is_bedrock_runtime_client, which only wraps a client when service_model.service_name == "bedrock-runtime". A boto3.client("bedrock-agent-runtime") client has a different service name and is passed straight through unwrapped, so calls to invoke_agent() (and invoke_flow() for Bedrock Flows) produce no spans at all.

This is a meaningful agent-run execution surface distinct from a single model invoke — it returns a streamed EventStream with agent reasoning/tool-use/observation chunks plus a final output, analogous to what's already traced for other agent frameworks in this repo (e.g. openai_agents, crewai, strands).

Braintrust docs status: unclear. https://www.braintrust.dev/docs/guides/tracing lists "AWS Bedrock" generically among supported providers but does not distinguish Bedrock Agents/invoke_agent from the base model-invoke surface, and no integration page documents agent-orchestration coverage.

What needs to be instrumented

SDK surface Already traced?
bedrock-agent-runtime client invoke_agent(agentId, agentAliasId, sessionId, inputText, ...) No
invoke_flow(...) (Bedrock Flows) No

Upstream sources

Local repo files inspected

  • py/src/braintrust/integrations/bedrock_runtime/integration.pyimport_names = ("botocore",), min_version = "1.34.116", only patcher is BedrockClientCreatorPatcher
  • py/src/braintrust/integrations/bedrock_runtime/patchers.py_is_bedrock_runtime_client checks service_model.service_name == "bedrock-runtime", explicitly excluding any other boto3 service name including bedrock-agent-runtime
  • py/src/braintrust/integrations/bedrock_runtime/tracing.py — wrappers exist only for converse, converse_stream, invoke_model, invoke_model_with_response_stream
  • Repo-wide grep for bedrock-agent, bedrock_agent, invoke_agent under py/src/braintrust/ — the only hit is an unrelated local test helper _invoke_agent in py/src/braintrust/integrations/langchain/test_deepagents.py (a LangChain/deepagents test fixture, unrelated to AWS)
  • README.md integrations table — no separate Bedrock Agents entry

Metadata

Metadata

Assignees

No one assigned

    Fields

    No fields configured for Feature.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions