-
Notifications
You must be signed in to change notification settings - Fork 7
fix(mastra): emit tags as first-class Braintrust tags, not metadata #2212
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
Cedric / ViaDézo1er (viadezo1er)
wants to merge
2
commits into
main
Choose a base branch
from
cedric/tags-export
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,22 @@ | ||
| --- | ||
| "braintrust": patch | ||
| --- | ||
|
|
||
| fix(mastra): Emit Mastra tags as first-class Braintrust tags | ||
|
|
||
| Root-span tags from the Mastra observability exporter are now additionally | ||
| logged to the top-level `tags` row field, which Braintrust surfaces as | ||
| first-class tags and filters on. Previously tags were only nested under | ||
| `metadata.tags`, where they were invisible to the tag UI. Matching | ||
| `@mastra/braintrust`, the first-class tags are attached only to the Mastra root | ||
| span. The existing `metadata.tags` field is retained for backward | ||
| compatibility, so this change is purely additive. | ||
|
|
||
| Note: Braintrust's trace-list tag filter is scoped to the trace root span | ||
| (`is_root`). In the zero-config and manual integrations the Mastra root span is | ||
| the Braintrust trace root, so tags are filterable everywhere. When Mastra runs | ||
| nested inside an existing Braintrust span, the tags land on a non-root span: | ||
| they remain filterable via summary / BTQL (which aggregates tags across all | ||
| spans in a trace) but not via the root-scoped trace-list form filter. | ||
|
|
||
| Ports mastra-ai/mastra#12057 (re-fixes #9849). |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,104 @@ | ||
| import { | ||
| afterEach, | ||
| beforeAll, | ||
| beforeEach, | ||
| describe, | ||
| expect, | ||
| test, | ||
| } from "vitest"; | ||
| import { configureNode } from "../node/config"; | ||
| import { _exportsForTestingOnly, initLogger } from "../logger"; | ||
| import { BraintrustObservabilityExporter } from "./mastra"; | ||
|
|
||
| try { | ||
| configureNode(); | ||
| } catch { | ||
| // Best-effort initialization for test environments. | ||
| } | ||
|
|
||
| type MastraExportedSpan = Parameters< | ||
| BraintrustObservabilityExporter["exportTracingEvent"] | ||
| >[0]["exportedSpan"]; | ||
|
|
||
| function makeSpan(overrides: Partial<MastraExportedSpan>): MastraExportedSpan { | ||
| return { | ||
| id: "span-1", | ||
| traceId: "trace-1", | ||
| name: "agent run", | ||
| type: "agent_run", | ||
| startTime: 1_000_000, | ||
| ...overrides, | ||
| }; | ||
| } | ||
|
|
||
| describe("BraintrustObservabilityExporter tags", () => { | ||
| let backgroundLogger: ReturnType< | ||
| typeof _exportsForTestingOnly.useTestBackgroundLogger | ||
| >; | ||
|
|
||
| beforeAll(async () => { | ||
| await _exportsForTestingOnly.simulateLoginForTests(); | ||
| }); | ||
|
|
||
| beforeEach(() => { | ||
| backgroundLogger = _exportsForTestingOnly.useTestBackgroundLogger(); | ||
| initLogger({ projectId: "test-project-id", projectName: "mastra.test.ts" }); | ||
| }); | ||
|
|
||
| afterEach(() => { | ||
| _exportsForTestingOnly.clearTestBackgroundLogger(); | ||
| }); | ||
|
|
||
| // A span is only logged if it was started, so drive both lifecycle events. | ||
| const runSpan = async ( | ||
| exporter: BraintrustObservabilityExporter, | ||
| span: MastraExportedSpan, | ||
| ) => { | ||
| await exporter.exportTracingEvent({ | ||
| type: "span_started", | ||
| exportedSpan: span, | ||
| }); | ||
| await exporter.exportTracingEvent({ | ||
| type: "span_ended", | ||
| exportedSpan: { ...span, endTime: 1_000_001 }, | ||
| }); | ||
| }; | ||
|
|
||
| test("tags surface as a top-level field on the root span only", async () => { | ||
| const exporter = new BraintrustObservabilityExporter(); | ||
| await runSpan( | ||
| exporter, | ||
| makeSpan({ id: "root", isRootSpan: true, tags: ["production", "beta"] }), | ||
| ); | ||
| await runSpan( | ||
| exporter, | ||
| makeSpan({ | ||
| id: "child", | ||
| name: "tool call", | ||
| type: "tool_call", | ||
| isRootSpan: false, | ||
| tags: ["should-not-appear"], | ||
| }), | ||
| ); | ||
|
|
||
| const rows = (await backgroundLogger.drain()) as any[]; | ||
| const byName = (name: string) => | ||
| rows.find((r) => r.span_attributes?.name === name); | ||
|
|
||
| const root = byName("agent run"); | ||
| expect(root).toBeDefined(); | ||
| // Braintrust surfaces the top-level `tags` field as first-class tags. | ||
| expect(root.tags).toEqual(["production", "beta"]); | ||
| // Backward compatibility: prior releases mirrored tags under metadata, so | ||
| // that mirror is retained alongside the first-class top-level field. | ||
| expect(root.metadata?.tags).toEqual(["production", "beta"]); | ||
|
|
||
| // Top-level `tags` are trace-level, so non-root spans carry none. | ||
| const child = byName("tool call"); | ||
| expect(child).toBeDefined(); | ||
| expect(child.tags).toBeUndefined(); | ||
| // Backward compatibility: prior releases mirrored any span's tags under | ||
| // metadata, regardless of root-ness, so that behavior is preserved here. | ||
| expect(child.metadata?.tags).toEqual(["should-not-appear"]); | ||
| }); | ||
| }); |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
When Mastra passes duplicate
tracingOptions.tagsvalues, forwarding the array directly to the top-level Braintrusttagsfield makesSpan.logrunvalidateTags(), which throws on duplicates. BecauseexportTracingEventcatches that exception, a duplicate-tagged rootspan_startedfails beforethis.spans.set(...), so the Mastra root span is not tracked and later child/end events can be dropped or detached; this was previously only metadata and did not hit tag validation. Deduplicate or otherwise sanitize the top-level tags while preserving the metadata mirror.Useful? React with 👍 / 👎.