tests: trace-shape schema and checker

[davidzhao]

What

The nesting of spans is an emergent property of many call sites. A refactor can move a span under the wrong parent while every existing test passes, because each test only asserts the one edge it was written for. This layer writes the rules down once and checks every span in every full-session test against them, and applies the same rules to a real run's export.

How

tests/trace_schema.py (test support, not shipped):

• SPAN_PARENTS: for every span name the framework emits, the parents it may have (ROOT for none, ANY for spans that follow their caller: rpc_call, event_loop_blocked). An unknown name is a violation, so a new span must be registered.
• MAY_OUTLIVE_PARENT: the four child/parent edges where the child is allowed to end after its parent, each with the reason (the three startup spans session.start() does not wait for, on_enter, and stalls whose end is one heartbeat late). Everything else must sit inside its parent, with 2 ms of slack.
• check_trace(): one trace id, every parent present, parent allowed by the schema, bounds, and the per-turn invariants: one agent_turn per lk.speech_id, lk.generation_count equal to the number of generation events, every eou_wait with an outcome. Returns the violations as one line each.
• Two span sources, same rules: from_readable_spans for an in-memory exporter, from_otlp_json for an export downloaded from LiveKit Cloud. As a CLI, uv run python -m tests.trace_schema traces.json prints the span summary and the violations, so a downloaded trace is validated in seconds (a partial export, keyed to one span, is checked with orphans tolerated).

assert_trace_well_formed(exporter.get_finished_spans()) now ends the full-session tests: the tool call and plain reply in test_agent_turn_span, the barge-in and handoff in test_coverage_spans, the hook and redaction sessions in test_eou_wait_span, the lifecycle and SIP sessions in test_startup_spans. Each scenario guards the whole tree.

Tests

tests/test_trace_schema.py: the schema is self-consistent; a sound trace passes; a misparented llm_request (the keyterm-detection mistake), an eou_detection outside its wait, an unknown span, a missing parent, a child outside its parent, a duplicated speech turn, a generation-count mismatch and a second trace id are each reported; the deliberate overruns pass; the OTLP and in-memory sources agree; a full fake session is well-formed.

Not covered here: the job-level spans (job_entrypoint, room_connect, job_shutdown) have no fake-session path and stay on their unit tests, and only a real room exercises the task contexts that leaked earlier, so the CLI on a downloaded export remains part of the workflow.

Stacked on #7143.

🤖 Generated with Claude Code

[devin-ai-integration]

🔍 Devin Review: 1 flag

Not posted on this PR by your GitHub settings — view it in Devin Review. (Configure)

[chenghao-mou]

A fallback can also trigger:

llm_fallback_adapter
└── llm_request_run
    └── llm_request
        └── llm_request_run

• llm_fallback_adapter → llm_request_run → llm_request
• tts_fallback_adapter → tts_request_run → tts_request
• tts_node → tts_stream_adapter → tts_request_run → tts_request