fix(voice): report e2e_latency for the first reply after a handoff

[longcw]

Problem: e2e_latency is computed from the user turn's metrics held in a local of the reply task, so the first reply after an agent handoff never gets it, and a tool reply after an inline AgentTask reports the whole sub-conversation as one latency. The slowest turns of a call are the ones missing from the message, the lk.e2e_latency span attribute, and the OTel histogram.

Fix: The session keeps the latest user turn no speech has answered, set where the reply task commits the user message. The tool reply reads it, a say() or generate_reply() created inside on_enter claims it, and it ends when a speech reports it, when the reply chain ends without a tool reply or a handoff, or when an on_enter returns without speaking.

Closes #7157. Supersedes #7161.

Context for reviewing and coding agents

How to see it

tests/test_e2e_latency_handoff.py has 15 cases; most fail on main. Each asserts e2e_latency == started_speaking_at - user.stopped_speaking_at on the assistant message that answers a turn, and its absence on the message that does not. The inline-task case is the flow user -> tool -> await AgentTask -> the task asks -> the user answers -> the task completes -> tool reply: the question carries the first turn's latency and the tool reply carries the answer turn's.

Which speeches take the pending turn

The tool reply in _pipeline_reply_task_impl reads it in place of the threaded user_metrics if not forwarded_text else None. The two are equal on the paths main covers, since a step that spoke has already cleared it. say() and _generate_reply() claim it at creation, only under the existing _OnEnterContextVar gate, so a nested AgentTask awaited from on_enter claims it in its own on_enter, and a reply created after that await claims whatever turn the task left open. A generate_reply() outside on_enter, such as an inactivity nudge or a tool's background task, never reads it. The realtime path reports nothing, as on main.

When the turn ends unanswered

Three places: the report block of the speech that owns it, by identity, so an interrupted reply leaves it for the next speech; the end of _pipeline_reply_task_impl when no tool reply follows, the session's agent is unchanged, and the agent is not an AgentTask that just completed; and the finally of on_enter, for the turn that was pending when on_enter started, so a turn the user commits while on_enter runs stays pending for its own reply chain. The AgentTask.done() check exists because complete() only resolves a future and the switch back to the awaiting tool happens later, so the inner activity's chain end must not drop the turn the outer tool reply will answer.

Why a session field and not the task-local ContextVar of #7161

The value has to cross activities. An on_enter task has no speech handle, and an inline task's on_enter runs while the outer tool is still awaiting, so neither a handle attribute nor the outer task's context reaches it. #7161 also carried the value unconditionally, so an LLM that spoke before the handoff call produced two e2e_latency values for one user turn, and consumed it in any _generate_reply, so a say() greeting left it for a later unrelated reply.

Why a speech spawned from on_enter that speaks later reports nothing

Once on_enter returns without speaking, the agent has declined the turn. A speech from a task it spawned is a new utterance, and the on_enter end clear is what keeps a handoff nothing in the conversation caused from reporting a latency from minutes earlier.

Test harness change

FakeVAD restarted its clock per stream, and only the first stream ever saw an audio frame, so a spoken turn after a handoff had no timing anchors in any test. It now shares one clock across streams and waits for a frame only once. The 19 unit modules that use it pass unchanged.

[devin-ai-integration]

Devin Review found 2 new potential issues.

1 flag not posted on this PR by your GitHub settings — view it in Devin Review. (Configure)

[chenghao-mou]

lgtm