google realtime: every tool-bearing session connects twice at startup

[anzemur]

Every Gemini realtime session opens two WebSocket connections at startup: the first one is established and then immediately torn down and replaced. It is easy to miss because everything still works, but the logs show Connecting to Gemini Realtime API... twice per session, with sessionShouldClose: true in between.

This affects any agent that declares at least one tool. A tool-less agent connects once.

Why it happens

RealtimeSession._tools starts empty:

// plugins/google/src/realtime/realtime_api.ts:461
private _tools: llm.ToolContext = llm.ToolContext.empty();

session() takes no arguments, so there is no way to construct a session that already knows its tools. The framework configures the session afterwards, via _updateSession(instructions, chatCtx, tools) (agents/src/voice/agent_activity.ts:624), and that applies tools last, behind two awaits (agents/src/llm/realtime.ts:196-221).

Meanwhile #mainTask has already started from the constructor, and it serialises the tools into the setup frame before connecting:

// realtime_api.ts:1018-1021
this.sessionShouldClose.clear();
const config = this.buildConnectConfig();   // reads this._tools, still empty
const session = await this.#client.live.connect({ ..., config });

So it is a race, and in practice the connect always wins. By the time updateTools runs, the setup frame is already on the wire without tools, and updateTools unconditionally asks for a restart:

// realtime_api.ts:796
async updateTools(tools: llm.ToolContext): Promise<void> {
  if (this._tools.equals(tools)) return;
  this._tools = tools;
  this.markRestartNeeded();
}

Note that the restart flag is cleared at the top of every loop iteration, so a restart requested before buildConnectConfig would be absorbed harmlessly. The reconnect happens purely because the request lands after the frame is built.

Not only tools

updateInstructions has the same shape (realtime_api.ts:686-690): when no session is established yet, it calls markRestartNeeded() too. Passing instructions into the RealtimeModel constructor avoids it, because the setter early-returns on string equality, but an agent that only sets instructions on the Agent gets the same reconnect. So this is really about session startup rather than about tools specifically.

Why the obvious fix is wrong

Guarding the restart on "no active session yet" and letting the pending connect pick the tools up does not work: that connect has already sent its setup frame. The session would come up with no tools and never receive them, which is worse than an extra reconnect.

Impact

• An extra WebSocket round trip on every session start.
• The first session is fully established, then discarded.
• More subtly, anything the plugin replays per connection happens twice. With historyConfig.initialHistoryInClientContent and a seeded chat context ending on a user turn, that turn is replayed as live input on each connect, so the agent opens the conversation twice.

Possible directions

1. Connect lazily, on first use, instead of from the constructor, so the framework has finished configuring the session before the setup frame is built. Changes observable behaviour for anything that waits on a session being active.
2. Give the framework a way to signal "configuration complete" before the plugin connects. Correct, but it is an agents-core change.
3. Let RealtimeModel options or session() carry the initial tools so _tools is non-empty from the start and updateTools early-returns. Smallest change, but needs agents-core to pass them through.

All three are decisions about the startup lifecycle rather than local fixes, which is why this is an issue and not a PR. Happy to implement whichever direction you prefer.

Reproduced on @livekit/agents 1.7.0 with @livekit/agents-plugin-google 1.7.0, model gemini-3.1-flash-live-preview.