Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
1a3d15c
feat(ai-gemini): add geminiTextInteractions() adapter for stateful In…
tombeckenham Apr 24, 2026
1f13ae4
ci: apply automated fixes
autofix-ci[bot] Apr 24, 2026
3745eae
refactor(ai-gemini): surface interactionId via CUSTOM event, drop cor…
tombeckenham Apr 24, 2026
a68f6cd
fix(ai-gemini): address CodeRabbit review feedback on Interactions ad…
tombeckenham Apr 24, 2026
c0acb7c
ci: apply automated fixes
autofix-ci[bot] Apr 24, 2026
17889d3
fix(ai-gemini): emit RUN_ERROR with spec-compliant flat message/code
tombeckenham Apr 24, 2026
1a12f33
feat(examples): wire Gemini Interactions into ts-react-chat, refresh …
tombeckenham Apr 24, 2026
e5f564e
test(ai-gemini): route adapter tests through public core APIs
tombeckenham Apr 24, 2026
d570edb
feat(ai-gemini): built-in tools on geminiTextInteractions()
tombeckenham Apr 24, 2026
7789f5e
ci: apply automated fixes
autofix-ci[bot] Apr 24, 2026
de36448
ci: apply automated fixes (attempt 2/3)
autofix-ci[bot] Apr 24, 2026
a54ddb8
refactor(ai-gemini)!: gate `geminiTextInteractions` behind `/experime…
tombeckenham Apr 27, 2026
6d2feaa
chore(ai-gemini): consolidate `geminiTextInteractions` changesets
tombeckenham Apr 27, 2026
d4c2997
Marked gemini interactions as experimental in example
tombeckenham Apr 27, 2026
7e2d89f
test(e2e): wire up stateful-interactions spec for geminiTextInteractions
tombeckenham Apr 29, 2026
af1aec6
chore(e2e): drop stale "aimock doesn't mock interactions" note
tombeckenham May 16, 2026
f971f30
ci: apply automated fixes
autofix-ci[bot] May 16, 2026
632718b
feat(ai-gemini): include threadId + parentRunId on Interactions RUN_S…
tombeckenham May 18, 2026
e18c2fb
feat(ai-gemini): export GeminiInteractionsCustomEvent discriminated u…
tombeckenham May 18, 2026
869869f
fix(ai-gemini): send Interactions input as string | Content[], not Tu…
tombeckenham May 18, 2026
7b819dd
ci: apply automated fixes
autofix-ci[bot] May 18, 2026
313840a
fix(ai-gemini): wire useChat into the Interactions API correctly
tombeckenham May 18, 2026
1ed4aef
fix(ai-gemini): correctly format Interactions API wire requests
tombeckenham May 18, 2026
ee37317
fix(ai-gemini): tighten experimental Interactions adapter type safety…
tombeckenham May 19, 2026
9e56160
ci: apply automated fixes
autofix-ci[bot] May 19, 2026
5fb183d
fix(ai-gemini): plug interactionId map leak; verify E2E chains the id
tombeckenham May 19, 2026
49ec71a
fix(ai-gemini): evict interactionId on abandonment after tool_calls turn
tombeckenham May 19, 2026
3ffc567
fix(ai-gemini): seal truncated AG-UI events + preserve interactionId …
tombeckenham May 19, 2026
e530796
docs(ai-gemini): correct stale API Reference signatures
tombeckenham May 19, 2026
43e389b
chore(e2e): consistent error handling + harden stateful-interactions …
tombeckenham May 19, 2026
e5a008a
chore(e2e): bump @copilotkit/aimock to ^1.27.0 for Step[] envelope su…
tombeckenham May 21, 2026
7d4c78e
fix(ai-gemini): align experimental adapter + tests with tightened main
tombeckenham May 21, 2026
04081df
fix(e2e): restore AudioGenUI wiring incidentally dropped by PR #527
tombeckenham May 21, 2026
910fca4
fix(ai-gemini): clear lint errors in experimental Interactions adapter
tombeckenham May 21, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions .changeset/gemini-text-interactions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
---
'@tanstack/ai-gemini': minor
---

feat(ai-gemini): add experimental `geminiTextInteractions()` adapter for Gemini's stateful Interactions API (Beta)

Routes through `client.interactions.create` instead of `client.models.generateContent`, so callers can pass `previous_interaction_id` via `modelOptions` and let the server retain conversation history. On each run, the returned interaction id is surfaced via an AG-UI `CUSTOM` event (`name: 'gemini.interactionId'`) emitted just before `RUN_FINISHED` β€” feed it back on the next turn via `modelOptions.previous_interaction_id`.

Exported from a dedicated `@tanstack/ai-gemini/experimental` subpath so the experimental status is load-bearing in your editor and bundle:

```ts
import { geminiTextInteractions } from '@tanstack/ai-gemini/experimental'
```

Scope: text/chat output with function tools, plus the built-in tools `google_search`, `code_execution`, `url_context`, `file_search`, and `computer_use`. Built-in tool activity is surfaced as AG-UI `CUSTOM` events named `gemini.googleSearchCall` / `gemini.googleSearchResult` (and the matching `codeExecutionCall`/`Result`, `urlContextCall`/`Result`, `fileSearchCall`/`Result` variants), carrying the raw Interactions delta payload. Function-tool `TOOL_CALL_*` events are unchanged, and `finishReason` stays `stop` when only built-in tools ran β€” the core chat loop has nothing to execute.

`google_search_retrieval`, `google_maps`, and `mcp_server` are not supported on this adapter and throw a targeted error explaining the alternative. Image/audio output via Interactions is also not routed through this adapter β€” use `geminiText()`, `geminiImage`, or `geminiSpeech` for those.

Marked `@experimental` β€” the underlying Interactions API is Beta and Google explicitly flags possible breaking changes.
256 changes: 243 additions & 13 deletions docs/adapters/gemini.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,26 +42,26 @@ const stream = chat({
import { chat } from "@tanstack/ai";
import { createGeminiChat } from "@tanstack/ai-gemini";

const adapter = createGeminiChat(process.env.GEMINI_API_KEY!, {
const adapter = createGeminiChat("gemini-2.5-pro", process.env.GEMINI_API_KEY!, {
// ... your config options
});

const stream = chat({
adapter: adapter("gemini-2.5-pro"),
adapter,
messages: [{ role: "user", content: "Hello!" }],
});
```

## Configuration

```typescript
import { createGeminiChat, type GeminiChatConfig } from "@tanstack/ai-gemini";
import { createGeminiChat, type GeminiTextConfig } from "@tanstack/ai-gemini";

const config: Omit<GeminiChatConfig, 'apiKey'> = {
const config: Omit<GeminiTextConfig, 'apiKey'> = {
baseURL: "https://generativelanguage.googleapis.com/v1beta", // Optional
};

const adapter = createGeminiChat(process.env.GEMINI_API_KEY!, config);
const adapter = createGeminiChat("gemini-2.5-pro", process.env.GEMINI_API_KEY!, config);
```


Expand Down Expand Up @@ -110,6 +110,192 @@ const stream = chat({
});
```

## Stateful Conversations β€” Interactions API (Experimental)

Gemini's [Interactions API](https://ai.google.dev/gemini-api/docs/interactions) (currently in Beta) offers server-side conversation state β€” the Gemini equivalent of OpenAI's Responses API. Instead of replaying the full message history on every turn, you pass a `previous_interaction_id` and the server retains the transcript. This also improves cache hit rates for repeated prefixes.

The `geminiTextInteractions` adapter routes through `client.interactions.create` and surfaces the server-assigned interaction id via an AG-UI `CUSTOM` event (`name: 'gemini.interactionId'`) emitted just before `RUN_FINISHED`, so you can chain turns.

> **⚠️ Experimental.** Google marks the Interactions API as Beta and explicitly flags possible breaking changes until it reaches general availability. The adapter is exported from the `@tanstack/ai-gemini/experimental` subpath so the experimental status is load-bearing in your editor and bundle. Text output, function tools, and the built-in tools `google_search`, `code_execution`, `url_context`, `file_search`, and `computer_use` are supported. `google_search_retrieval`, `google_maps`, and `mcp_server` still throw on this adapter β€” use `geminiText()` for those or wait for follow-up work.

### Basic Usage

```typescript
import { chat } from "@tanstack/ai";
import { geminiTextInteractions } from "@tanstack/ai-gemini/experimental";

// Turn 1: introduce yourself, capture the interaction id.
let interactionId: string | undefined;

for await (const chunk of chat({
adapter: geminiTextInteractions("gemini-2.5-flash"),
messages: [{ role: "user", content: "Hi, my name is Amir." }],
})) {
if (chunk.type === "CUSTOM" && chunk.name === "gemini.interactionId") {
interactionId = (chunk.value as { interactionId?: string }).interactionId;
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

errr we can't have this in the docs!

}
}

// Turn 2: only send the new turn's content β€” the server has the history.
for await (const chunk of chat({
adapter: geminiTextInteractions("gemini-2.5-flash"),
messages: [{ role: "user", content: "What is my name?" }],
modelOptions: {
previous_interaction_id: interactionId,
},
})) {
// ...stream "Your name is Amir." back to the client.
}
```

### Wiring with `useChat` (React)

The Interactions API is stateful and **does not accept multi-turn history without a `previous_interaction_id`** β€” if a chat client sends `[user, assistant, user]` to a fresh interaction the adapter throws `cannot send prior conversation history on a fresh interaction`. To make `useChat` work, persist the server-assigned id and send it back on the next turn:

**Server route** (e.g. TanStack Start handler):

```typescript
import {
chat,
chatParamsFromRequestBody,
toServerSentEventsResponse,
} from "@tanstack/ai";
import { geminiTextInteractions } from "@tanstack/ai-gemini/experimental";

export async function POST({ request }: { request: Request }) {
const params = await chatParamsFromRequestBody(await request.json());

// The client sends body.previousInteractionId; AG-UI maps `body` into
// `forwardedProps` on the wire.
const previousInteractionId =
typeof params.forwardedProps.previousInteractionId === "string"
? params.forwardedProps.previousInteractionId
: undefined;

const stream = chat({
adapter: geminiTextInteractions("gemini-2.5-flash"),
messages: params.messages,
modelOptions: {
previous_interaction_id: previousInteractionId,
store: true, // required for chaining on the next turn
},
});

return toServerSentEventsResponse(stream);
}
```

**React client**:

```tsx
import { useEffect, useMemo, useState } from "react";
import { fetchServerSentEvents, useChat } from "@tanstack/ai-react";
import type { GeminiInteractionsCustomEventValue } from "@tanstack/ai-gemini/experimental";

function GeminiChat() {
const [interactionId, setInteractionId] = useState<string | undefined>();

const body = useMemo(
() => (interactionId ? { previousInteractionId: interactionId } : {}),
[interactionId],
);

const { messages, setMessages, sendMessage } = useChat({
connection: fetchServerSentEvents("/api/chat"),
body,
onCustomEvent: (eventType, data) => {
if (eventType === "gemini.interactionId") {
const value = data as
| GeminiInteractionsCustomEventValue<"gemini.interactionId">
| undefined;
if (value?.interactionId) setInteractionId(value.interactionId);
}
},
});

// Switching provider/model resets the server-side chain β€” drop the id
// AND the local message history together, otherwise the next turn
// ships multi-turn messages with no previous_interaction_id and the
// adapter errors out.
const [provider, setProvider] = useState("gemini-interactions");
useEffect(() => {
setInteractionId(undefined);
setMessages([]);
}, [provider]);

// ...render messages, call sendMessage(input)
}
```

The full working example is in [`examples/ts-react-chat`](https://github.com/TanStack/ai/tree/main/examples/ts-react-chat) β€” see `src/routes/index.tsx` for the client and `src/routes/api.tanchat.ts` for the route.

### How it differs from `geminiText`

| Concern | `geminiText` | `geminiTextInteractions` |
| --- | --- | --- |
| Underlying endpoint | `models:generateContent` | `interactions:create` |
| Conversation state | Stateless β€” send full history each turn | Stateful β€” server retains transcript via `previous_interaction_id` |
| Provider options shape | camelCase (`stopSequences`, `responseModalities`, `safetySettings`) | snake_case (`generation_config`, `response_modalities`, `previous_interaction_id`) |
| Built-in tools | `google_search`, `code_execution`, `url_context`, `file_search`, `google_maps`, `google_search_retrieval`, `computer_use` | `google_search`, `code_execution`, `url_context`, `file_search`, `computer_use` (only the first four stream `CUSTOM` event activity; `computer_use` is accepted in the request but does not currently emit per-delta events) |
| Stability | GA | Experimental (Google Beta) |

### Provider Options

The adapter exposes Interactions-specific options on `modelOptions`:

```typescript
import { geminiTextInteractions } from "@tanstack/ai-gemini/experimental";

const stream = chat({
adapter: geminiTextInteractions("gemini-2.5-flash"),
messages,
modelOptions: {
// Stateful chaining β€” passed only on turn 2+.
previous_interaction_id: "int_abc123",

// Persist the interaction server-side (default true). Must be true for
// previous_interaction_id to work on the *next* turn.
store: true,

// Per-request system instruction (interaction-scoped β€” re-specify each turn).
system_instruction: "You are a helpful assistant.",

// snake_case generation config distinct from geminiText's camelCase one.
generation_config: {
thinking_level: "LOW",
thinking_summaries: "auto",
stop_sequences: ["<done>"],
},

response_modalities: ["text"],
},
});
```

### Reading the interaction id

The server's interaction id arrives as an AG-UI `CUSTOM` event emitted just before `RUN_FINISHED`:

```typescript
for await (const chunk of stream) {
if (chunk.type === "CUSTOM" && chunk.name === "gemini.interactionId") {
const id = (chunk.value as { interactionId: string }).interactionId;
// Persist `id` wherever you store per-user conversation pointers β€”
// pass it back on the next turn as `previous_interaction_id`.
}
}
```

### Caveats

- **Multi-turn history requires `previous_interaction_id`.** The Interactions API has no stateless replay path β€” sending more than one message in `messages` without a `previous_interaction_id` throws. Chat UIs that maintain local history must capture the server-assigned id and chain (see [Wiring with `useChat`](#wiring-with-usechat-react)). On provider/model switch, also clear the local message buffer.
- **Tools, `system_instruction`, and `generation_config` are interaction-scoped.** Per Google's docs these are NOT inherited from a prior interaction via `previous_interaction_id` β€” pass them again on every turn you need them.
- `store: false` is incompatible with `previous_interaction_id` (no state to recall) and with `background: true`.
- Retention (as of the time of writing): **55 days on the Paid Tier, 1 day on the Free Tier.** See [Google's Interactions API docs](https://ai.google.dev/gemini-api/docs/interactions) for current retention policy.
- Built-in tools in scope (`google_search`, `code_execution`, `url_context`, `file_search`, `computer_use`) are wired through as request tools. Per-delta activity for the four search/exec tools streams back as AG-UI `CUSTOM` events β€” `gemini.googleSearchCall` / `gemini.googleSearchResult` (and the matching `codeExecutionCall`/`Result`, `urlContextCall`/`Result`, `fileSearchCall`/`Result`) β€” carrying the raw Interactions delta. `computer_use` is accepted in the request but the Interactions API does not currently emit per-delta `CUSTOM` events for it. Function-tool `TOOL_CALL_*` events are unchanged, and `finishReason` stays `stop` when only built-in tools ran.
- `google_search_retrieval`, `google_maps`, and `mcp_server` still throw a targeted error on this adapter. Use `geminiText()` for the first two, or wait for a dedicated follow-up for `mcp_server`.
- Image and audio output via Interactions aren't routed through this adapter yet β€” it's text-only. Use `geminiImage` / `geminiSpeech` for non-text generation for now.

## Model Options

Gemini supports various model-specific options:
Expand Down Expand Up @@ -324,33 +510,66 @@ These models use the dedicated `generateImages` API.

## API Reference

### `geminiText(config?)`
### `geminiText(model, config?)`

Creates a Gemini text/chat adapter using environment variables.

**Parameters:**

- `model` - The model name (e.g. `"gemini-2.5-pro"`)
- `config.baseURL?` - Custom base URL (optional)

**Returns:** A Gemini text adapter instance.

### `createGeminiText(apiKey, config?)`
### `createGeminiChat(model, apiKey, config?)`

Creates a Gemini text/chat adapter with an explicit API key.

**Parameters:**

- `model` - The model name (e.g. `"gemini-2.5-pro"`)
- `apiKey` - Your Gemini API key
- `config.baseURL?` - Custom base URL (optional)

**Returns:** A Gemini text adapter instance.

### `geminiSummarize(config?)`
### `geminiTextInteractions(model, config?)` (experimental)

Creates a Gemini Interactions API text adapter using environment variables. Backs the stateful conversation pattern via `previous_interaction_id`.

**Returns:** A Gemini Interactions text adapter instance.

### `createGeminiTextInteractions(model, apiKey, config?)` (experimental)

Creates a Gemini Interactions API text adapter with an explicit API key.

- `model` - The model name (e.g. `gemini-2.5-flash`)
- `apiKey` - Your Google API key
- `config.baseURL?` - Custom base URL (optional)

**Returns:** A Gemini Interactions text adapter instance.
Comment thread
coderabbitai[bot] marked this conversation as resolved.

### `geminiSummarize(model, config?)`

Creates a Gemini summarization adapter using environment variables.

**Parameters:**

- `model` - The model name (e.g. `"gemini-2.5-pro"`)
- `config.baseURL?` - Custom base URL (optional)

**Returns:** A Gemini summarize adapter instance.

### `createGeminiSummarize(apiKey, config?)`
### `createGeminiSummarize(apiKey, model, config?)`

Creates a Gemini summarization adapter with an explicit API key.

**Parameters:**

- `apiKey` - Your Gemini API key
- `model` - The model name (e.g. `"gemini-2.5-pro"`)
- `config.baseURL?` - Custom base URL (optional)

**Returns:** A Gemini summarize adapter instance.

### `geminiImage(model, config?)`
Expand All @@ -376,15 +595,26 @@ Creates a Gemini image adapter with an explicit API key.

**Returns:** A Gemini image adapter instance.

### `geminiTTS(config?)`
### `geminiSpeech(model, config?)`

Creates a Gemini text-to-speech adapter using environment variables.

Creates a Gemini TTS adapter using environment variables.
**Parameters:**

- `model` - The model name (e.g. `"gemini-2.5-flash-preview-tts"`)
- `config.baseURL?` - Custom base URL (optional)

**Returns:** A Gemini TTS adapter instance.

### `createGeminiTTS(apiKey, config?)`
### `createGeminiSpeech(model, apiKey, config?)`

Creates a Gemini TTS adapter with an explicit API key.
Creates a Gemini text-to-speech adapter with an explicit API key.

**Parameters:**

- `model` - The model name (e.g. `"gemini-2.5-flash-preview-tts"`)
- `apiKey` - Your Gemini API key
- `config.baseURL?` - Custom base URL (optional)

**Returns:** A Gemini TTS adapter instance.

Expand Down
Loading
Loading