Skip to content

Feat: Adding Gemini tools - URL Context and Grounding with Google Search #5959

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 58 commits into from
Jul 27, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
58 commits
Select commit Hold shift + click to select a range
5ff5993
feat: Adding more settings and control over Gemini
HahaBill Jun 19, 2025
afcb66d
feat: Adding parameter titles and descriptions + translation to all l…
HahaBill Jun 22, 2025
ac96e99
feat: adding more translations
HahaBill Jun 22, 2025
121e243
Merge branch 'RooCodeInc:main' into feat/finer-grained-control-gemini
HahaBill Jun 23, 2025
a20774e
Merge branch 'RooCodeInc:main' into feat/finer-grained-control-gemini
HahaBill Jun 23, 2025
67b4762
feat: adding `contextLimit` implementation from `maxContextWindow` PR…
HahaBill Jun 25, 2025
9595f76
feat: max value for context limit to model's limit + converting descr…
HahaBill Jun 25, 2025
26b1f53
feat: all languages translated
HahaBill Jun 26, 2025
8f468f4
feat: changing profile-specific threshold in context management setti…
HahaBill Jun 26, 2025
24e8ed5
Merge branch 'RooCodeInc:main' into feat/finer-grained-control-gemini
HahaBill Jun 26, 2025
98e813d
feat: max value of maxOutputTokens is model's maxTokens + adding more…
HahaBill Jun 26, 2025
91c16cb
feat: improve unit tests and adding `data-testid` to slider and check…
HahaBill Jun 26, 2025
1497edd
fix: small changes in geminiContextManagement descriptions + minor fix
HahaBill Jun 26, 2025
a204169
fix: Switching from "Gemini Context Management" to "Token Management
HahaBill Jun 29, 2025
83f02d5
fix: input field showed NaN -> annoying UX
HahaBill Jun 29, 2025
c438277
fix: Removing redundant "tokens" after the "set context limit"'s chec…
HahaBill Jul 2, 2025
f384f73
fix: Changing the translation to be consistent with the english one
HahaBill Jul 3, 2025
449a8c2
fix: more translations
HahaBill Jul 3, 2025
f8c04c9
fix: translations
HahaBill Jul 3, 2025
645b2fc
Merge branch 'RooCodeInc:main' into feat/finer-grained-control-gemini
HahaBill Jul 3, 2025
7e5a59d
fix: removing contextLimit and token management related code
HahaBill Jul 7, 2025
edb96c6
fix: removing `contextLimit` test and removing token management in tr…
HahaBill Jul 7, 2025
cae3de9
fix: changing from `Advanced Features` to `Tools` to be consistent wi…
HahaBill Jul 7, 2025
ae2e895
fix: adding `try-catch` block for `generateContentStream`
HahaBill Jul 8, 2025
a5f46b4
feat: Include citations + improved type safety
HahaBill Jul 8, 2025
bf01618
Merge branch 'RooCodeInc:main' into feat/finer-grained-control-gemini
HahaBill Jul 8, 2025
63c7b25
feat: adding citation for streams (generateContextStream)
HahaBill Jul 8, 2025
151601b
Merge branch 'RooCodeInc:main' into feat/finer-grained-control-gemini
HahaBill Jul 8, 2025
055bd79
fix: set default values for `topP`, `topK` and `maxOutputTokens`
HahaBill Jul 8, 2025
3ff4c1e
Merge branch 'RooCodeInc:main' into feat/finer-grained-control-gemini
HahaBill Jul 9, 2025
4200cff
fix: changing UI/UX according to the review/feedback from `daniel-lxs`
HahaBill Jul 9, 2025
0d72f08
fix: updating the `Gemini.spec.tsx` unit test
HahaBill Jul 9, 2025
d18b143
fix: more changes from the feedback/review from `daniel-lxs`
HahaBill Jul 9, 2025
22eb360
Merge branch 'RooCodeInc:main' into feat/finer-grained-control-gemini
HahaBill Jul 14, 2025
7e9d252
fix: adding sources at the end of the stream to preserve
HahaBill Jul 14, 2025
ae0a3b7
Merge branch 'RooCodeInc:main' into feat/finer-grained-control-gemini
HahaBill Jul 15, 2025
8d48fcc
Merge branch 'RooCodeInc:main' into feat/finer-grained-control-gemini
HahaBill Jul 16, 2025
d1386a5
Merge branch 'RooCodeInc:main' into feat/finer-grained-control-gemini
HahaBill Jul 17, 2025
8ca442e
fix: change the description for grounding with google search and url …
HahaBill Jul 17, 2025
a853329
Merge branch 'feat/finer-grained-control-gemini' of https://github.co…
HahaBill Jul 17, 2025
1c2aa36
fix: adding translations
HahaBill Jul 17, 2025
88a7eb4
fix: removing redundant extra translations - a mistake made by the agent
HahaBill Jul 18, 2025
847756c
fix: remove duplicate translation keys in geminiSections and geminiPa…
roomote-agent Jul 18, 2025
3b8c801
fix: delete topK, topP and maxOutputTokens from Gemini
HahaBill Jul 19, 2025
728aded
fix: deleting topK, topP and maxOutputTokens from translations/locales
HahaBill Jul 19, 2025
9832b51
fix: adjust spacing between labels and descriptions + sentence casing
HahaBill Jul 19, 2025
abf3f1d
Merge branch 'RooCodeInc:main' into feat/adding-gemini-tools
HahaBill Jul 20, 2025
a6e8408
Merge branch 'RooCodeInc:main' into feat/adding-gemini-tools
HahaBill Jul 21, 2025
49a66d8
Merge remote-tracking branch 'origin/main' into feat/adding-gemini-tools
mrubens Jul 23, 2025
3ab7f13
Merge branch 'RooCodeInc:main' into feat/adding-gemini-tools
HahaBill Jul 23, 2025
b7b78df
Merge branch 'RooCodeInc:main' into feat/adding-gemini-tools
HahaBill Jul 23, 2025
5f63d15
Merge branch 'RooCodeInc:main' into feat/adding-gemini-tools
HahaBill Jul 24, 2025
c537802
Merge branch 'RooCodeInc:main' into feat/adding-gemini-tools
HahaBill Jul 25, 2025
8f21918
fix: adding maxOutputTokens back and removing unknown type
HahaBill Jul 25, 2025
5f8d0c2
fix: internalizing error Gemini error message
HahaBill Jul 25, 2025
5876976
fix: updating tests in Gemini and Vertex to adjust to the new error l…
HahaBill Jul 27, 2025
8fad431
Merge branch 'RooCodeInc:main' into feat/adding-gemini-tools
HahaBill Jul 27, 2025
816f634
fix: address PR review feedback for Gemini tools feature
daniel-lxs Jul 27, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions packages/types/src/provider-settings.ts
Original file line number Diff line number Diff line change
Expand Up @@ -169,6 +169,8 @@ const lmStudioSchema = baseProviderSettingsSchema.extend({
const geminiSchema = apiModelIdProviderModelSchema.extend({
geminiApiKey: z.string().optional(),
googleGeminiBaseUrl: z.string().optional(),
enableUrlContext: z.boolean().optional(),
enableGrounding: z.boolean().optional(),
})

const geminiCliSchema = apiModelIdProviderModelSchema.extend({
Expand Down
137 changes: 137 additions & 0 deletions src/api/providers/__tests__/gemini-handler.spec.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,137 @@
import { describe, it, expect, vi } from "vitest"
import { t } from "i18next"
import { GeminiHandler } from "../gemini"
import type { ApiHandlerOptions } from "../../../shared/api"

describe("GeminiHandler backend support", () => {
it("passes tools for URL context and grounding in config", async () => {
const options = {
apiProvider: "gemini",
enableUrlContext: true,
enableGrounding: true,
} as ApiHandlerOptions
const handler = new GeminiHandler(options)
const stub = vi.fn().mockReturnValue((async function* () {})())
// @ts-ignore access private client
handler["client"].models.generateContentStream = stub
await handler.createMessage("instr", [] as any).next()
const config = stub.mock.calls[0][0].config
expect(config.tools).toEqual([{ urlContext: {} }, { googleSearch: {} }])
})

it("completePrompt passes config overrides without tools when URL context and grounding disabled", async () => {
const options = {
apiProvider: "gemini",
enableUrlContext: false,
enableGrounding: false,
} as ApiHandlerOptions
const handler = new GeminiHandler(options)
const stub = vi.fn().mockResolvedValue({ text: "ok" })
// @ts-ignore access private client
handler["client"].models.generateContent = stub
const res = await handler.completePrompt("hi")
expect(res).toBe("ok")
const promptConfig = stub.mock.calls[0][0].config
expect(promptConfig.tools).toBeUndefined()
})

describe("error scenarios", () => {
it("should handle grounding metadata extraction failure gracefully", async () => {
const options = {
apiProvider: "gemini",
enableGrounding: true,
} as ApiHandlerOptions
const handler = new GeminiHandler(options)

const mockStream = async function* () {
yield {
candidates: [
{
groundingMetadata: {
// Invalid structure - missing groundingChunks
},
content: { parts: [{ text: "test response" }] },
},
],
usageMetadata: { promptTokenCount: 10, candidatesTokenCount: 5 },
}
}

const stub = vi.fn().mockReturnValue(mockStream())
// @ts-ignore access private client
handler["client"].models.generateContentStream = stub

const messages = []
for await (const chunk of handler.createMessage("test", [] as any)) {
messages.push(chunk)
}

// Should still return the main content without sources
expect(messages.some((m) => m.type === "text" && m.text === "test response")).toBe(true)
expect(messages.some((m) => m.type === "text" && m.text?.includes("Sources:"))).toBe(false)
})

it("should handle malformed grounding metadata", async () => {
const options = {
apiProvider: "gemini",
enableGrounding: true,
} as ApiHandlerOptions
const handler = new GeminiHandler(options)

const mockStream = async function* () {
yield {
candidates: [
{
groundingMetadata: {
groundingChunks: [
{ web: null }, // Missing URI
{ web: { uri: "https://example.com" } }, // Valid
{}, // Missing web property entirely
],
},
content: { parts: [{ text: "test response" }] },
},
],
usageMetadata: { promptTokenCount: 10, candidatesTokenCount: 5 },
}
}

const stub = vi.fn().mockReturnValue(mockStream())
// @ts-ignore access private client
handler["client"].models.generateContentStream = stub

const messages = []
for await (const chunk of handler.createMessage("test", [] as any)) {
messages.push(chunk)
}

// Should only include valid citations
const sourceMessage = messages.find((m) => m.type === "text" && m.text?.includes("[2]"))
expect(sourceMessage).toBeDefined()
if (sourceMessage && "text" in sourceMessage) {
expect(sourceMessage.text).toContain("https://example.com")
expect(sourceMessage.text).not.toContain("[1]")
expect(sourceMessage.text).not.toContain("[3]")
}
})

it("should handle API errors when tools are enabled", async () => {
const options = {
apiProvider: "gemini",
enableUrlContext: true,
enableGrounding: true,
} as ApiHandlerOptions
const handler = new GeminiHandler(options)

const mockError = new Error("API rate limit exceeded")
const stub = vi.fn().mockRejectedValue(mockError)
// @ts-ignore access private client
handler["client"].models.generateContentStream = stub

await expect(async () => {
const generator = handler.createMessage("test", [] as any)
await generator.next()
}).rejects.toThrow(t("common:errors.gemini.generate_stream", { error: "API rate limit exceeded" }))
})
})
})
3 changes: 2 additions & 1 deletion src/api/providers/__tests__/gemini.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ import { Anthropic } from "@anthropic-ai/sdk"

import { type ModelInfo, geminiDefaultModelId } from "@roo-code/types"

import { t } from "i18next"
import { GeminiHandler } from "../gemini"

const GEMINI_20_FLASH_THINKING_NAME = "gemini-2.0-flash-thinking-exp-1219"
Expand Down Expand Up @@ -129,7 +130,7 @@ describe("GeminiHandler", () => {
;(handler["client"].models.generateContent as any).mockRejectedValue(mockError)

await expect(handler.completePrompt("Test prompt")).rejects.toThrow(
"Gemini completion error: Gemini API error",
t("common:errors.gemini.generate_complete_prompt", { error: "Gemini API error" }),
)
})

Expand Down
3 changes: 2 additions & 1 deletion src/api/providers/__tests__/vertex.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ import { Anthropic } from "@anthropic-ai/sdk"

import { ApiStreamChunk } from "../../transform/stream"

import { t } from "i18next"
import { VertexHandler } from "../vertex"

describe("VertexHandler", () => {
Expand Down Expand Up @@ -105,7 +106,7 @@ describe("VertexHandler", () => {
;(handler["client"].models.generateContent as any).mockRejectedValue(mockError)

await expect(handler.completePrompt("Test prompt")).rejects.toThrow(
"Gemini completion error: Vertex API error",
t("common:errors.gemini.generate_complete_prompt", { error: "Vertex API error" }),
)
})

Expand Down
Loading