Skip to content

Feat: Adding Gemini tools - URL Context and Grounding with Google Search #5959

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 51 commits into
base: main
Choose a base branch
from

Conversation

HahaBill
Copy link

@HahaBill HahaBill commented Jul 19, 2025

Related GitHub Issue

Related to this PR: #4895

Closes: #4519

Roo Code Task Context (Optional)

Description

Test Procedure

Pre-Submission Checklist

  • Issue Linked: This PR is linked to an approved GitHub Issue (see "Related GitHub Issue" above).
  • Scope: My changes are focused on the linked issue (one major feature/fix per PR).
  • Self-Review: I have performed a thorough self-review of my code.
  • Testing: New and/or updated tests have been added to cover my changes (if applicable).
  • Documentation Impact: I have considered if my changes require documentation updates (see "Documentation Updates" section below).
  • Contribution Guidelines: I have read and agree to the Contributor Guidelines.

Screenshots / Videos

Documentation Updates

Additional Notes

Get in Touch


Important

Adds URL context and grounding with Google Search features to the Gemini provider, including UI updates and tests.

  • Behavior:
    • Adds enableUrlContext and enableGrounding options to geminiSchema in provider-settings.ts.
    • Updates GeminiHandler in gemini.ts to support URL context and grounding with Google Search.
    • Adds error handling for grounding metadata extraction in gemini.ts.
  • UI Components:
    • Updates ApiOptions.tsx and Gemini.tsx to include checkboxes for enabling URL context and grounding.
    • Adds tests for UI components in Gemini.spec.tsx.
  • Tests:
    • Adds gemini-handler.spec.ts to test GeminiHandler with URL context and grounding configurations.
    • Includes tests for UI changes in Gemini.spec.tsx.
  • Localization:
    • Updates localization files to include new strings for URL context and grounding settings.

This description was created by Ellipsis for 728aded. You can customize this summary. It will automatically update as commits are pushed.

HahaBill added 30 commits June 19, 2025 19:27
- with topP, topK, maxOutputTokens
- allow users to enable URL context and Grounding Research
… + working with profile-specific thresholding
…iption and titles to settings for translation purposes
…ng will also change in Gemini context management

- sync between Context Management Settting <-> Gemini Context Management with regards to thresholding
@HahaBill HahaBill requested a review from jr as a code owner July 19, 2025 20:39
@HahaBill HahaBill marked this pull request as draft July 19, 2025 20:39
@dosubot dosubot bot added size:L This PR changes 100-499 lines, ignoring generated files. enhancement New feature or request labels Jul 19, 2025
@hannesrudolph hannesrudolph added the Issue/PR - Triage New issue. Needs quick review to confirm validity and assign labels. label Jul 19, 2025
@HahaBill HahaBill marked this pull request as ready for review July 19, 2025 21:58
@dosubot dosubot bot added the UI/UX UI/UX related or focused label Jul 19, 2025
"geminiParameters": {
"urlContext": {
"title": "URL संदर्भ सक्षम करें",
"description": "जब प्रतिक्रियाएं उत्पन्न करता है, अतिरिक्त संदर्भ के लिए Gemini को URL तक पहुंचने और संसाधित करने की अनुमति देता है। वेब सामग्री विश्लेषण वाली कार्यों के लिए उपयोगी।"
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Typo/grammatical note: In the description for "urlContext", the phrase "जब प्रतिक्रियाएं उत्पन्न करता है" has a number mismatch. Consider revising it (for example, to "जब प्रतिक्रिया उत्पन्न होती है") for better grammatical accuracy.

Suggested change
"description": "जब प्रतिक्रियाएं उत्पन्न करता है, अतिरिक्त संदर्भ के लिए Gemini को URL तक पहुंचने और संसाधित करने की अनुमति देता है। वेब सामग्री विश्लेषण वाली कार्यों के लिए उपयोगी।"
"description": "जब प्रतिक्रिया उत्पन्न होती है, अतिरिक्त संदर्भ के लिए Gemini को URL तक पहुंचने और संसाधित करने की अनुमति देता है। वेब सामग्री विश्लेषण वाली कार्यों के लिए उपयोगी।"

@HahaBill
Copy link
Author

@mrubens Hi, I created this PR for you and did the changes that you requested: adding more spacing between the label and the explanatory text.

Grounding with Google Search is the name of the tool in case you're wondering why it is not Enable grounding with Google search.

Screenshot 2025-07-19 at 23 06 36

@daniel-lxs daniel-lxs moved this from Triage to PR [Needs Review] in Roo Code Roadmap Jul 19, 2025
@daniel-lxs daniel-lxs moved this from PR [Needs Review] to PR [Needs Prelim Review] in Roo Code Roadmap Jul 19, 2025
@hannesrudolph hannesrudolph added PR - Needs Preliminary Review and removed Issue/PR - Triage New issue. Needs quick review to confirm validity and assign labels. labels Jul 19, 2025
@mrubens
Copy link
Collaborator

mrubens commented Jul 23, 2025

Hmm, there's something up with the checkboxes here - I need to check and uncheck a couple times to be able to save. Maybe we need to do a migration in providersettings?

@HahaBill
Copy link
Author

Hmm, there's something up with the checkboxes here - I need to check and uncheck a couple times to be able to save. Maybe we need to do a migration in providersettings?

Looking into this right now

Copy link
Collaborator

@daniel-lxs daniel-lxs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @HahaBill Thank you for the work on splitting you PR, I just have a couple of questions and suggestions.

Let me know what you think!

temperature: this.options.modelTemperature ?? 0,
...(tools.length > 0 ? { tools } : {}),
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I noticed that maxTokens is being passed to getModel() but isn't included in the config object. Should we add maxOutputTokens: this.options.modelMaxTokens ?? maxTokens ?? undefined to the config to ensure token limits are properly enforced?

}
const config = rawConfig as unknown as GenerateContentConfig
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a specific reason for the type casting as unknown as GenerateContentConfig? This pattern appears twice in the code. If the types don't align properly, would it be better to create a proper type mapping or update the type definitions to avoid potential runtime issues?

}
} catch (error) {
if (error instanceof Error) {
throw new Error(`Gemini Generate Context Stream error: ${error.message}`)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this error user-facing? if so, can we internationalize it?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we add tests for error scenarios? For example:

  • What happens when grounding metadata extraction fails?
  • How does the handler behave with malformed grounding metadata?
  • What if the API returns an error when tools are enabled?

@daniel-lxs daniel-lxs moved this from PR [Needs Prelim Review] to PR [Changes Requested] in Roo Code Roadmap Jul 23, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request PR - Changes Requested size:L This PR changes 100-499 lines, ignoring generated files. UI/UX UI/UX related or focused
Projects
Status: PR [Changes Requested]
Development

Successfully merging this pull request may close these issues.

Finer-grained control of Gemini models
5 participants