feat: add Gemini agentic video adapter by groupthinking · Pull Request #1608 · groupthinking/EventRelay

groupthinking · 2026-09-02T07:45:19Z

What changed

Adds an opt-in adapter for Google's agentic video understanding path via the Interactions API.

accepts YouTube URLs and file URIs
supports agentic and static processing per video, including mixed requests
defaults to gemini-3.7-flash
returns a durable execution receipt with model, sources, processing modes, text, and token count
validates inputs before a provider call

Why

Google AI Studio announced agentic video understanding on September 1, 2026. The capability actively inspects transcript, frames, and audio with adaptive frame sampling, which maps directly to EventRelay's video-to-event pipeline and the See → Act → Record → Review loop.

Sources:

Verification

python -m compileall src/integration/gemini_agentic_video.py tests/unit/test_gemini_agentic_video.py — passed
dependency-free fake-provider harness — passed request construction, async invocation, receipt, and validation
full pytest was not available in the local execution environment (No module named pytest); repository CI remains the merge gate

No production deployment or default-path switch is included.

vercel · 2026-09-02T07:45:26Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
v0-uvai	Error		Sep 8, 2026 3:27am UTC

coderabbitai · 2026-09-02T07:45:27Z

Important

🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Advanced

Run ID: 6486f772-77d9-43b1-8ca2-76029a898a21

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

github-actions · 2026-09-02T07:45:31Z

Dependency Review

✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.

Snapshot Warnings

⚠️: No snapshots were found for the head SHA 01c0b41.

Ensure that dependencies are being submitted on PR branches and consider enabling retry-on-snapshot-warnings. See the documentation for more information and troubleshooting advice.

Scanned Files

None

groupthinking · 2026-09-02T08:47:59Z

Execution receipt — security workflow diagnosis

Verified failing run: Security Scan 33605167703, job trivy (100167379816).

The container build stops before Trivy executes because:

npm ci --workspace=apps/web --production --legacy-peer-deps
Invalid: lock file's @ai-sdk/gateway@4.0.23 does not satisfy @ai-sdk/gateway@4.0.72
Invalid: lock file's postcss@8.5.21 does not satisfy postcss@8.5.26
ERROR: process ... did not complete successfully: exit code: 1

The PR changes only:

src/integration/gemini_agentic_video.py
tests/unit/test_gemini_agentic_video.py

Interpretation: the failed security workflow is a repository package/lock synchronization blocker, not a vulnerability finding from this adapter. Trivy was skipped because the image never built.

Action: recorded here rather than mixing an unrelated lockfile rewrite into this focused PR. The next safe repair is to reconcile the web workspace lockfile in its canonical dependency-maintenance work, then rerun Security Scan.

Other exact-head checks currently verified successful: CI, Coverage, CodeQL, Secret Scan, and Dependency Review.

groupthinking · 2026-09-03T07:02:26Z

Execution receipt — Gemini 3.8 adoption boundary and benchmark scope

New verified fact (2026-09-02): Google released Gemini 3.8 Flash, and Vercel AI Gateway exposes it as google/gemini-3.8-flash with video input. Main now uses that gateway path in 25e6fbb.

Separate verified fact: Google's current agentic-video guide still documents processing: "agentic" only for Gemini 3.7 Flash, 3.6 Flash, and 3.5 Flash-Lite. It does not document Gemini 3.8 for this specialized Interactions API mode.

Sources:

Decision: TEST. Keep this draft's default at gemini-3.7-flash until Google explicitly confirms 3.8 support for processing: "agentic". Do not treat the generic gateway video path and the Interactions API agentic path as equivalent.

Re-scoped value of this PR: retain it as the explicit agentic-processing/receipt adapter and use it to benchmark against the main gateway extractor on the same videos:

Video Pack schema completeness and grounded timestamps
Token use and actual cost
Latency
Failure behavior
Provenance/receipt quality

The adapter already permits a model override through GEMINI_AGENTIC_VIDEO_MODEL, so no speculative model-pin change is needed today.

vercel

Additional Suggestion:

TypeScript build fails with "Variable 'promise' is used before being assigned" because promise is referenced inside its own initializer closure under strict definite-assignment analysis

Copilot

🟡 Changes recommended

The locked SDK version loses response text, and malformed video URIs still reach the provider.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Adds an opt-in Gemini Interactions API adapter for agentic and static video analysis.

Changes:

Adds video input and execution receipt models.
Supports mixed processing modes and asynchronous provider calls.
Tests request construction, receipts, and basic validation.

File summaries

File	Description
`src/integration/gemini_agentic_video.py`	Implements the Gemini video adapter.
`tests/unit/test_gemini_agentic_video.py`	Adds adapter unit tests.

Review details

Files reviewed: 2/2 changed files
Comments generated: 2
Review effort level: Balanced

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

+        usage = getattr(response, "usage", None)
+        total_tokens = getattr(usage, "total_tokens", None)
+        return AgenticVideoReceipt(
+            output_text=str(getattr(response, "output_text", "")),


+        for video in videos:
+            if not video.uri.strip():
+                raise ValueError("Video URI must not be empty")
+            item = {
+                "type": "video",
+                "uri": video.uri,


github-actions · 2026-09-08T13:29:54Z

⚠️ Repository reconciliation: this PR does not reference exactly one canonical issue.

Please add a Closes #<issue> reference in the PR description so it can be tracked against the delivery plan.

See governance: #898

groupthinking added 2 commits September 2, 2026 02:45

feat: add Gemini agentic video adapter

3f9cf5e

test: cover Gemini agentic video adapter

18656dc

github-actions Bot added the python label Sep 2, 2026

groupthinking marked this pull request as ready for review September 8, 2026 03:26

Copilot AI balanced review requested due to automatic review settings September 8, 2026 03:26

Merge branch 'main' into feat/gemini-agentic-video

01c0b41

Copilot started reviewing on behalf of groupthinking September 8, 2026 03:26 View session

vercel Bot had a problem deploying to Preview September 8, 2026 03:27 Failure

vercel Bot reviewed Sep 8, 2026

View reviewed changes

Copilot AI reviewed Sep 8, 2026

View reviewed changes

github-actions Bot mentioned this pull request Sep 8, 2026

[automation] Repository drift report #1584

Open

groupthinking linked an issue Sep 8, 2026 that may be closed by this pull request

Agent help #1679

Closed

Navigation Menu

feat: add Gemini agentic video adapter - #1608

feat: add Gemini agentic video adapter#1608
groupthinking wants to merge 3 commits into
mainfrom
feat/gemini-agentic-video

groupthinking commented Sep 2, 2026

Uh oh!

vercel Bot commented Sep 2, 2026 •

edited

Loading

Uh oh!

coderabbitai Bot commented Sep 2, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Sep 2, 2026 •

edited

Loading

Uh oh!

groupthinking commented Sep 2, 2026

Uh oh!

groupthinking commented Sep 3, 2026

Uh oh!

vercel Bot left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

github-actions Bot commented Sep 8, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

groupthinking commented Sep 2, 2026

What changed

Why

Verification

Uh oh!

vercel Bot commented Sep 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

coderabbitai Bot commented Sep 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Sep 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Dependency Review

Snapshot Warnings

Scanned Files

Uh oh!

groupthinking commented Sep 2, 2026

Execution receipt — security workflow diagnosis

Uh oh!

groupthinking commented Sep 3, 2026

Execution receipt — Gemini 3.8 adoption boundary and benchmark scope

Uh oh!

vercel Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

github-actions Bot commented Sep 8, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

vercel Bot commented Sep 2, 2026 •

edited

Loading

coderabbitai Bot commented Sep 2, 2026 •

edited

Loading

github-actions Bot commented Sep 2, 2026 •

edited

Loading