Skip to content

docs(gemini): document service_tier on Vertex AI as tier headers - #3205

Open
mika-lt wants to merge 2 commits into
BerriAI:mainfrom
mika-lt:litellm_gemini_vertex_service_tier
Open

mika-lt wants to merge 2 commits into
BerriAI:mainfrom
mika-lt:litellm_gemini_vertex_service_tier

Conversation

@mika-lt

@mika-lt mika-lt commented Nov 20, 2026

Copy link
Copy Markdown
Contributor

TLDR

Documents how service_tier reaches Vertex AI once BerriAI/litellm#34959 merges. That PR sends a Vertex tier as the X-Vertex-AI-LLM-Shared-Request-Type header, since Vertex answers 400 to the serviceTier body field

Changes

The Gemini page's service_tier section now has one table for what gemini/ and vertex_ai/ each send for every value. auto stays priority on the Gemini API and sends no tier on Vertex, and an unknown value such as scale answers 400 unless drop_params is set. The section also covers the Provisioned Throughput fallback of the header, the response service_tier that Vertex now reports from usageMetadata.trafficType, and Vertex batch rows

The Vertex PayGo tutorial and the Vertex page now point at service_tier, and their trafficType tables list ON_DEMAND_FLEX, which Vertex reports for Flex PayGo

Each new behavior carries <SinceVersion v="1.112.0" />, the first release planned to carry #34959. This PR should merge after #34959

Checks

uv run --no-project --with pyyaml python3 scripts/check-docs.py docs printed No structural problems found.

@vercel

vercel Bot commented Nov 20, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
litellm Ready Ready Preview Nov 21, 2026 12:03am UTC

Request Review

@vercel

vercel Bot commented Nov 20, 2026

Copy link
Copy Markdown

@mateo-berri is attempting to deploy a commit to the Berri AI Team on Vercel.

A member of the Team first needs to authorize it.

This branch was successfully deployed

1 active deployment
Preview — 5f24bb91 Deployed Nov 21, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants