Skip to content

Conversation

@miguelangelmorenochacon
Copy link

@miguelangelmorenochacon miguelangelmorenochacon commented Oct 27, 2025

Problem

The labels parameter was not being forwarded to Vertex AI API requests, preventing proper tracking of Google partnership labels for billing and analytics purposes.

This issue prevented users from passing custom labels through the LiteLLM proxy to Vertex AI, which is important for resource tracking and billing attribution in Google Cloud environments.

Solution

This PR adds support for forwarding the labels parameter to Vertex AI API requests:

  1. Type Definition: Added labels field to the RequestBody TypedDict in litellm/types/llms/vertex_ai.py

  2. Request Transformation: Modified _transform_request_body in litellm/llms/vertex_ai/gemini/transformation.py to:

    • Extract labels from optional parameters
    • Support both direct labels parameter and conversion from OpenAI-style metadata.requester_metadata
    • Only add labels to Vertex AI endpoints (excluded from Google GenAI/AI Studio endpoints)
    • Properly include labels in the request body sent to the API

Testing

The changes have been tested locally with Docker, verifying that:

  • Labels are correctly included in POST requests to Vertex AI
  • Labels are properly formatted: "labels": {"goog-sample": "test-value"}
  • Labels work with both direct parameter and metadata conversion
  • Labels are excluded from Google GenAI endpoints as expected

Usage Example

Configuration

model_list:
  - model_name: gemini-flash
    litellm_params:
      model: vertex_ai/gemini-2.5-flash
      vertex_project: "your-vertex-project-id"
      vertex_location: "your-vertex-location"
      labels:
        goog-partner: your-value
      allowed_openai_params: ["labels"]

litellm_settings:
  drop_params: True
  set_verbose: True
  json_logs: True

general_settings:
  master_key: "sk-1234"

Expected Output

When using this configuration, labels are correctly forwarded to Vertex AI:

POST Request Sent from LiteLLM:
curl -X POST \
https://us-east1-aiplatform.googleapis.com/v1/projects/your-project/locations/your-location/publishers/google/models/gemini-2.5-flash:generateContent \
-H 'Content-Type: application/json' -H 'Authorization: Bearer ***' \
-d '{"contents": [{"role": "user", "parts": [{"text": "test"}]}], "generationConfig": {}, "labels": {"goog-partner": "your-value"}}'

Note: The labels parameter is now successfully included in the request body.

Changes

Modified Files

  • litellm/types/llms/vertex_ai.py - Added labels field to RequestBody type
  • litellm/llms/vertex_ai/gemini/transformation.py - Added labels extraction and forwarding logic

Related Issues

Fixes #13692

Additional Context

This implementation follows Google Cloud's labels specification and ensures backward compatibility by:

  • Making labels optional
  • Only applying labels to Vertex AI endpoints (not Google GenAI)
  • Supporting both direct labels parameter and metadata conversion

@vercel
Copy link

vercel bot commented Oct 27, 2025

@miguelangelmorenochacon is attempting to deploy a commit to the CLERKIEAI Team on Vercel.

A member of the Team first needs to authorize it.

@CLAassistant
Copy link

CLAassistant commented Oct 27, 2025

CLA assistant check
All committers have signed the CLA.

This fix addresses the issue where the 'labels' parameter was not being
forwarded to Vertex AI API requests, preventing Google partnership labels
from being properly tracked.

Changes:
- Added 'labels' field to RequestBody TypedDict in vertex_ai.py
- Updated _transform_request_body to extract and pass labels to API
- Labels are now correctly included in the request body sent to Vertex AI

Fixes BerriAI#13692
@miguelangelmorenochacon miguelangelmorenochacon force-pushed the fix/vertex-ai-labels-forwarding-upstream branch from 40063df to 8781321 Compare October 27, 2025 16:50
@miguelangelmorenochacon miguelangelmorenochacon marked this pull request as draft October 27, 2025 17:00
Trigger CI to verify all checks with latest changes
@miguelangelmorenochacon miguelangelmorenochacon deleted the fix/vertex-ai-labels-forwarding-upstream branch October 28, 2025 15:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: [LITELLM Proxy] Support for passing labels parameter to Vertex AI models

2 participants