Skip to content

feat(litellm): add team name and user name support to LiteLLM credential#6487

Open
raghava3107 wants to merge 3 commits into
FlowiseAI:mainfrom
raghava3107:feature/litellm-team-user-headers
Open

feat(litellm): add team name and user name support to LiteLLM credential#6487
raghava3107 wants to merge 3 commits into
FlowiseAI:mainfrom
raghava3107:feature/litellm-team-user-headers

Conversation

@raghava3107
Copy link
Copy Markdown

@raghava3107 raghava3107 commented Jun 6, 2026

Summary

Adds a Custom Headers field to the LiteLLM credential, enabling users to pass any LiteLLM proxy headers in API requests. Also adds a comprehensive test suite for the ChatLiteLLM node.

Why is this needed?

LiteLLM is an AI gateway/proxy that routes requests to 100+ LLM providers (OpenAI, Anthropic, Bedrock, etc.) through a unified API. Organizations use LiteLLM for:

  • Cost tracking — track spend per team, customer, or environment
  • Tag-based routing — route requests to specific model deployments based on tags
  • Rate limiting — enforce per-team or per-customer rate limits
  • Audit logging — track who made each request

LiteLLM supports several request headers for these features (docs):

Header Purpose
x-litellm-tags Tag-based routing and spend tracking (comma-separated)
x-litellm-customer-id End-user/customer tracking
x-litellm-end-user-id End-user identification
x-litellm-spend-logs-metadata Custom metadata in spend logs (JSON string)

Currently, Flowise's LiteLLM integration only supports an API key — there's no way to pass these headers to the proxy.

Solution

Credential (LitellmApi.credential.ts):

  • Added optional Custom Headers field (JSON string)
  • Example: {"x-litellm-tags": "team:activation,env:prod"}

Chat model (ChatLitellm.ts):

  • Parses the custom headers JSON from credential
  • Passes them as defaultHeaders in the OpenAI-compatible client configuration
  • Malformed JSON is silently ignored (no crash)

Tests (ChatLitellm.test.ts) — 6 tests:

  • ✅ Basic initialization with API key and model
  • ✅ Custom headers passed correctly to the client
  • ✅ Malformed JSON headers handled gracefully
  • ✅ Backward compatibility without headers
  • ✅ Works without basePath or custom headers
  • ✅ Optional parameters (maxTokens, topP, timeout)

Backward Compatibility

The Custom Headers field is optional. Existing LiteLLM configurations continue to work without any changes.

Files Changed

File Change
packages/components/credentials/LitellmApi.credential.ts Added litellmCustomHeaders field
packages/components/nodes/chatmodels/ChatLitellm/ChatLitellm.ts Parse and pass custom headers
packages/components/nodes/chatmodels/ChatLitellm/ChatLitellm.test.ts New — 6 unit tests

Test Results

PASS nodes/chatmodels/ChatLitellm/ChatLitellm.test.ts
  ChatLitellm
    ✓ initializes with basic config (API key and model only)
    ✓ passes custom headers from credential when provided
    ✓ ignores malformed custom headers JSON gracefully
    ✓ works without custom headers (backward compatible)
    ✓ works without basePath or custom headers
    ✓ passes optional parameters when provided

Test Suites: 1 passed, 1 total
Tests:       6 passed, 6 total

LiteLLM proxy supports team-based routing for billing and access control.
This adds optional Team Name and User Name fields to the LiteLLM credential,
and passes them as x-litellm-team and x-litellm-user headers in API requests.

This enables organizations using LiteLLM proxy with team-scoped API keys to
explicitly identify the team and user making requests, which is useful for:
- Team-based cost tracking and billing
- Per-team rate limiting
- Audit logging

Both fields are optional and backward-compatible — existing LiteLLM
configurations continue to work without changes.
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces optional 'Team Name' and 'User Name' fields to the LiteLLM API credentials. These fields are then extracted and passed as custom headers ('x-litellm-team' and 'x-litellm-user') in the configuration of the ChatLiteLLM model. No review comments were provided, and there is no feedback to address.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Replace team/user-specific fields with a generic Custom Headers field
that supports any LiteLLM proxy header (x-litellm-tags, x-litellm-customer-id,
x-litellm-end-user-id, x-litellm-spend-logs-metadata, etc.).

Headers are passed as a JSON string in the credential and sent as
defaultHeaders in the OpenAI-compatible client configuration.
Malformed JSON is handled gracefully.

Add comprehensive test suite covering:
- Basic initialization with API key and model
- Custom headers passed correctly to the client
- Malformed JSON headers handled gracefully
- Backward compatibility without headers
- Optional parameters (maxTokens, topP, timeout)
Ensure JSON.parse result is a non-array object before using as headers.
Rejects arrays, strings, and numbers that would parse but are invalid headers.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant