Skip to content

docs(decisions): document input_json parts on /v1/decisions - #3148

Merged
mateo-berri merged 10 commits into
BerriAI:mainfrom
zoveki:litellm_decisions_input_json
Nov 20, 2026
Merged

mateo-berri merged 10 commits into
BerriAI:mainfrom
zoveki:litellm_decisions_input_json

Conversation

@zoveki

@zoveki zoveki commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

Summary

This documents the input_json content part on /v1/decisions, which LiteLLM #52805 adds for OpenAI and LiteLLM #52807 ports to the 1.108.x LTS line. The request fields table, the features table and the availability table now list it. A new JSON input section covers the 400 for developer and system messages, the drop_params behavior, the hide-secrets check of the keys and string values inside data, with a numbered name for a redacted key that would clash with another, how LLM Shield swaps the keys, strings and numbers inside data, the 400 Cisco AI Defense answers when it would redact a JSON request, and how a load-balanced group routes JSON requests, with fallbacks and cooldowns. The cooldown case answers a 429 on the LTS line, so the page says so. With drop_params on, a deployment without JSON support can answer from the rest of the input, so the page suggests a model group of its own for JSON deployments. The hide-secrets and LLM Shield pages now link to the JSON input section

The Supported column names OpenAI only, since no Azure Decisions model answers a call yet and azure/ has no row in the supported providers list. The Notes column says azure/ deployments send the parts on, which a unit test in #52805 checks

The Gemini row now names v1.111.0-rc.1, the release first_release.py 51220 gives for Gemini Decision 1, instead of saying it is not in a release yet. The Gemini provider page's badge names v1.111.0 and its sentence says first in v1.111.0-rc.1, as CONTRIBUTING.md asks

The JSON row names v1.112.0-rc.1 and v1.108.14, the first releases first_release.py 52805 and first_release.py 52807 give. v1.108.13 shipped on Nov 19 without #52807. This PR stays a draft until #52805 merges, and then the names and the behavior text are checked again against the merged code

Proof

E2E exception at c2e3052. The diff is docs only, so there is no before leg. check-docs.py passes at c2e3052 against the merge base ada2e67, and the JSON input section's curl example ran against a proxy at #52805's d85536a675e. Against today's main the checker also flags docs/proxy/lens/deployment/docker.md, a page main removed in #3190, which this PR does not touch

$ uv run --no-project --with pyyaml python3 scripts/check-docs.py docs --base=ada2e67045
Checked 1047 markdown files in: docs
No structural problems found.

The curl example returns OpenAI's decision through a proxy at #52805's d85536a675e. The run changed only the URL to a tunnel to the QA proxy and the model to that proxy's OpenAI gpt-6-luna group

Command and output
$ curl -s -D hdr.txt -w '\nHTTP %{http_code}\n' -X POST 'http://127.0.0.1:28471/v1/decisions' \
-H "Authorization: Bearer $LITELLM_API_KEY" \
-H 'Content-Type: application/json' \
-d '{
  "model": "litellm-qa-decisions",
  "input": [
    {
      "type": "message",
      "role": "user",
      "content": [
        {"type": "input_text", "text": "Evaluate the ticket using the order evidence."},
        {"type": "input_json", "data": {"ticket": {"body": "Please refund the duplicate charge."}, "order": {"charges": [49, 49]}}}
      ]
    }
  ],
  "questions": [
    {"type": "predicate", "name": "duplicate_charge", "instructions": "Does the ticket describe a duplicate charge supported by the order?"}
  ]
}'
{"model":"gpt-6-luna","answers":[{"type":"predicate","name":"duplicate_charge","probability":1.0}],"usage":{"input_tokens":189,"input_tokens_details":{"cached_tokens":0,"cache_write_tokens":0},"output_tokens":0,"output_tokens_details":{"reasoning_tokens":0},"total_tokens":189}}
HTTP 200
$ grep -i -E '^x-litellm-(model-api-base|response-cost):' hdr.txt
x-litellm-model-api-base: https://api.openai.com
x-litellm-response-cost: 1.89e-05

The 400 that names input_json, the drop_params drop, the mixed-group routing, the fallback, the hide-secrets redaction with its numbered keys and the LLM Shield swaps of keys and numbers the page describes are each a live cell in the proof of LiteLLM #52805 at d85536a675e, and of LiteLLM #52807 at 3e5f3d895d6 for the LTS line

Caveats (if any)

Medium

  • The JSON row's release names hold only if #52805 merges before the rc/1.112.0 cut on Mon, Nov 23 at 3am PT and #52807 merges before v1.108.14 is tagged. first_release.py checks both again right before this PR merges

Low

  • The cooldown lines, a 400 on main and the router's 429 on the 1.108.x LTS line, come from reading both routers, since both code PRs' live runs turn cooldowns off

@vercel

vercel Bot commented Nov 19, 2026

Copy link
Copy Markdown

@mateo-berri is attempting to deploy a commit to the Berri AI Team on Vercel.

A member of the Team first needs to authorize it.

@vercel

vercel Bot commented Nov 19, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
litellm Ready Ready Preview Nov 20, 2026 7:35pm UTC

Request Review

… messages, fallbacks and cooldowns, and date the Gemini page
@mateo-berri
mateo-berri marked this pull request as draft November 20, 2026 05:21
@zoveki
zoveki marked this pull request as ready for review November 20, 2026 21:33

@mateo-berri mateo-berri left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@mateo-berri
mateo-berri merged commit 185ae0b into BerriAI:main Nov 20, 2026
6 checks passed

This branch was successfully deployed

1 active deployment
Preview — c2e3052f Deployed Nov 20, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants