[Frontend] Align tool_choice="required" behavior with OpenAI when tools is empty

[n0gu-furiosa]

Purpose

When a user specifies tool_choice="required" with an empty tools, the OpenAI-compatible server does not currently handle this scenario in the most appropriate way. The response returned is as follows:

$ curl http://localhost:8000/v1/chat/completions -H 'Content-type: application/json' -d '
{
    "model": "meta-llama/Llama-3.1-8B-Instruct",
    "messages": [{"role": "user", "content": ""}],
    "tool_choice": "required",
    "tools": []
}'
{"object":"error","message":"Grammar error: Unsatisfiable schema: required item is unsatisfiable","type":"BadRequestError","param":null,"code":400}

This BadRequestError is not raised by the model validator but by the guided decoding backend, because the request generates an unsatisfiable JSON schema:

{"type":"array","minItems":1,"items":{"type":"object","anyOf":[]}

Although this behavior technically blocks invalid requests, it is not the most user-friendly. The issue becomes more problematic when stream=True: in this case, the request is not validated before upgrading to SSE. Instead, the error surfaces after the model has begun generating results:

$ curl http://localhost:8000/v1/chat/completions -H 'Content-type: application/json' -d '
{
    "model": "meta-llama/Llama-3.1-8B-Instruct",
    "messages": [{"role": "user", "content": ""}],
    "tool_choice": "required",
    "tools": [],
    "stream": true
}'
data: {"error": {"object": "error", "message": "Grammar error: Unsatisfiable schema: required item is unsatisfiable", "type": "BadRequestError", "param": null, "code": 400}}

data: [DONE]

This PR adds explicit validation of tool length when tool_choice="required".

Test Plan

Manually ran vllm serve meta-llama/Llama-3.1-8B-Instruct --enable-auto-tool-choice --tool-call-parser llama3_json and executed the curl command above.

Test Result

$ curl http://localhost:8000/v1/chat/completions -H 'Content-type: application/json' -d '
{
    "model": "meta-llama/Llama-3.1-8B-Instruct",
    "messages": [{"role": "user", "content": ""}],
    "tool_choice": "required",
    "tools": [],
    "stream": true
}'
{"object":"error","message":"[{'type': 'value_error', 'loc': ('body',), 'msg': 'Value error, When using `tool_choice` as \"required\", `tools` must contain at least one tool', 'input': {'model': 'meta-llama/Llama-3.1-8B-Instruct', 'messages': [{'role': 'user', 'content': ''}], 'tool_choice': 'required', 'tools': [], 'stream': True}, 'ctx': {'error': ValueError('When using `tool_choice` as \"required\", `tools` must contain at least one tool')}}]","type":"Bad Request","param":null,"code":400}

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

[gemini-code-assist]

Code Review

This pull request introduces an explicit validation check for tool_choice in the OpenAI server implementation. Specifically, it ensures that if tool_choice is set to "required", the tools list cannot be empty. This change prevents a downstream error from the guided decoding backend and provides a much clearer error message to the user, which is a significant improvement, especially for streaming responses. The implementation is correct, robust, and placed appropriately within the Pydantic model validator. I find no issues with the proposed change.

[chaunceyjiang]

Could you help verify how the OpenAI online service handles this situation?

[n0gu-furiosa]

@chaunceyjiang I checked, and it turns out that the OpenAI API behaves as if tool_choice were "none" when tools is an empty list (or "auto", I believe they behave the same way in this context).

Request payload:

{
    "model": "gpt-4o",
    "messages": [
        {
            "role": "user",
            "content": "How is the weather today in Seoul?"
        }
    ],
    "stream": false,
    "tool_choice": "required",
    "tools": []
}

Response body:

    "choices": [
        {
            "index": 0,
            "message": {
                "role": "assistant",
                "content": "I currently don't have real-time weather data. You might want to check a reliable weather website or use a weather app for the most current information in Seoul.",
                "refusal": null,
                "annotations": []
            },
            "logprobs": null,
            "finish_reason": "stop"
        }
    ],

To align with this behavior, I think the simplest approach would be to override tool_choice to "none" if tool_choice is "required" and tools is an empty list, since much of the code assumes that tool_choice "required" must return tool calls. If you agree, I can update the PR this way. Do you have any thoughts or concerns?

[chaunceyjiang]

Sounds good.
/cc @aarnphm

[n0gu-furiosa]

@chaunceyjiang Applied in 781573d. Also changed the title of this PR.

Updated test result:

$ curl http://localhost:8000/v1/chat/completions -H 'Content-type: application/json' -d '
{
    "model": "meta-llama/Llama-3.1-8B-Instruct",
    "messages": [{"role": "user", "content": "How is the weather today in Seoul?"}],
    "max_completion_tokens": 20,
    "tool_choice": "required",
    "tools": []
}'

{
  "id": "chatcmpl-40d600203d804754a8d4c58d4cae5c8b",
  "object": "chat.completion",
  "created": 1752726568,
  "model": "meta-llama/Llama-3.1-8B-Instruct",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "I'm not able to access real-time information. However, I can suggest some ways to find the",
        "refusal": null,
        "annotations": null,
        "audio": null,
        "function_call": null,
        "tool_calls": [],
        "reasoning_content": null
      },
...

[n0gu-furiosa]

Hi @chaunceyjiang, just a friendly reminder to take a look at this when you have a chance. Thanks in advance.

[chaunceyjiang]

Thanks~

LGTM.

/cc @DarkLight1337 @aarnphm