Support Reasoning via Deepseek and Openrouter API (OpenAI Compatible)

[gururise]

The Thinking indicator never shows up if using the Deepseek API or OpenRouter (via OpenAI compatibility):

Config:

  {
    "name": "deepseek-ai/DeepSeek-R1",
    "modelUrl": "https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-32B",
    "websiteUrl": "https://deepseek.com/",
    "logoUrl": "https://huggingface.co/datasets/huggingchat/models-logo/resolve/main/deepseek-logo.png",
    "description": "The first reasoning model from DeepSeek. Outperforms OpenAI GPT-4-o1 on multiple benchmarks.",
    "reasoning": {
      "type": "tokens",
      "beginToken": "<think>",
      "endToken": "</think>"
    },
    "promptExamples": [
      {
        "title": "Rs in strawberry",
        "prompt": "how many R in strawberry?"
      },
      {
        "title": "Larger number",
        "prompt": "9.11 or 9.9 which number is larger?"
      },
      {
        "title": "Measuring 6 liters",
        "prompt": "I have a 6- and a 12-liter jug. I want to measure exactly 6 liters."
      }
    ],
    "endpoints": [
      {
        "type": "openai",
        "baseURL": "https://openrouter.ai/api/v1",
        "apiKey": "<API KEY REMOVED>"
      }
    ]
  },

The problem is DeekSeek API returns the reasoning in a separate reasoning_content field in the response.

reasoning_content = response.choices[0].message.reasoning_content
content = response.choices[0].message.content

As far as I can tell, none of the three existing methods will work to read this extra field in the response.

• tokens
• summary
• regex

Example streaming response:

from openai import OpenAI
client = OpenAI(api_key="<DeepSeek API Key>", base_url="https://api.deepseek.com")

# Round 1
messages = [{"role": "user", "content": "9.11 and 9.8, which is greater?"}]
response = client.chat.completions.create(
    model="deepseek-reasoner",
    messages=messages,
    stream=True
)

reasoning_content = ""
content = ""

for chunk in response:
    if chunk.choices[0].delta.reasoning_content:
        reasoning_content += chunk.choices[0].delta.reasoning_content # This is where the reasoning content is
    else:
        content += chunk.choices[0].delta.content

Please add support for DeepSeek API and OpenRouter API by adding an additional parameter to parse an extra field in the response for reasoning.

[nsarrazin]

Hi thanks for opening this! Yeah this is not ideal since the way we currently handle reasoning is directly on the token stream (find begin/end tokens, things like that) which is endpoint agonistic but this would require endpoint specific-code.

Not yet sure what the best way to tackle this is, I'm open to suggestions!

[gururise]

Looks like OpenRouter is pushing a new Reasoning Model Standard:

By including the "include_reasoning" parameter to send back reasoning tokens. Its still unclear if its part of the token stream or a separate field in the response.

[gururise]

After playing around with the include_reasoning parameter in OpenRouter, here's how it works:

The request (via OpenAI SDK):

completion = client.chat.completions.create(
  extra_headers={
    "HTTP-Referer": "<YOUR_SITE_URL>", # Optional. Site URL for rankings on openrouter.ai.
    "X-Title": "<YOUR_SITE_NAME>", # Optional. Site title for rankings on openrouter.ai.
  },
  model="deepseek/deepseek-r1",
  messages=[
    {
      "role": "user",
      "content": "What is the meaning of life?",
    }
  ],
  extra_body={
        "include_reasoning": True  # Include the parameter for downstream processing
    }
)

completion.choices[0]:

Choice(finish_reason='stop', index=0, logprobs=None, message=ChatCompletionMessage(content="<MODEL CONTENT HERE>", refusal=None, role='assistant', audio=None, function_call=None, tool_calls=None, reasoning="<REASONING HERE>"), native_finish_reason='stop')

The response is returned in the reasoning field.

Since this is intended to be a new standard (and is deployed across all the Open Router reasoning models), it would be nice if HuggingChat would support this.

[kentastickal]

@gururise any idea how to get reasoning back in using streaming? It's not compatible with Open AI's SDK at the very least :/
'reasoning' to be specific

[gururise]

@gururise any idea how to get reasoning back in using streaming? It's not compatible with Open AI's SDK at the very least :/ 'reasoning' to be specific

Check my code snippet above. Seems to work on OpenAI SDK, at least for non-streaming. I haven't tested w/streaming.