Support extra_body parameters #1069
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #1069 +/- ##
==========================================
- Coverage 99.59% 99.32% -0.27%
==========================================
Files 34 34
Lines 2206 2230 +24
==========================================
+ Hits 2197 2215 +18
- Misses 6 10 +4
- Partials 3 5 +2 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
|
anyone can review？I also need this pr |
|
I moved the ExtraBody field from ChatCompletionRequestExtensions into the ChatCompletionRequest struct. |
ArtdragonXoX
changed the title
|
anyone can review？I also need this pr |
feat(chat): fix ExtraBody embedding and add comprehensive tests
|
@sashabaranov Can you check this pr please? |
There was a problem hiding this comment.
Pull Request Overview
This PR adds support for extra_body parameters in chat completion requests, enabling integration with Gemini API's thinking mode and other extended features. The implementation allows extra fields to be merged directly into the request body while removing the extra_body key itself.
- Adds
ExtraBodyfield toChatCompletionRequeststruct with proper documentation - Modifies request serialization to merge extra body fields into the main request payload
- Includes comprehensive test coverage for the new functionality
Reviewed Changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| chat.go | Adds ExtraBody field to ChatCompletionRequest and implements request body merging logic |
| chat_test.go | Adds comprehensive test to verify ExtraBody fields are properly merged and extra_body key is removed |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
| // The body map is used to dynamically construct the request payload for the chat completion API. | ||
| // Instead of relying on a fixed struct, the body map allows for flexible inclusion of fields | ||
| // based on their presence, avoiding unnecessary or empty fields in the request. |
There was a problem hiding this comment.
The comment describes general benefits of using a map but doesn't explain the specific purpose of this code block, which is to handle ExtraBody field merging. Consider updating to: 'Create a dynamic request body by merging ExtraBody fields into the main request payload.'
| // The body map is used to dynamically construct the request payload for the chat completion API. | |
| // Instead of relying on a fixed struct, the body map allows for flexible inclusion of fields | |
| // based on their presence, avoiding unnecessary or empty fields in the request. | |
| // Create a dynamic request body by merging ExtraBody fields into the main request payload. | |
| // This approach allows flexible inclusion of additional fields, ensuring the request contains all necessary data. |
Copilot uses AI. Check for mistakes.
|
|
||
| // Deserialize JSON to map[string]any | ||
| var body map[string]any | ||
| _ = json.Unmarshal(jsonData, &body) |
There was a problem hiding this comment.
The json.Unmarshal error is being ignored with blank identifier. This could cause silent failures if the JSON unmarshaling fails. The error should be checked and returned.
| _ = json.Unmarshal(jsonData, &body) | |
| err = json.Unmarshal(jsonData, &body) | |
| if err != nil { | |
| return | |
| } |
Copilot uses AI. Check for mistakes.
| req, err := c.newRequest( | ||
| ctx, | ||
| http.MethodPost, | ||
| c.fullURL(urlSuffix, withModel(request.Model)), | ||
| withBody(request), | ||
| withBody(body), // Main request body. | ||
| withExtraBody(extraBody), // Merge ExtraBody fields. |
There was a problem hiding this comment.
I don't think you are using this correctly: withExtraBody merges extraBody into request body. However, in the example you've mentioned it goes as a separate field:
curl "https://generativelanguage.googleapis.com/v1beta/openai/chat/completions" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer GEMINI_API_KEY" \
-d '{
"model": "gemini-2.5-flash",
"messages": [{"role": "user", "content": "Explain to me how AI works"}],
"extra_body": {
"google": {
"thinking_config": {
"include_thoughts": true
}
}
}
}'
There was a problem hiding this comment.
Thank you for your inspection. I have seen it working correctly during testing and use. For example:
_, err := client.CreateChatCompletion(context.Background(), openai.ChatCompletionRequest{
Model: "doubao-seed-1-6-250615",
Messages: []openai.ChatCompletionMessage{
{
Role: openai.ChatMessageRoleUser,
Content: "Hello!",
},
},
ExtraBody: map[string]any{
"thinking":map[string]any {
"type": "disabled",
},
},
})
After the request is processed as the body, it will change to:
{
"model": "doubao-seed-1-6-250615",
"messages": [{"role": "user", "content": "Hello!"}],
"thinking": {
"type": "disabled"
}
}
For the gemini example, I'm sorry that I didn't give a clear example that caused your misunderstanding. In Gemini's official API, they have additional support for extra_body fields. Let me explain the example call in the documentation:
from openai import OpenAI
client = OpenAI(
api_key="GEMINI_API_KEY",
base_url="https://generativelanguage.googleapis.com/v1beta/openai/"
)
response = client.chat.completions.create(
model="gemini-2.5-flash",
messages=[{"role": "user", "content": "Explain to me how AI works"}],
extra_body={
'extra_body': {
"google": {
"thinking_config": {
"thinking_budget": 800,
"include_thoughts": True
}
}
}
}
)
They use OpenAI's official python library OpenAI. In OpenAI extra_body will be promoted to root. The final body is:
{
"messages": [
{
"role": "user",
"content": "Explain to me how AI works"
}
],
"model": "gemini-2.5-flash",
"extra_body": {
"google": {
"thinking_config": {
"thinking_budget": 800,
"include_thoughts": true
}
}
}
}
So Gemini's example is actually to ensure that extra_body is still under extra_body after being extracted to root, so there is an extra layer of extra_body.
Like the example in #1025, the requirement is to extract the contents of extra_body to the root.
I put a sample code here, which can print out the processed body, you can test it:
import httpx
from openai import OpenAI
def log_request(request):
print("\n===== HTTP Request Details ======")
print(f"Method: {request.method}")
print(f"URL: {request.url}")
print("Headers:")
for key, value in request.headers.items():
print(f" {key}: {value}")
print("Body:")
print(request.content.decode())
print("================================\n")
client = OpenAI(
api_key="GEMINI_API_KEY",
base_url="https://generativelanguage.googleapis.com/v1beta/openai/",
http_client=httpx.Client(event_hooks={"request": [log_request]}),
)
response = client.chat.completions.create(
model="gemini-2.5-flash",
messages=[{"role": "user", "content": "Explain to me how AI works"}],
extra_body={
'extra_body': {
"google": {
"thinking_config": {
"thinking_budget": 800,
"include_thoughts": True
}
}
}
}
)
print(response)
…ompletion requests
Describe the change
add extra_body param support in ChatCompletionRequestExtensions
Provide OpenAI documentation link
https://ai.google.dev/gemini-api/docs/openai#thinking