[BUG] Responses : add structured output for sdk #14206
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
Can this PR be reviewed and merged on priority? |
|
We are trying to use this in responses sdk can this be merged on priority |
There was a problem hiding this comment.
move mock test inside test_litellm/ so it runs through the github action
There was a problem hiding this comment.
I have removed the code that is causing tests to fail. It was functionality i added in other PR, but was removed later, I suppose its test case remained. I have removed it. After merge of this PR, that should be fixed
There was a problem hiding this comment.
It is still failing due to some error that is out of scope of this PR
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
Fix text_format parameter handling in litellm.responses() for structured output
Relevant issues
Fixes #13789
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
tests/litellm/directory, Adding at least 1 test is a hard requirement - see detailsmake test-unittesting
Type
🐛 Bug Fix
🧹 Refactoring
Changes
The issue was that the response_format parameter was not defined for SDK-related requests, hence even after specifying it in the request, the response was not a structured output. This used to work fine for Litellm proxy because when openai client is used, it sends the "text" param to Litellm, and then Litellm handles it correctly. But when for SDK implementation, OpenAI Python SDK uses "response_format" which is later converted to "text" param(this was not in litellm hence it used to fail) before sending to the api call. I have added that and now both sdk and proxy are able to accept and give structured output.