-
-
Couldn't load subscription status.
- Fork 10.9k
[Bug] Qwen reasoning parser #27241
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
[Bug] Qwen reasoning parser #27241
Conversation
Signed-off-by: ahao-anyscale <[email protected]>
Signed-off-by: ahao-anyscale <[email protected]>
| @staticmethod | ||
| def verify_and_update_config(vllm_config: "VllmConfig") -> None: | ||
| structured_outputs_config = vllm_config.structured_outputs_config | ||
| if structured_outputs_config.reasoning_parser == "": |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How do you start vLLM?
This field structured_outputs_config.reasoning_parser can be set via the parameter --reasoning-parser qwen3.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is the user intended to supply these arguments? I seems that it would make more sense to automatically set them for reasoning models. I saw that gptoss had its reasoning-parser set automatically via verify_and_update_config. Thanks for reviewing!
Purpose
When running with Qwen/Qwen3-30B-A3B,
reasoning_contentis empty despite correctly formed<think>tag. Relevant issue: ray-project/ray#57919Test Plan
Run requests with Qwen/Qwen3-30B-A3B and observe that
reasoning_contentis populated instead ofcontent.Essential Elements of an Effective PR Description Checklist
supported_models.mdandexamplesfor a new model.