-
Notifications
You must be signed in to change notification settings - Fork 248
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
getting a loop with lmstudio local server "Object not generated according to the schema, fallback to manual parsing" #49
Comments
I'm getting the same error with LMStudio LM-Studio-0.3.9-6 on Linux and [email protected] when calling npm run dev "1+1=" . In LMStudio I'm using deepseek-r1-distill-qwen-7b. I also tried with llama-3.2-1b-instruct, same error. |
local llm's structured output is not fully tested. only gemini (1.5flash, 2.0flash) and openai (gpt-4o/-mini) are tested 2 things we still need to figure out: vercel ai-sdk's structured output on local llm, is it good? local llm's own structured output handling, is it good? |
is there any example of json schema that i can test? |
Just load a model and test it. The only one that got remotely close for me was Gemma2, also Google. It still fails periodically, though. |
gemma2-9b? can u give me list or smth of working models? |
Log:
Step 1 / Budget used 0.00%
Gaps: [ 'what is the capital of France?' ]
Object not generated according to the schema, fallback to manual parsing
undefined <- [search, answer, reflect]
{ answer: 'Paris' }
Waiting 1s...
Step 2 / Budget used 0.08%
Gaps: []
Object not generated according to the schema, fallback to manual parsing
undefined <- [search, answer, reflect]
{ answer: 'Paris' }
Waiting 1s...
model is qwen2.5-7b-instruct, identifier is same as in .env file. i think i need some example json schema for structured output, but idk where i can find it, help pls
The text was updated successfully, but these errors were encountered: