-
Notifications
You must be signed in to change notification settings - Fork 526
Open
Labels
bugSomething isn't workingSomething isn't workingstatus: needs triageNew issues that have not yet been reviewed or categorized.New issues that have not yet been reviewed or categorized.
Description
Did you check docs and existing issues?
- I have read all the NeMo-Guardrails docs
- I have updated the package to the latest version before submitting this issue
- (optional) I have used the develop branch
- I have searched the existing issues of NeMo-Guardrails
Python version (python --version)
Python 3.12.5
Operating system/version
MacOS 15.3.2
NeMo-Guardrails version (if you must use a specific version and not the latest
No response
Describe the bug
models:
- type: main
engine: openai
model: gpt-4.1-2025-04-14
parameters:
temperature: 0.2
stream: true
for this config trying to work with the nemoguardrails server with stream: true it is failing with the issue below:
File "/Users/Swayam/Documents/Projects/pandemos/client/.venv/lib/python3.12/site-packages/nemoguardrails/library/self_check/input_check/actions.py", line 72, in self_check_input
response = await llm_call(llm, prompt, stop=stop)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/Swayam/Documents/Projects/pandemos/client/.venv/lib/python3.12/site-packages/nemoguardrails/actions/llm/utils.py", line 122, in llm_call
raise LLMCallException(e)
nemoguardrails.actions.llm.utils.LLMCallException: LLM Call Exception: 'AsyncStream' object has no attribute 'model_dump'
Steps To Reproduce
- Create an OpenAI config with stream true
- Start the nemoguardrails server
- Try to ask any query or use a custom openai client with local base url to ask AsyncOpenAI streaming queries.
Expected Behavior
- Streaming response should be returned neatly.
Actual Behavior
- Internal server error occurs.
Metadata
Metadata
Assignees
Labels
bugSomething isn't workingSomething isn't workingstatus: needs triageNew issues that have not yet been reviewed or categorized.New issues that have not yet been reviewed or categorized.