Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Support for Mixed Messages Containing Both tool_call and Text from ChatCompletionClient #5155

Open
zjsxply opened this issue Jan 23, 2025 · 1 comment
Milestone

Comments

@zjsxply
Copy link

zjsxply commented Jan 23, 2025

What feature would you like to be added?

The ChatCompletionCache should support messages that contain both tool_call and text (content) fields simultaneously. Currently, the implementation raises warnings and defaults to processing only tool_call, ignoring the content field. This is limiting and does not align with the behavior of certain models, such as gpt-4o, which can naturally return mixed messages.

/data/xxx/anaconda3/envs/apr/lib/python3.13/site-packages/autogen_ext/models/cache/_chat_completion_cache.py:132: UserWarning: Both tool_calls and content are present in the message. This is unexpected. content will be ignored, tool_calls will be used.
  result = await self.client.create(

For example, a tool_call might include instructions to invoke a tool, and the content field could provide supplemental context or fallback output, both of which are important for downstream applications.

The desired feature should allow a single message to contain both tool_call and content fields without raising warnings or dropping data.

Why is this needed?

Models such as gpt-4o are designed to return mixed messages. Ignoring part of the response reduces the utility of these models and misaligns with their intended behavior.

@ekzhu ekzhu changed the title Add Support for Mixed Messages Containing Both tool_call and Text in ChatCompletionCache Add Support for Mixed Messages Containing Both tool_call and Text ChatCompletionClient Jan 23, 2025
@ekzhu
Copy link
Collaborator

ekzhu commented Jan 23, 2025

The warning comes from the openai client. Though the issue is more general, the CreateResult field current doesn't allow both text and tool call in the same result.

@jackgerrits Something we need to fix.

@ekzhu ekzhu added this to the 0.4.x milestone Jan 23, 2025
@ekzhu ekzhu changed the title Add Support for Mixed Messages Containing Both tool_call and Text ChatCompletionClient Add Support for Mixed Messages Containing Both tool_call and Text from ChatCompletionClient Jan 23, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants