Skip to content

ChatMessage.from_openai_dict_format wraps list-of-parts content in TextContent, breaking msg.text contract #12445

Description

@Vedant-Agarwal

Describe the bug

ChatMessage.from_openai_dict_format only handles content as a plain string for user, system, and developer messages. When content is a list of content parts (the standard OpenAI Chat Completions format for multimodal messages, and also valid for text-only messages), the whole list is passed verbatim into a single TextContent. Only the tool role branch handles list content.

As a result:

  • msg.text returns a list instead of str, violating the str | None contract of ChatMessage.text / TextContent.text.
  • msg.to_dict() and msg.to_openai_dict_format() emit invalid content (a list nested where a string is expected), so the message cannot be round-tripped or sent back to OpenAI.
  • Image/file parts are silently corrupted into "text" instead of ImageContent / FileContent.

To Reproduce

from haystack.dataclasses import ChatMessage

b64 = "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mP8/x8AAwMCAO+ip1sAAAAASUVORK5CYII="
msg = ChatMessage.from_openai_dict_format({
    "role": "user",
    "content": [
        {"type": "text", "text": "What is in this image?"},
        {"type": "image_url", "image_url": {"url": f"data:image/png;base64,{b64}"}},
    ],
})

print(type(msg.text))  # <class 'list'> — should be str
print(msg.images)      # [] — the image part is lost
print(msg.to_openai_dict_format())
# {'role': 'user', 'content': [{'type': 'text', 'text': 'What is in this image?'}, ...]}
# is what you'd expect, but instead 'content' is the raw list stringified inside a TextContent,
# producing an invalid OpenAI payload on re-serialization

Expected behavior

  • {"type": "text"} parts become TextContent, {"type": "image_url"} parts with base64 data URLs become ImageContent, {"type": "file"} parts with inline file_data become FileContent (the exact inverse of to_openai_dict_format, which already produces these part formats).
  • msg.text returns the first text part as str.
  • system/developer messages accept lists of text parts.
  • Unsupported parts (non-data image URLs, file_id references, unknown types) raise ValueError instead of silently corrupting the message.

Describe your environment (please complete the following information):

  • OS: macOS
  • Haystack version: current main (d10abda, 3.2.0rc0); also reproducible on 2.x releases
  • Python version: 3.13

I have a fix ready and will open a PR shortly.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions