Skip to content

DocumentToImageContent raises ValueError on bad inputs, blocking LLMDocumentContentExtractor partial-success #12447

Description

@Harsh23Kashyap

Describe the bug

DocumentToImageContent.run() raises ValueError on per-document validation failures (unsupported MIME type, missing page_number for PDFs), which makes the documented partial-success contract of LLMDocumentContentExtractor (failed_documents return) unreachable for any batch that contains even one invalid document.

The behavior is inconsistent with how DocumentToImageContent.run() already handles a different class of failure: a valid PDF whose requested page does not exist. There, run() returns None for that document's image_contents entry and logs a warning (see test_run_none_images). Per-document validation failures, by contrast, short-circuit the whole batch. The downstream LLMDocumentContentExtractor.run / run_async is explicitly designed around None-based per-document failure handling (docstring + test_run_with_mixed_success_and_failure), so a user with a heterogeneous document collection cannot reach the documented failed_documents return path today.

Error message

ValueError: Document with file path '...sample_docx.docx' has an unsupported MIME type 'application/vnd.openxmlformats-officedocument.wordprocessingml.document'. Please ensure that the documents you are trying to convert are of the supported types: image/bmp, image/gif, image/jpeg, image/png, image/tiff, image/webp, application/pdf.

For the missing-page-number case, the equivalent is:

ValueError: Document with ID '...' comes from the PDF file '...sample_pdf_1.pdf' but is missing the 'page_number' key in its metadata. Please ensure that PDF documents you are trying to convert have this key set.

Both raise from _extract_image_sources_info in haystack/components/converters/image/image_utils.py, before the per-document loop in DocumentToImageContent.run runs.

Expected behavior

A batch containing one valid and one invalid document should return:

  • image_contents: [<ImageContent for the valid doc>, None]
  • a single WARNING log line from DocumentToImageContent naming the bad document(s)

…so that LLMDocumentContentExtractor.run(docs) can route the bad document to failed_documents (with content_extraction_error in metadata) and the good document to documents. This is the same shape the extractor already uses for the "PDF page that does not exist" case (precedent: test_run_none_images).

To Reproduce

import os
from haystack import Document
from haystack.components.converters.image.document_to_image import DocumentToImageContent

os.chdir("/Users/harshkashyap/Projects/Open Source/haystack")  # for test/test_files

converter = DocumentToImageContent(root_path="test/test_files")
docs = [
    Document(content="", meta={"file_path": "images/apple.jpg"}),                       # valid
    Document(content="", meta={"file_path": "pdf/sample_pdf_1.pdf", "page_number": 1}),  # valid
    Document(content="text", meta={"file_path": "docx/sample_docx.docx"}),               # unsupported MIME
]
result = converter.run(documents=docs)
print(result)

Traceback:

Traceback (most recent call last):
  File "...", line 17, in <module>
    result = converter.run(documents=docs)
  File ".../haystack/components/converters/image/document_to_image.py", line 124, in run
    images_source_info = _extract_image_sources_info(...)
  File ".../haystack/components/converters/image/image_utils.py", line 122, in _extract_image_sources_info
    raise ValueError(...)
ValueError: Document with file path '...sample_docx.docx' has an unsupported MIME type '...'

The third document is invalid, but the first two are perfectly convertible. The first two are dropped because the validation step in _extract_image_sources_info raises before the per-document loop in run finishes.

Additional context

The existing tests test_run_with_invalid_file_path and test_run_with_pdf_missing_page_number (in test/components/converters/image/test_document_to_image_content.py) currently enforce the ValueError behavior with pytest.raises. They would need to be updated to expect the new None + warning behavior.

Existing precedent for the proposed behavior: test_run_none_images, which mocks _batch_convert_pdf_pages_to_images to return an empty dict (simulating a missing PDF page) and asserts that image_contents has None for that entry plus a logged warning.

Possible fix directions (open to maintainer preference):

  1. DocumentToImageContent._extract_image_sources_info returns a list of image_info | None; run keeps the existing image_contents: list[ImageContent | None] shape. Per-document validation failures become None entries. This matches the existing test_run_none_images pattern.
  2. Add a raise_on_failure: bool = True flag on DocumentToImageContent; LLMDocumentContentExtractor passes raise_on_failure=False to opt into partial-success. Strict (current) behavior remains the default.
  3. Keep the current ValueError behavior and instead update LLMDocumentContentExtractor to translate the exception into a per-document failed_documents entry (requires walking the inputs to map the failing path back to a document ID).

I'm happy to open a PR once the maintainers confirm which direction is preferred.

FAQ Check

  • Have you had a look at our FAQ page? Yes.

System:

  • OS: macOS 15.x
  • Haystack version: main @ ba92ec9 (3.2.0rc0)
  • Python version: 3.13

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions