Skip to content

Python: fix(openai): emit image_generation_call result from streaming output_item.done - #8424

Merged
Evan Mattson (moonbox3) merged 3 commits into
microsoft:mainfrom
anishmehta24:fix/openai-responses-stream-image-generation-done
Sep 17, 2026
Merged

Evan Mattson (moonbox3) merged 3 commits into
microsoft:mainfrom
anishmehta24:fix/openai-responses-stream-image-generation-done

Conversation

@anishmehta24

Copy link
Copy Markdown
Contributor

Motivation & Context

When the hosted image_generation tool runs with stream=True and the default partial_images=0, the Responses API delivers the final image exactly once: as a completed image_generation_call item on response.output_item.done. The streaming parser in _chat_client.py only built image contents from response.image_generation_call.partial_image events, and its response.output_item.done handler had no branch for image_generation_call, so the image was silently dropped and never even reached the "unparsed event" debug log. The response ended up with only text and reasoning, history providers persisted no reference to the image, and because responses.retrieve returns result: null for the item, the bytes were lost. The non-streaming parser handled the item correctly, so the two paths were not at parity. Affects agent-framework-openai and agent-framework-foundry.

Description & Review Guide

  • Major changes: the non-streaming image_generation_call mapping is factored into a private _image_generation_item_to_contents(item) helper (same mime detection via detect_media_type_from_base64 with image/png fallback, same data: URI, same image_id and raw_representation), placed next to the existing _shell_item_to_contents. Both _parse_response_from_openai and a new image_generation_call branch in the streaming response.output_item.done handler call it, so streaming now emits image_generation_tool_call and image_generation_tool_result with the final image.
  • Impact: the partial_image handler is unchanged. With partial_images > 0, callers get the preview frames flagged is_partial_image followed by the final result under the same image_id; no dedupe is done, the final render supersedes the previews. Items without a result (for example a failed generation) emit the call plus a result with outputs=None, matching the non-streaming path.
  • Focus for reviewers: the four-line elif in the output_item.done handler and whether emitting on a result-less item is the desired semantics. Tests: three new tests next to test_parse_chunk_from_openai_with_file_search_call_done cover the final image, the result-less item, and streaming vs non-streaming parity via to_dict().

Prepared with AI assistance (Claude Code) and human-verified: the new tests fail on main and pass with the fix; poe fmt, poe lint, poe pyright -P openai, poe mypy -P openai and the openai test suite (511 passed, 95 skipped) were run locally.

Related Issue

Fixes #8422

Contribution Checklist

  • The code builds clean without any errors or warnings
  • All unit tests pass, and I have added new tests where possible
  • The PR follows the Contribution Guidelines
  • This PR is linked to an issue and there is no other open PR for this issue
  • This is not a breaking change.

🤖 Generated with Claude Code

…item.done

With the default `partial_images=0` the Responses API delivers the final
image only once, as a completed `image_generation_call` item on
`response.output_item.done`. The streaming parser had no branch for that
item type, so the image was silently dropped and `AgentResponse` contained
no image content, while the non-streaming parser mapped it correctly.

Factor the non-streaming mapping into `_image_generation_item_to_contents`
and use it from both parsers so streaming emits the same
`image_generation_tool_call` + `image_generation_tool_result` contents.

Fixes microsoft#8422

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@agent-framework-automation agent-framework-automation Bot added the python Usage: [Issues, PRs], Target: Python label Sep 16, 2026
@moonbox3
Evan Mattson (moonbox3) added this pull request to the merge queue Sep 17, 2026
Merged via the queue into microsoft:main with commit e1a901a Sep 17, 2026
46 checks passed

This branch was successfully deployed

1 active deployment
github-app-auth — 641a8401 Deployed Sep 16, 2026 by anishmehta24 via add_label #23005
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

python Usage: [Issues, PRs], Target: Python

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Python: streaming Responses parser drops the final image of image_generation_call

4 participants