Python: streaming Responses parser drops the final image of `image_generation_call`

[CristinaStn]

Package: agent-framework-openai (affects agent-framework-foundry, which inherits the parser)
Versions checked: 1.10.0, 1.13.0, 1.14.3 (latest on PyPI) and main @ 6c3c58a4b2d8834ad3dce3d0b01ad10f7edaf352 — all affected
File: python/packages/openai/agent_framework_openai/_chat_client.py, _parse_chunk_from_openai

Summary

When the hosted image_generation tool is used with stream=True and partial_images is not
requested (the default, 0), the generated image never reaches the AgentResponse. The final
render is delivered by the Responses API on response.output_item.done as a completed
image_generation_call item whose result holds the base64 image, but the streaming parser has no
branch for that item. It only creates image_generation_tool_call / image_generation_tool_result
contents from response.image_generation_call.partial_image events, i.e. from progressive preview
frames. The non-streaming parser handles the completed item correctly, so streaming and
non-streaming are not at parity.

Consequences with the default tool configuration:

• AgentResponse.messages contains no image content (only text_reasoning / text).
• HistoryProviders persist a message with no reference to the image.
• Any consumer built on the documented content types silently loses the image; the only way to get
  it is to dig into response.raw_representation (which is also what the Foundry docs sample does).

Reproduction

import asyncio
from agent_framework import Agent
from agent_framework_foundry import FoundryChatClient
from azure.identity import AzureCliCredential

async def main():
    client = FoundryChatClient(
        project_endpoint="<project endpoint>",
        model="gpt-5",  # same on gpt-5.4, gpt-5.6-terra
        credential=AzureCliCredential(),
        default_headers={"x-ms-oai-image-generation-deployment": "gpt-image-2"},
    )
    agent = Agent(
        client=client,
        instructions="You are a helpful assistant.",
        tools=[client.get_image_generation_tool(model="gpt-image-2")],  # no partial_images
    )
    stream = agent.run("generate an image of a black cat with yellow eyes", stream=True)
    async for update in stream:
        ev = update.raw_representation.raw_representation
        if getattr(ev, "type", None) == "response.output_item.done" and ev.item.type == "image_generation_call":
            print("provider sent final image, base64 length:", len(ev.item.result))  # ~2,000,000
    final = await stream.get_final_response()
    print([c.type for m in final.messages for c in m.contents])
    # observed: ['text_reasoning', 'text']
    # expected: [..., 'image_generation_tool_call', 'image_generation_tool_result', 'text']

asyncio.run(main())

Raw SSE sequence observed from Foundry (identical on every model, no partial frames):

response.output_item.added        item.type=image_generation_call status=in_progress
response.image_generation_call.in_progress
response.image_generation_call.generating
response.image_generation_call.completed
response.output_item.done         item.type=image_generation_call status=completed result_len=2401164
response.completed                output=[image_generation_call(result_len=2401164), message]

Passing partial_images=1 makes the image appear (via the partial_image handler), which confirms
the diagnosis, but it persists a preview frame rather than the final render and depends on the
service emitting previews.

Code

Non-streaming path maps the completed item (works):

agent-framework/python/packages/openai/agent_framework_openai/_chat_client.py

Lines 2985 to 3007 in 6c3c58a

	case "image_generation_call": # ResponseOutputImageGenerationCall
	image_output: Content | None = None
	if item.result is not None:
	# item.result contains raw base64 string
	# so we call detect_media_type_from_base64 to get the media type and fallback to image/png
	image_output = Content.from_uri(
	uri=f"data:{detect_media_type_from_base64(data_str=item.result) or 'image/png'}"
	f";base64,{item.result}",
	raw_representation=item.result,
	)
	image_id = item.id
	contents.append(
	Content.from_image_generation_tool_call(
	image_id=image_id,
	raw_representation=item,
	)
	)
	contents.append(
	Content.from_image_generation_tool_result(
	image_id=image_id,
	outputs=image_output,
	raw_representation=item,
	)

Streaming path only maps partial_image events:

agent-framework/python/packages/openai/agent_framework_openai/_chat_client.py

Lines 3441 to 3471 in 6c3c58a

	case "response.image_generation_call.partial_image":
	# Handle streaming partial image generation
	image_base64 = event.partial_image_b64
	partial_index = event.partial_image_index
	image_output = Content.from_uri(
	uri=f"data:{detect_media_type_from_base64(data_str=image_base64) or 'image/png'}"
	f";base64,{image_base64}",
	additional_properties={
	"partial_image_index": partial_index,
	"is_partial_image": True,
	},
	raw_representation=event,
	)
	image_id = getattr(event, "item_id", None)
	contents.append(
	Content.from_image_generation_tool_call(
	image_id=image_id,
	raw_representation=event,
	)
	)
	contents.append(
	Content.from_image_generation_tool_result(
	image_id=image_id,
	outputs=image_output,
	raw_representation=event,
	)
	)
	case "response.output_text.annotation.added":
	# Handle streaming text annotations (file citations, file paths, etc.)
	annotation: Any = event.annotation

Streaming response.output_item.done handles reasoning, mcp_call, web_search_call,
file_search_call, shell items, custom_tool_call, tool_search_call — but not
image_generation_call:

agent-framework/python/packages/openai/agent_framework_openai/_chat_client.py

Lines 3568 to 3623 in 6c3c58a

	case "response.output_item.done":
	done_item = event.item
	if getattr(done_item, "type", None) == "reasoning":
	encrypted_content = getattr(done_item, "encrypted_content", None)
	if encrypted_content:
	contents.append(
	Content.from_text_reasoning(
	id=getattr(done_item, "id", None),
	text="",
	protected_data=encrypted_content,
	raw_representation=done_item,
	)
	)
	elif getattr(done_item, "type", None) == "mcp_call":
	call_id = getattr(done_item, "id", None) or getattr(done_item, "call_id", None) or ""
	output_text = getattr(done_item, "output", None)
	parsed_output: list[Content] | None = (
	[Content.from_text(text=output_text)] if isinstance(output_text, str) else None
	)
	contents.append(
	Content.from_mcp_server_tool_result(
	call_id=call_id,
	output=parsed_output,
	raw_representation=done_item,
	)
	)
	elif getattr(done_item, "type", None) in ("web_search_call", "file_search_call"):
	contents.append(self._parse_search_tool_result_content(done_item))
	elif getattr(done_item, "type", None) in ("shell_call", "local_shell_call", "shell_call_output"):
	# Shell items are parsed here (not on `response.output_item.added`) because the
	# command/output is only populated on the completed item.
	contents.extend(self._shell_item_to_contents(done_item, local_shell_tool_name))
	elif getattr(done_item, "type", None) == "custom_tool_call":
	custom_tool_call = cast(ResponseCustomToolCall, done_item)
	contents.append(
	self._parse_hosted_function_call_content(
	custom_tool_call,
	name=custom_tool_call.name,
	arguments=custom_tool_call.input,
	)
	)
	elif getattr(done_item, "type", None) == "tool_search_call":
	tool_search_call = cast(ResponseToolSearchCall, done_item)
	contents.append(
	self._parse_hosted_function_call_content(
	tool_search_call,
	name="tool_search",
	arguments=tool_search_call.arguments,
	)
	)
	elif getattr(done_item, "type", None) == _AZURE_AI_SEARCH_CALL_OUTPUT_TYPE:
	pass
	case _:
	if not isinstance(event.type, str) or not event.type.startswith(_AZURE_AI_SEARCH_OUTPUT_EVENT_PREFIX):
	logger.debug("Unparsed event of type: %s: %s", event.type, event)

response.output_item.added lists ImageGenerationCall in its comment of handled types but has no
case for it either (line 3295).

Note the drop is silent: because the outer match does hit case "response.output_item.done", the
image item never reaches the case _: fallback, so not even the "Unparsed event of type ..." debug
line is emitted for it. With agent_framework.openai at DEBUG the only trace is the earlier
output_item.added event being reported as unparsed:

_chat_client.py:_parse_chunk_from_openai - Unparsed event of type: response.output_item.added: ResponseOutputItemAddedEvent(item=ImageGenerationCall(id='ig_...', result=None, status='in_progress', ...
_chat_client.py:_parse_chunk_from_openai - Unparsed event of type: response.image_generation_call.in_progress: ...
_chat_client.py:_parse_chunk_from_openai - Unparsed event of type: response.image_generation_call.generating: ...
_chat_client.py:_parse_chunk_from_openai - Unparsed event of type: response.image_generation_call.completed: ...
(nothing for response.output_item.done, which carries the 2.5 MB result)

Origin

• The streaming image handling was introduced by PR Python: Add tool call/result content types and update connectors and samples #2971 "Add tool call/result content types and
  update connectors and samples" (merged 2026-01-08, 3f7ea350dc), in
  python/packages/core/agent_framework/openai/_responses_client.py. It added the partial_image
  handler and the accompanying sample openai_responses_client_streaming_image_generation.py, which
  sets partial_images, so the final-item case was never exercised.
• PR Python: [BREAKING] Python: Provider-leading client design & OpenAI package extraction #4818 (2026-03-25, 5e056b672e) moved the parser to agent_framework_openai/_chat_client.py
  unchanged in this respect.
• Streaming support itself came in PR Python: OpenAI Responses Image Generation Stream Support, Sample and Unit Tests #1853 (merged 2025-11-11), "with partial images" — the
  partial_image handler and its sample were written around preview frames from the start.
• Bisecting every commit of _chat_client.py since then: no revision has ever handled
  image_generation_call on response.output_item.done. This is a gap since the feature was added,
  not a regression from a specific release. It surfaced for us in the second week of September 2026
  because the Foundry Responses endpoint used to emit one partial_image frame even without
  partial_images (which the handler above picked up) and stopped doing so; the same code, same
  dependency versions and same deployments that returned images up to 2026-09-08 return none now.
  The documented contract (OpenAI image generation guide: partial_images 0 → only the final image,
  delivered on the completed item) was never handled by the streaming parser.

Also verified: the stored response (responses.retrieve) returns the image item with
"result": null, and no include value restores it, so the streamed output_item.done event is
the only moment the bytes are available. Dropping it there loses the image for good.

Proposed fix

Add an image_generation_call branch to the response.output_item.done handler mirroring the
non-streaming mapping (call + result content, result as a data: URI). Patch against main:

             case "response.output_item.done":
                 done_item = event.item
-                if getattr(done_item, "type", None) == "reasoning":
+                if getattr(done_item, "type", None) == "image_generation_call":
+                    image_result = getattr(done_item, "result", None)
+                    if image_result:
+                        image_id = getattr(done_item, "id", None)
+                        contents.append(
+                            Content.from_image_generation_tool_call(
+                                image_id=image_id, raw_representation=done_item
+                            )
+                        )
+                        contents.append(
+                            Content.from_image_generation_tool_result(
+                                image_id=image_id,
+                                outputs=Content.from_uri(
+                                    uri=f"data:{detect_media_type_from_base64(data_str=image_result) or 'image/png'}"
+                                    f";base64,{image_result}",
+                                    raw_representation=image_result,
+                                ),
+                                raw_representation=done_item,
+                            )
+                        )
+                elif getattr(done_item, "type", None) == "reasoning":
                     encrypted_content = getattr(done_item, "encrypted_content", None)

When partial_images is also set, consumers already keep the last result per image_id, so the
final render simply supersedes the preview frames.

[CristinaStn]

Proof from a real run (added after filing)

agent_framework.openai logger at DEBUG during one streamed image request. The parser reports the
output_item.added image item and the three progress events as unparsed; the output_item.done
event that carries the result produces no line at all (it matches the done case and falls through):

13:12:09.025 | DEBUG | _chat_client.py:_parse_chunk_from_openai:3178 - Unparsed event of type: response.output_item.added: ResponseOutputItemAddedEvent(item=ImageGenerationCall(id='ig_0fe397a81f463db1006aaa6b74afac819599a01d90a2e99aba', result=None, status='in_progress', type='image_generation_call', response_id='resp_0fe397a81f463db1006aaa6b7277f08195b6665a84bd74b88b'), output_index=1, sequence_number=4, type='response.output_item.added')
13:12:16.018 | DEBUG | _chat_client.py:_parse_chunk_from_openai:3384 - Unparsed event of type: response.image_generation_call.in_progress: ResponseImageGenCallInProgressEvent(item_id='ig_0fe397a8...', output_index=1, sequence_number=5, ...)
13:12:16.088 | DEBUG | _chat_client.py:_parse_chunk_from_openai:3384 - Unparsed event of type: response.image_generation_call.generating: ResponseImageGenCallGeneratingEvent(item_id='ig_0fe397a8...', output_index=1, sequence_number=6, ...)
13:12:22.103 | DEBUG | _chat_client.py:_parse_chunk_from_openai:3384 - Unparsed event of type: response.image_generation_call.completed: ResponseImageGenCallCompletedEvent(item_id='ig_0fe397a8...', output_index=1, sequence_number=7, ...)
                (sequence_number=8 = response.output_item.done for ig_0fe397a8... with the base64 result — no log line, silently dropped)
13:12:22.603 | DEBUG | _chat_client.py:_parse_chunk_from_openai:3178 - Unparsed event of type: response.output_item.added: ResponseOutputItemAddedEvent(item=ResponseOutputMessage(id='msg_0fe397a8...', content=[], role='assistant', status='in_progress', type='message', ...), output_index=2, sequence_number=9, ...)
13:12:22.605 | DEBUG | _chat_client.py:_parse_chunk_from_openai:3384 - Unparsed event of type: response.output_text.done: ResponseTextDoneEvent(content_index=0, item_id='msg_0fe397a8...', output_index=2, sequence_number=11, text='', ...)
13:12:22.606 | DEBUG | _chat_client.py:_parse_chunk_from_openai:3384 - Unparsed event of type: response.content_part.done: ResponseContentPartDoneEvent(content_index=0, item_id='msg_0fe397a8...', output_index=2, part=ResponseOutputText(annotations=[], text='', type='output_text', logprobs=[]), sequence_number=12, ...)

The same run as persisted by CosmosHistoryProvider (agent-framework-azure-cosmos): the assistant
message holds only the reasoning item and an empty text — nothing references the image that was
generated (encrypted reasoning payload and Cosmos system fields trimmed):

{
  "session_id": "87e32ef8-82cc-4ed7-a3c6-7846b863f6e0",
  "source_id": "azure_cosmos_history",
  "message": {
    "type": "message",
    "role": "assistant",
    "contents": [
      {
        "type": "text_reasoning",
        "protected_data": "gAAAAABqqmm4dSw8XJEny1_h2nwGHGTXKDnq...",
        "id": "rs_0fe397a81f463db1006aaa69b60c0c8195b8d16b66df336335",
        "additional_properties": {}
      },
      { "type": "text", "text": "", "additional_properties": {} }
    ],
    "author_name": "GeneralAssistant",
    "additional_properties": {}
  }
}

Expected: image_generation_tool_call and image_generation_tool_result contents for
ig_0fe397a81f463db1006aaa6b74afac819599a01d90a2e99aba between the reasoning and the text, as the
non-streaming parser would produce.

[anishmehta24]

Thanks for the thorough report, the diagnosis matches what I see in the code. I've put together a fix that adds the missing image_generation_call branch to the streaming response.output_item.done handler; rather than duplicating the mapping, it factors the existing non-streaming logic into a small shared helper so both paths emit identical image_generation_tool_call / image_generation_tool_result contents (same mime detection and data: URI). It includes unit tests for the done event with and without a result, plus a parity test comparing streaming and non-streaming output. The partial_image path is untouched, so with partial_images > 0 the final render simply follows the previews under the same image_id. PR incoming shortly.

[agooll]

Thanks for the detailed diagnosis and proposed patch.

Before I start working on this, are you planning to submit the fix yourself? If not, and the maintainers agree with the proposed direction, I’d be happy to add focused regression tests for the streaming image-generation path and implement the smallest compatible fix.