[deanrie]

What

stream_batch(n) builds one GetChatCompletionResponse proto with n outputs and n Response(proto, i) objects over it, then feeds every chunk to responses[0].process_chunk(chunk). Since the buffering change in _ResponseProtoDecorator, process_chunk appends deltas to buffers on the instance, and _sync_buffers_to_proto materializes them into the shared proto only when that same instance is read. responses[1..n-1] have their own empty buffers and _proto_in_sync = True, so:

• inside the loop, responses[1].content is "" on every iteration;
• after the loop, responses[1].content is "" until responses[0].content (or .proto) has been read once, then it is correct.

The same goes for reasoning_content and encrypted_content. The docstring promises "one Response for each of the n responses, updated with each chunk".

Repro on main without a server:

proto = chat_pb2.GetChatCompletionResponse(outputs=[chat_pb2.CompletionOutput(index=i) for i in range(2)])
responses = [Response(proto, i) for i in range(2)]
responses[0].process_chunk(chunk(["Hel", "Bon"]))
responses[0].process_chunk(chunk(["lo", "jour"]))
responses[1].content  # ''
responses[0].content  # 'Hello'
responses[1].content  # 'Bonjour'

The existing test_stream_batch reads chunks[i][1].content, which comes from the chunk, and last_response[0] before last_response[1], so it did not see this.

Change

_ResponseProtoDecorator and Response take an optional keyword shared_with: another decorator over the same proto that receives the stream. process_chunk and _sync_buffers_to_proto delegate to it. stream_batch in sync/chat.py and aio/chat.py now creates first = Response(proto, 0) and Response(proto, i, shared_with=first) for the rest. sample_batch, defer_batch and the single-response paths never call process_chunk on a sibling and are unchanged. stream_batch is on its way out (PendingDeprecationWarning), but it is still public, so it should return what it documents until it goes.

Tests

test_stream_batch_every_response_reads_the_stream_as_it_arrives in tests/sync/chat_test.py and tests/aio/chat_test.py: read responses[1].content inside the loop (expect the partial text) and, in the sync variant, read responses[1] before responses[0] after the loop. Both fail on main. pytest -n 4: 871 passed; ruff check, ruff format --check, pyright clean. CHANGELOG entry under Unreleased / Fixed, which will need a one-line merge with #232 if that lands first.

[deanrie]

#202 from @latent-9 reports the same read-order problem in stream_batch and has been open since August; I missed it when opening this PR. Both branches fix the reported behaviour: #202's two regression tests pass on this branch, and this PR's test (which also checks the intermediate value on the second iteration) passes on #202.

The difference is where the cost goes. #202 calls _sync_buffers_to_proto() after every chunk, and that method collapses each buffer into one string, so the next chunk joins the full text again. For stream_batch that brings back the quadratic copying that #46 introduced the buffers to remove (a 100 KB answer in 1 KB chunks copies about 5 MB per output). This PR keeps the buffers lazy and has responses[1..n-1] read through responses[0], so per-chunk work stays as it is on main and a read of any response materialises once.

Leaving this open as the alternative shape; if #202 is preferred, this one can be closed and its intermediate-value assertion added there.