Skip to content

files: await an async progress callback on the async upload path - #234

Open
deanrie wants to merge 2 commits into
xai-org:mainfrom
deanrie:files-async-progress
Open

deanrie wants to merge 2 commits into
xai-org:mainfrom
deanrie:files-async-progress

Conversation

@deanrie

@deanrie deanrie commented Oct 23, 2026

Copy link
Copy Markdown

Stacked on #232 (same file, same helper); the diff here is the last commit. Merge #232 first, or merge this and #232 comes with it.

Checklist

Description

AsyncClient.files.upload accepts an on_progress callback and calls it from the three async chunk generators through the sync _invoke_progress. When the callback is an async def, the natural shape in async code and one the ProgressCallback type did not forbid, the call returns a coroutine that nobody awaits: the callback body never runs, and Python warns coroutine '...' was never awaited once per chunk.

async def progress(uploaded, total): ...
await client.files.upload("big.bin", on_progress=progress)
# main: progress body never runs; RuntimeWarning per chunk

Change

  • _invoke_progress returns whatever the callback returned.
  • The async generators call a new _async_invoke_progress, which awaits that value when it is awaitable. Sync callbacks and tqdm-like objects take the same path as before (their return value is None).
  • ProgressCallback admits callables returning Awaitable[None], so an async def type-checks.

The sync client is unchanged: it cannot await, and an async def passed there still produces the same warning it always did; that remains a caller error on the sync path.

Tests

tests/aio/files_test.py: an async def callback on AsyncClient.files.upload is called for both chunks with (uploaded, total) and ends at (len(data), len(data)), with RuntimeWarning turned into an error for the duration. Fails on main with the never-awaited warning. uv run pytest tests/aio/files_test.py tests/sync/files_test.py: 75 passed. ruff check, ruff format --check, pyright clean on the touched files.

@deanrie
deanrie requested a review from a team as a code owner October 23, 2026 09:54
…s from the current position

_invoke_progress tried progress(uploaded, total) and, on TypeError, fell
back to progress(chunk_size). The except also caught TypeErrors raised
inside the user's callback, so a two-argument callback that failed on
its own (the docstring's own example does when total is None:
100 * uploaded / total) was silently called again with one argument and
the user saw 'missing 1 required positional argument: total' instead of
their error. Arity now comes from inspect.signature once per call;
callables that cannot be inspected get the documented two-argument form.

_chunk_file_from_fileobj (sync and async) computed total_bytes as the
absolute end offset, so a file object positioned past a header reported
a total the upload could never reach. It is now end minus current
position, which is what gets uploaded.

Tests: one-argument plain function gets chunk sizes; a TypeError from
inside the callback propagates with its own message; a BytesIO read past
a header reports (payload, payload) on the last call. Both new behaviours
fail on main.
AsyncClient.files.upload accepted an async def on_progress callback, called
it from the async chunk generators through the sync _invoke_progress, and
dropped the returned coroutine. The callback body never ran and Python
emitted 'coroutine ... was never awaited' once per chunk. Writing the
callback as a coroutine is the natural thing to do in async code and the
type did not forbid it.

_invoke_progress now returns the callback's result; the three async
generators call _async_invoke_progress, which awaits the result when it
is awaitable. Sync callbacks and tqdm-like objects are unchanged.
ProgressCallback admits awaitable-returning callables.

Test: an async def callback on AsyncClient.files.upload runs on every
chunk with RuntimeWarning raised as an error; fails on main.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant