Repository navigation
files: decide progress callback arity from its signature; total counts from the current position - #232
Open
deanrie wants to merge 1 commit into
Open
files: decide progress callback arity from its signature; total counts from the current position#232deanrie wants to merge 1 commit into
deanrie wants to merge 1 commit into
Conversation
3 of 4 tasks
…s from the current position _invoke_progress tried progress(uploaded, total) and, on TypeError, fell back to progress(chunk_size). The except also caught TypeErrors raised inside the user's callback, so a two-argument callback that failed on its own (the docstring's own example does when total is None: 100 * uploaded / total) was silently called again with one argument and the user saw 'missing 1 required positional argument: total' instead of their error. Arity now comes from inspect.signature once per call; callables that cannot be inspected get the documented two-argument form. _chunk_file_from_fileobj (sync and async) computed total_bytes as the absolute end offset, so a file object positioned past a header reported a total the upload could never reach. It is now end minus current position, which is what gets uploaded. Tests: one-argument plain function gets chunk sizes; a TypeError from inside the callback propagates with its own message; a BytesIO read past a header reports (payload, payload) on the last call. Both new behaviours fail on main.
deanrie
force-pushed
the
files-progress-fixes
branch
from
October 24, 2026 09:54
e477bb4 to
1b19b31
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Checklist
files.py; no issue filed)Description
Two defects in the
files.uploadprogress path, same for the sync and async clients.1.
_invoke_progressusedexcept TypeErrorto detect arity. It calledprogress(uploaded, total)and, onTypeError, retried withprogress(chunk_size). Thatexceptalso catches aTypeErrorraised inside the user's callback, so the callback was called a second time with one argument and the user sawprogress() missing 1 required positional argument: 'total'instead of their own error. The docstring's own example triggers it:With a non-seekable stream
totalisNone,100*uploaded/totalraisesTypeError, and the SDK reports the wrong failure. Arity is now read once frominspect.signature; a callable that cannot be inspected gets the documented two-argument form; exceptions from the callback propagate unchanged. tqdm-style objects with.updateare handled as before.2.
_chunk_file_from_fileobjreported the file's full length astotal. It seeked to the end and used that offset, ignoring the current position. A file object that has been read past a header (common when the caller inspects magic bytes first) got atotalthe upload could never reach:(uploaded, total) == (10, 16)for 10 uploaded bytes. Nowtotal = end - current_position, the bytes that will actually be sent.Tests
tests/sync/files_test.py:TypeErrorontotal=Nonesurfaces its own message (unsupported operand ... NoneType), not the arity one;BytesIOread past a header reports(len(payload), len(payload))on the last call.The last two fail on
main.uv run pytest tests/sync/files_test.py tests/aio/files_test.py: 74 passed.ruff check,ruff format --check,pyrightclean on the touched files.