Skip to content

files: decide progress callback arity from its signature; total counts from the current position - #232

Open
deanrie wants to merge 1 commit into
xai-org:mainfrom
deanrie:files-progress-fixes
Open

deanrie wants to merge 1 commit into
xai-org:mainfrom
deanrie:files-progress-fixes

Conversation

@deanrie

@deanrie deanrie commented Oct 22, 2026

Copy link
Copy Markdown

Checklist

  • I have read both the CONTRIBUTING.md and Contributor License Agreement documents.
  • I have created an issue or feature request and received approval from xAI maintainers. (two small defects found by reading files.py; no issue filed)
  • I have tested my changes locally and they pass all CI checks.
  • I have added necessary documentation or updated existing documentation. (CHANGELOG)

Description

Two defects in the files.upload progress path, same for the sync and async clients.

1. _invoke_progress used except TypeError to detect arity. It called progress(uploaded, total) and, on TypeError, retried with progress(chunk_size). That except also catches a TypeError raised inside the user's callback, so the callback was called a second time with one argument and the user saw progress() missing 1 required positional argument: 'total' instead of their own error. The docstring's own example triggers it:

def progress(uploaded, total):
    print(f"Uploaded {uploaded}/{total} bytes ({100*uploaded/total:.1f}%)")

With a non-seekable stream total is None, 100*uploaded/total raises TypeError, and the SDK reports the wrong failure. Arity is now read once from inspect.signature; a callable that cannot be inspected gets the documented two-argument form; exceptions from the callback propagate unchanged. tqdm-style objects with .update are handled as before.

2. _chunk_file_from_fileobj reported the file's full length as total. It seeked to the end and used that offset, ignoring the current position. A file object that has been read past a header (common when the caller inspects magic bytes first) got a total the upload could never reach: (uploaded, total) == (10, 16) for 10 uploaded bytes. Now total = end - current_position, the bytes that will actually be sent.

Tests

tests/sync/files_test.py:

  • one-argument plain function receives chunk sizes summing to the payload;
  • a two-argument callback that raises TypeError on total=None surfaces its own message (unsupported operand ... NoneType), not the arity one;
  • a BytesIO read past a header reports (len(payload), len(payload)) on the last call.

The last two fail on main. uv run pytest tests/sync/files_test.py tests/aio/files_test.py: 74 passed. ruff check, ruff format --check, pyright clean on the touched files.

…s from the current position

_invoke_progress tried progress(uploaded, total) and, on TypeError, fell
back to progress(chunk_size). The except also caught TypeErrors raised
inside the user's callback, so a two-argument callback that failed on
its own (the docstring's own example does when total is None:
100 * uploaded / total) was silently called again with one argument and
the user saw 'missing 1 required positional argument: total' instead of
their error. Arity now comes from inspect.signature once per call;
callables that cannot be inspected get the documented two-argument form.

_chunk_file_from_fileobj (sync and async) computed total_bytes as the
absolute end offset, so a file object positioned past a header reported
a total the upload could never reach. It is now end minus current
position, which is what gets uploaded.

Tests: one-argument plain function gets chunk sizes; a TypeError from
inside the callback propagates with its own message; a BytesIO read past
a header reports (payload, payload) on the last call. Both new behaviours
fail on main.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant