Skip to content

feat: Windows on ARM64 (NVIDIA RTX Spark) support - #261

Open
lstein wants to merge 13 commits into
mainfrom
feat/windows-arm64-rtx-spark
Open

lstein wants to merge 13 commits into
mainfrom
feat/windows-arm64-rtx-spark

Conversation

@lstein

@lstein lstein commented Sep 15, 2026

Copy link
Copy Markdown
Collaborator

Summary

Invoke could not be installed on Windows on ARM64 (NVIDIA's RTX Spark developer preview): every marker treated win32 as x64, the lockfile carried only win_amd64 wheels, and several dependencies have no ARM64 build at the pinned versions. With this PR, uv sync --frozen and uv pip install on a native ARM64 Python 3.12 produce a working install with NVIDIA's torch 2.14.0+cu134 (from https://pypi.nvidia.com/nvtorch_oot/), CI proves it on GitHub's windows-11-arm runners, and every other platform resolves exactly as before apart from one deliberate change (bcrypt 5 replaces passlib).

How it fits together:

  • pyproject.toml forks the packages whose ARM64 wheels start at newer versions (numpy 2.3, onnx 1.18, onnxruntime 1.24, sentencepiece 0.2.2, bitsandbytes 0.50.1, cryptography 46.0.x, PyWavelets 1.10) behind sys_platform == 'win32' and platform_machine == 'ARM64', routes torch/torchvision to NVIDIA's index there for every extra, and sets uv's required-environments so a wheel-only package without an ARM64 wheel fails at lock time.
  • OpenCV, blake3 and tiktoken have no ARM64 wheel anywhere. A new dispatch-only workflow builds them on windows-11-arm and publishes them, with a flat index.html, as assets of release win-arm64-wheels-v1; pyproject consumes that page as a flat index and pins the exact versions so the manual --find-links path prefers the wheels over PyPI's sdists. Pull-request runs of that workflow only build and verify; tags are immutable in practice because the lockfile pins the assets by hash.
  • scripts/check_aarch64_lock.py becomes scripts/check_platform_locks.py: the aarch64 check is unchanged, and win32/ARM64 additionally requires torch from NVIDIA's registry and walks the whole dependency closure for win_arm64 wheels (uv's required-environments cannot see packages that also publish an sdist).
  • Runtime: mediapipe (no solutions API on ARM64) and umap/numba (no wheel for 3.12) are excluded there; the face nodes import mediapipe lazily and name the platform when it is the cause, the image map falls back to a PCA projection, password hashing calls bcrypt directly (passlib 1.7 cannot use bcrypt >= 4.1, the first release with ARM64 wheels), the bitsandbytes guards survive a build whose native library fails to load, and startup logs the torch build and host architecture. One shim is specific to NVIDIA's build: without a CUDA device it dies with an access violation in torch.cuda.is_current_stream_capturing(), which transformers calls on every forward pass, so on such a machine the app (and the test session) make it answer False.
  • Docs cover the native interpreter request (cpython-3.12-windows-aarch64-none), the install options, and the platform's feature differences.

The launcher needs a companion change (arch detection, aarch64 uv, native Python request, arm64 package build); it is prepared but not part of this PR. pins.json needs no change: the launcher's bootstrap path installs from the release's own lockfile.

Related Issues / Discussions

NVIDIA RTX Spark Developer Preview: https://forums.developer.nvidia.com/t/rtx-spark-developer-preview/377106

QA Instructions

Local (Linux, worktree venv):

  • uv lock then uv lock --locked: resolved; the diff vs main is the ARM64 forks plus bcrypt 3.2.2 → 5.0.0 and passlib removed (verified by parsing both locks: no other package's version or wheel list changed).
  • uv run --no-project --with 'packaging<26' --python 3.12 scripts/check_platform_locks.py uv.locklinux/aarch64 OK, win32/ARM64 OK (115 packages in each ARM64 universe, none without a wheel).
  • python3 scripts/check_pins.py → consistent.
  • pytest tests/app/services/auth tests/app/services/image_index tests/app/routers/test_utilities.py tests/backend/image_util tests/test_imports.py tests/test_check_platform_locks.py tests/test_check_pins.py → pass.
  • Full suite (pytest -n logical): 8250 passed; 13 failures are the same GPU tests that fail identically on an untouched main checkout because both GPUs on this machine were at 99%/82% use (HIP out-of-memory in test_flux2_working_memory, test_model_cache_*, test_pid_chunked_equivalence), plus the repo-lock check that passes since the lock was regenerated.
  • ruff check / ruff format --check clean; pnpm -C docs build → 326 pages, no new warnings.

GitHub:

  • build-win-arm64-wheels: OpenCV, blake3 and tiktoken built and smoke-imported on windows-11-arm; release win-arm64-wheels-v1 carries the three wheels and index.html; the publish job's uv pip compile --find-links <page> verification passed. (First OpenCV attempt failed: MSVC has no __fp16, which the DNN Winograd kernel uses once NEON_FP16 is in the CPU baseline; fixed by pinning the baseline to NEON + DOTPROD dispatch.)
  • PR CI on the final commit: every check green. The py3.12: windows-arm64 lane installs from the lock on a native ARM64 interpreter (platform.machine() == "ARM64" asserted), imports the self-built wheels, and runs the full suite: 7,993 passed, 177 skipped, 9 xfailed in 13 minutes (the lane takes 18 minutes end to end without uv's cache). Its earlier runs surfaced four ARM64-only failures, all fixed: the stream-capture crash above (which had taken an xdist worker down and left ~2,500 tests unrun), torch 2.14 rejecting an SDPA test's explicit mask combined with is_causal=True, a frame fixture that overflowed uint8 (numpy 2 no longer wraps), and a MediaPipe gate test that assumed a non-ARM64 host.

Not verified (no hardware): CUDA generation on sm_121, cuDNN SDPA priority on this torch build, cudaMallocAsync acceptance in configure_torch_cuda_allocator(), bitsandbytes 0.50.2's CUDA kernels, VRAM accounting on unified memory. The docs mark the platform as a developer preview.

Review

Four independent read-only reviews on the pushed diff (adversarial fresh-context; correctness/spec; architecture/operational safety/performance; tests/docs/product). Material findings resolved: the wheel workflow's pull-request trigger could republish non-reproducible wheels over the assets the lock pins by hash (an in-flight run was cancelled, publishing is now dispatch-only under a concurrency group, and wheels are verified before upload); a remaining top-level import mediapipe would have failed test_imports on ARM64; the woff2 font test needs brotli, absent on ARM64; the checker's closure walk skipped a package's extras once a plain edge had reached it (fixed, with a fixture that fails against the old walk); the MediaPipe error attributed every import failure to the platform; the admin guide still generated password hashes with passlib; bitsandbytes guards catch (ImportError, OSError, RuntimeError) rather than everything; the ARM64 test lane no longer caches uv's store; docs wording for patchmatch, woff2 and launcher support corrected. Final blocker-only review of the candidate: no blockers, every earlier resolution confirmed at its head, and a 400-universe old-vs-new lock comparison across the non-ARM64 platforms showing no change beyond bcrypt/passlib. The three CI-driven commits after it (the stream-capture shim and three test fixes) had a separate blocker-only review.

Compatibility / Rollout

  • The published wheel's metadata now carries ARM64-only exact pins; nothing changes for other platforms' resolution.
  • Stored password hashes are unaffected: bcrypt verifies the $2b$12$ hashes passlib produced (covered by a regression test), and new hashes use the same cost factor.
  • Bumping a self-built wheel means dispatching the workflow with a new release tag and updating the tag and pins in pyproject.toml together, then re-locking; never overwrite a tag that a tagged Invoke release's lock names.
  • The launcher change ships separately; until then Windows ARM64 users install manually (documented).

Checklist

  • The PR has a short but descriptive title, suitable for a changelog
  • Meaningful regression coverage added / updated where needed; obsolete tests/code removed
  • Persisted-state and API changes include required migrations / compatibility validation
  • Relevant performance/efficiency opportunities considered; material claims have evidence
  • Material review findings resolved and relevant checks rerun
  • Documentation added / updated (if applicable)
  • Updated What's New copy (if doing a release after this PR)

🤖 Generated with Claude Code

https://claude.ai/code/session_01NiaFTnYH9q9468Naf4PV3X

lstein and others added 2 commits September 14, 2026 22:14
Adds a dispatch-only workflow that builds opencv-contrib-python, blake3 and
tiktoken on the windows-11-arm runner and attaches them, with a flat index
page, to a GitHub release for pyproject's Windows ARM64 sources to consume.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NiaFTnYH9q9468Naf4PV3X
@github-actions github-actions Bot added the ci-cd label Sep 15, 2026
lstein and others added 4 commits September 14, 2026 22:20
Windows ARM64 has no wheels for mediapipe's legacy solutions API, numba or
passlib-compatible bcrypt. Face nodes import lazily and name the platform
when used, the image map falls back to a PCA projection without umap-learn,
password hashing calls bcrypt directly, bitsandbytes guards survive a build
whose native library fails to load, and startup logs the torch build.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NiaFTnYH9q9468Naf4PV3X
check_aarch64_lock.py becomes check_platform_locks.py: the aarch64 check is
unchanged, and win32/ARM64 additionally requires torch from NVIDIA's index
and a win_arm64 wheel for every package in the closure. python-tests gains
a 3.12-only windows-11-arm lane.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NiaFTnYH9q9468Naf4PV3X
pyproject forks the packages whose Windows ARM64 wheels start at newer
versions, routes torch to NVIDIA's out-of-tree index and OpenCV, blake3 and
tiktoken to the release-hosted wheel page on that platform, and enforces
ARM64 wheel availability with uv's required-environments. Docs cover the
native ARM64 Python request, the install options and the feature differences.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NiaFTnYH9q9468Naf4PV3X
MSVC has no __fp16 type, which the DNN Winograd kernel uses once NEON_FP16
is in the CPU baseline; pin the baseline to NEON with DOTPROD dispatch.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NiaFTnYH9q9468Naf4PV3X
lstein and others added 6 commits September 14, 2026 22:55
Adds the win32/ARM64 resolution: torch 2.14.0+cu134 from NVIDIA's index,
the release-hosted opencv/blake3/tiktoken wheels, and the ARM64-capable
versions of numpy, onnx, onnxruntime, sentencepiece, bitsandbytes,
cryptography and PyWavelets. Other platforms change only by bcrypt 5.0.0
replacing passlib.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NiaFTnYH9q9468Naf4PV3X
Skip the woff2 font test where brotli is absent, name the platform in the
MediaPipe error only on Windows ARM64, cover transitive/extra/sdist paths
in the lock checker tests, and correct the platform docs.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NiaFTnYH9q9468Naf4PV3X
Pull-request runs of the wheel workflow only build and verify; publishing
is dispatch-only under a concurrency group, and tags are treated as
immutable because the lockfile pins the assets by hash. The closure walk
now follows a package's extras even after a plain edge reached it, the
ARM64 test lane skips uv's cache, and the umap fallback is a cached loader.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NiaFTnYH9q9468Naf4PV3X
The windows-arm64 lane imports the self-built wheels explicitly (tiktoken is
never imported by the suite), and the platform lock checker walks the test
extra the lane installs and reports edges the lock cannot resolve.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NiaFTnYH9q9468Naf4PV3X
NVIDIA's ARM64 torch dies with an access violation in
torch.cuda.is_current_stream_capturing() when no CUDA device is present,
which transformers calls on every forward pass. Answer False there, at app
startup and in the test session. Also split the SDPA pass-through test:
torch 2.14 rejects an explicit mask together with is_causal=True.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NiaFTnYH9q9468Naf4PV3X
@lstein

lstein commented Sep 17, 2026

Copy link
Copy Markdown
Collaborator Author

Installation and testing instructions: https://claude.ai/artifact/WP7PPdt7EutshYHLi92uKj

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants