Skip to content

rerun-sdk: add build-rerun-sdk.yml for riscv64 wheels - #1085

Merged
luhenry merged 9 commits into
mainfrom
rerun-sdk
Sep 8, 2026
Merged

rerun-sdk: add build-rerun-sdk.yml for riscv64 wheels#1085
luhenry merged 9 commits into
mainfrom
rerun-sdk

Conversation

@luhenry

@luhenry luhenry commented Sep 6, 2026

Copy link
Copy Markdown
Member

Compiles the rerun_bindings PyO3 extension (rerun_py/) against the rerun workspace's ~760 riscv64 crates (datafusion, lance, tonic/tokio, hdf5-pure). Upstream publishes no riscv64 wheel.

Mirrors upstream's reusable_build_and_upload_wheels.yml MODE=pr build (its own default Cargo features).

Differs from upstream

  • Ships no rerun-cli binary - the wgpu/egui native viewer is a separate ~16-core job needing a wasm/JS toolchain, out of scope here; RERUN_ALLOW_MISSING_BIN=1 skips build.rs's requirement for it, so import rerun/logging/gRPC work but rerun.spawn()/serve() raise instead of launching a viewer.
  • yum install protobuf-compiler protobuf-devel + PROTOC_INCLUDE=/usr/include - lance needs protoc at build time; upstream's wheel-packaged protoc has no riscv64 build.
  • --no-build-isolation - build.rs refuses to run inside pip's isolated build venv.

Matrix: cp312/cp313/cp314 only - our registry has no riscv64 pyarrow (a hard runtime dependency) for cp310/cp311, the abi3 floor pyo3 declares.

Testing

  • Skips tests/e2e_redap_tests and tests/api_sandbox (upstream sandboxes, not the SDK's regression tests) and tests/integration/test_headless_viewer.py (spawns the unbuilt rerun-cli)
  • Skips a handful of files needing av/torchvision/datafusion(py)/polars, none of which have riscv64 wheels anywhere

License: OK

Built on cp312; results pending.

luhenry added a commit that referenced this pull request Sep 6, 2026
rerun's own .cargo/config.toml unconditionally points PYO3_CONFIG_FILE
at rerun_py/pyo3-build.cfg for every cargo invocation, expecting a pixi
activation hook to have written it first; outside pixi the file never
exists and pyo3-build-config's build script fails to open it.
lance-core 9.0.0's SIMD_SUPPORT closure has cfg arms for
aarch64/x86_64/loongarch64 with no catch-all, so it evaluates to () instead
of SimdSupport on riscv64 and fails to compile (E0308). No compatible
9.0.x release fixes it, so a riscv64-fixed copy of the crate is vendored
from its crates.io tarball at build time and wired in via
[patch.crates-io].
crates.io's API returns 403 for requests with no User-Agent header.
lance-linalg 9.0.0's f32x8/f32x16/f64x4/f64x8/i32x8 SIMD wrapper types
are defined only for aarch64/x86_64/loongarch64, with unconditional
trait impls referencing them, so the crate does not compile at all on
riscv64 (E0425: cannot find type). Adds a portable scalar fallback for
each type, mirroring the crate's own u8x16 fallback shape (which
already had one). Verified against the crate's own SIMD unit test
suite by temporarily disabling the aarch64 arms on a native aarch64
host, forcing the new fallback path to run for real: all 24
lance_linalg::simd tests pass.
…_bindings package

rerun_bindings/ is a mixed Python/Rust package (__init__.py re-exports
from the nested compiled submodule, maturin's default layout when a
python-source directory matching the crate name exists), so
rerun_bindings.__file__ is always __init__.py; the compiled .so lives
at rerun_bindings.rerun_bindings.__file__.
…mports)

Run 6 got past the compile and the compiled-extension smoke check and
into pytest collection, surfacing three independent issues:
- test_asset3d.py reads a workspace-level tests/assets/mesh/cube.glb
  fixture that CIBW_TEST_SOURCES didn't stage.
- test_readers.py's @pytest.mark.local_only is normally registered by
  tests/e2e_redap_tests/conftest.py, which pytest never loads once
  that directory is excluded; filterwarnings=error turns the resulting
  unknown-mark warning into a collection failure. -o markers=local_only
  registers it directly.
- Four more unit/integration test files reach
  rerun.experimental.dataloader's decoders, which import torchvision or
  datafusion (both already dropped from CIBW_TEST_REQUIRES, no riscv64
  wheel anywhere) -- missed by the earlier direct-import grep since
  they pull the dependency in transitively.
…he real test suite

Run 7 collected cleanly (941 passed, 70 failed, 14 errors) and revealed
three independent, unrelated problems:
- HDF5/MP4 test fixtures are Git LFS pointers without lfs: true on the
  checkout, fed straight into their native parsers ('HDF5 signature not
  found' / 'MP4 ... box with a larger size than it').
- Three more fixture paths outside rerun_py/tests needed staging:
  crates/store/re_importer/.../tests/assets (mcap), examples/rust/
  animated_urdf/data (urdf), and the repo-root Cargo.toml itself
  (test_version.py parses its own version out of it).
- A handful of individual tests need the unbuilt rerun-cli binary
  (rrd merge/compare), av (PyAV), or the datafusion Python package
  (CatalogClient) -- ignored/deselected alongside the same three
  reasons already excluded elsewhere in this suite.
…timing flake

Run 8 collected and ran the whole suite cleanly: 983 passed, 9 skipped,
5 deselected, only test_multiprocessing_gc failing. Its own comment
says torch.multiprocessing (used because torch is on CIBW_TEST_REQUIRES)
'causes more issues' than stdlib multiprocessing for this fork+gc.collect()
race; it consistently hits the test's hardcoded 5s join() timeout on this
runner. Matches gotcha 14's torch-flake shape exactly (a slower-runner
hard timeout, not a build bug) -- deselect, don't chase.
@luhenry
luhenry merged commit 2a36666 into main Sep 8, 2026
6 checks passed
@luhenry
luhenry deleted the rerun-sdk branch September 8, 2026 14:08
@luhenry luhenry linked an issue Sep 8, 2026 that may be closed by this pull request
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

rerun-sdk riscv64 support

1 participant