feat(bootstrap)!: source hyperd from the PyPI tableauhyperapi wheels

[StefanSteiner]

Motivation

hyperdb-bootstrap fetched hyperd from Tableau's Hyper Java API zip, whose
filename embeds an opaque build_id (e.g. r07abb490) that cannot be derived
from the version. Discovering it meant scraping the public releases page — and
that scraper (src/scrape.rs, behind --latest) has been broken for three-plus
releases without anyone noticing, because its tests ran against a synthetic
fixture rather than the live page. Two independent defects:

• the heading regex expects <h3>VERSION [DATE]</h3>, but Docusaurus renders
  0.0.26479 <!-- -->[September 3 2026], which \s* cannot span;
• the build-id capture hardcodes (rc[a-z0-9]+), while every build id since
  0.0.24457 has been r + hex.

The PyPI tableauhyperapi wheels carry the same engine behind a fully
constructible URL, and PyPI publishes a sha256 per file:

https://files.pythonhosted.org/packages/py3/t/tableauhyperapi/tableauhyperapi-{version}-py3-none-{wheel_tag}.whl

So --latest is deleted, not fixed: with a constructible URL and published
digests there is nothing left for it to do.

The bytes are unchanged

The strongest evidence this migration is faithful — the hyperd extracted from
the macosx_13_0_arm64 wheel is bit-identical to the hyperd in the Java zip
for the same release, sha256
aef5c81970bb4d84d06fb9513d5ffd722526fce779632a0c5f63d87b6450e478. To be precise
about what is and isn't established here: that cross-envelope equality was
established upstream of this PR and I did not re-derive it. What I verified
locally is that the wheel path produces exactly that binary:

$ shasum -a 256 .hyperd/current/hyperd
aef5c81970bb4d84d06fb9513d5ffd722526fce779632a0c5f63d87b6450e478

$ stat -f%z .hyperd/current/hyperd
277836448

$ file .hyperd/current/hyperd
.hyperd/current/hyperd: Mach-O 64-bit executable arm64

$ .hyperd/current/hyperd --version
Hyper version main.0.0.26479.r96880f6a

Same build, different envelope. Note the --version line corroborates this
independently: the wheel's binary self-reports build r96880f6a, which is exactly
the build_id the Java zip carries for 0.0.26479 (see chore/post-rc-cleanup's
8c99d20). It is the same engine build reached by a different filename.

This is not a claim that the wheel matches the engine currently on main
(0.0.26359, build r07abb490) — that is a different build. The equality is
same-release, cross-envelope.

Both binaries report minos 13.0, so the macosx_13_0 wheel tag is not a
raised support floor — no contributor loses support. Wheels are ~3.6–4.5% smaller
than the Java zips.

Bumping the pin no longer means downloading four ~80 MB archives and hashing them
by hand — the digests come off the JSON API. They are still committed: a hash
in git is an attestation independent of the host serving the bytes.

Breaking changes

Public API removed:

Removed	Replacement
PinnedRelease::build_id, InstalledHyperd::build_id	.version is the only release identifier
PinnedRelease::version_tag()	.version
VersionSource::ScrapeLatest, the scrape module	none — deleted
Error::Http, Error::HttpStatus, Error::ScrapeFailed	none — served the scraper only
CLI --latest, --build-id	--version X alone is now a complete source
regex, reqwest, rustls deps	none — no in-process HTTP client remains

url::build_download_url is now fallible (Result<String, Error>); a platform
with no pinned wheel tag is Error::MissingWheelTag rather than a guess.
PinnedRelease::wheel_tag_for(Platform) is new.

The install layout is keyed on the version alone: <dest>/0.0.26479/ (was
<dest>/0.0.26479.r96880f6a/), and current/VERSION now contains just
0.0.26479. Nothing in the repo reads that file programmatically.

Dropping reqwest/rustls also retires the rustls crypto-provider workaround
the CHANGELOG records as a past breaking change (rustls-no-provider plus a
OnceLock installing ring). Verified aws-lc-rs is absent workspace-wide and
that hyperdb-bootstrap no longer reaches reqwest, rustls, or regex at all.

Versioning: breaking, released as 1.0.0-rc.2

Decided — no reviewer action needed. The break is now marked honestly and
the version is pinned, which are two independent things:

• The subject is feat(bootstrap)!: and the migration commit carries a
  BREAKING CHANGE: footer enumerating every removed item. The break is real
  and is recorded as such.
• An empty follow-up commit, chore: release 1.0.0-rc.2, carries a
  Release-As: 1.0.0-rc.2 footer, which pins the next release inside the
  1.0.0-rc line.

Without that footer the ! would compute 2.0.0: the workspace is already on
1.x, and bump-minor-pre-major in release-please-config.json is gated on
version.isPreMajor (major < 1), so it does not apply at 1.0.0-rc.1.
Release-As: wins regardless — release-please's DefaultVersioningStrategy
returns a CustomVersionUpdate from the RELEASE AS note before it reads the
breaking-change tally. This is the same mechanism, and the same empty-commit
shape, that produced 1.0.0-rc.1 (7bf2dff), and it is what
docs/GITHUB_OPERATIONS.md → Pre-releases
documents.

Simulated against release-please@17.11.2 (the version
release-please-action@v5 pins) using its real parseConventionalCommits and
DefaultVersioningStrategy: 1.0.0-rc.1 → 1.0.0-rc.2 under a merge
commit and under a squash merge. Dropping the footer from the same input
yields 2.0.0-rc.1, which is the outcome this pin exists to prevent.

Do not merge a release PR that says anything other than 1.0.0-rc.2.
#253 merges first, and release-please will run on that push and open a
chore(main): release … PR computed from #253's commits alone — before this
PR's Release-As: footer is on main. That PR will show the wrong version.
It updates itself once this PR lands; leave it alone until then.

Why wheel tags live in the pin file

They are deliberately not hardcoded in Rust. They are not guaranteed stable
across releases (arm64 wheels only exist from 0.0.19484; a future macOS floor
bump would change macosx_13_0_arm64), and a wrong tag is a silent 404 on one
platform only. Keeping them as pin data makes any such change a visible pin edit,
and build.rs now fails the build if any supported platform lacks one.
(Empirically the four tags are unchanged from 0.0.19484 through 0.0.26479.)

Stronger verify

hyperdb-bootstrap verify now cross-checks every pinned digest against the digest
PyPI publishes for that exact wheel filename, on top of HEAD-ing the four URLs —
so it validates the exact pinned bytes rather than merely that the CDN serves
something at that path, and it catches a stale [wheel_tag] explicitly:

$ cargo run --release -p hyperdb-bootstrap --bin hyperdb-bootstrap -- verify
verifying hyperd 0.0.26479...
  OK    macos-arm64      [200] .../tableauhyperapi-0.0.26479-py3-none-macosx_13_0_arm64.whl
        digest matches PyPI
  OK    macos-x86_64     [200] .../tableauhyperapi-0.0.26479-py3-none-macosx_10_11_x86_64.whl
        digest matches PyPI
  OK    linux-x86_64     [200] .../tableauhyperapi-0.0.26479-py3-none-manylinux2014_x86_64.whl
        digest matches PyPI
  OK    windows-x86_64   [200] .../tableauhyperapi-0.0.26479-py3-none-win_amd64.whl
        digest matches PyPI
all platforms reachable with matching digests.

npm dual pin

npm-build-publish.yml carries its own independent pin (this is why 0.7.1 once
shipped npm with a different engine than crates.io). It is migrated to the wheel
URL, HYPERD_BUILD_ID is gone, and the matrix gains hyperd-wheel-tag.
verify-npm-hyperd-pin.py keeps its version + digest cross-check and gains a
wheel-tag cross-check, since the tag is a new drift vector.

Empirically confirmed (not assumed) that the matrix hyperd-sha256 values are
digests of the downloaded archive, not the extracted binary — the step hashes
hyperd-archive.whl — so they are the wheel digests and equal the toml's
[sha256].

I did not make the workflow read the toml directly. The matrix must be static
YAML, so it would need a tomllib call inside the per-platform matrix job, which
runs on a Windows runner under git-bash where python3 may not be on PATH — an
untestable portability risk for no correctness gain over the guard. Recommended
as a follow-up alongside converting the matrix to a setup-job JSON output.

Verification

Run on macOS arm64 (Apple Silicon):

• cargo build -p hyperdb-bootstrap and cargo build --workspace — clean
• cargo test -p hyperdb-bootstrap — 26 lib + 4 integration + 1 doc, 0 failed
  (up from 12 lib + 2 integration; +16 runnable tests)
• cargo test --workspace — 1584 passed, 0 failed, against the freshly
  downloaded engine
• make test — 1519 passed, 0 failed
• cargo fmt --all -- --check and
  cargo clippy --workspace --all-targets --all-features -- -D warnings — clean
• npx markdownlint-cli2 — zero new findings; the 5 in AGENTS.md /
  DEVELOPMENT.md are the pre-existing MD040 backlog, identical before and after
• verify-npm-hyperd-pin.py passes against the new toml, and fails correctly
  when a wheel tag is perturbed
• The workflow's find-based extraction was replayed against the real wheel entry
  names: HYPER_DIR=hyperd-raw/tableauhyperapi/bin/hyper, hyperd found,
  LICENSE*/NOTICE* glob matches dist-info/

extract.rs is unchanged apart from doc comments — its "skip one optional
top-level directory, then require a lib/hyper or bin/hyper pair" logic already
absorbs the tableauhyperapi wrapper. New tests pin that against the real wheel
entry names, including the Windows hyperd.exe + crashdumper.exe case.

Not verified here

• Only macOS arm64 was executed. The other three platforms rest on CI. Their
  URLs and digests are machine-verified (all four HEAD 200 with matching PyPI
  digests), and I downloaded the Windows and Linux wheels to confirm their
  bin/hyper/ contents (hyperd.exe + crashdumper.exe; hyperd) and digests —
  but no hyperd was run on Linux or Windows.
• The engine links only system frameworks (otool -L) and bin/hyper/ contains
  no shared libraries on any of the three wheels I inspected, so the npm
  shared-library copy loop finds nothing to copy — as before.

---

Repeated here as a trailing footer so the version is pinned on the merge commit
too: this repo merges with merge_commit_message = PR_BODY, so the body below
becomes the merge commit's body. Same value as the footer on
chore: release 1.0.0-rc.2, so it is redundant rather than conflicting.

Release-As: 1.0.0-rc.2

[StefanSteiner]

Merge note: overlap with chore/post-rc-cleanup

That branch is landing an independent hyperd bump to the same version
(8c99d20 fix(bootstrap): bump pinned hyperd to 0.0.26479 (r96880f6a)), so the two
branches overlap in 12 files, not just the pin. Worth knowing before merging:

• hyperdb-bootstrap/hyperd-version.toml — trivial. Both set
  version = "0.0.26479"; that branch adds build_id = "r96880f6a", this one deletes
  build_id outright. Take this branch's file.
• hyperdb-bootstrap/CHANGELOG.md — both add bullets under one ## [Unreleased].
  Keep both; merge into the single existing ### Changed rather than creating a
  sibling (MD024).
• src/error.rs, src/lib.rs, src/bin/hyperdb-bootstrap.rs, tests/integration.rs —
  that branch's 0cc65e2 fixes five stale hyperd-bootstrap → hyperdb-bootstrap
  references in these files, which this PR also rewrites. I made the identical edit
  here (33ae1e0) so the merge resolves to the correct text whichever side wins,
  instead of risking a resolution that silently reverts that fix.
• src/release.rs, AGENTS.md, DEVELOPMENT.md, Makefile, both SKILL.md copies —
  mechanical; this PR's rewrites supersede, but re-check that
  a46f551 docs(agents): require running markdownlint survives in AGENTS.md.

On the benchmark tracker

docs/hyperd-release-benchmarks.md gets no row from this PR, deliberately.
Per AGENTS.md, a row is owed for an engine bump or a material API change, and
conflating the two makes a delta unattributable. This migration is provably
byte-identical at a fixed version (sha256 aef5c819…e478), so it moves no
engine variable — the 0.0.26359 → 0.0.26479 movement and its A/B numbers belong
to the bump PR.

[StefanSteiner]

Follow-up: LICENSE-HYPERD selection is traversal-order dependent

Recording a small behavior change this PR causes that I am deliberately not
fixing here, so it isn't lost.

The "Extract LICENSE from hyperd archive" step picks its file with a single
combined glob:

LICENSE_FILE=$(find hyperd-raw -iname "LICENSE*" -o -iname "NOTICE*" | head -1)

The wheel's tableauhyperapi-<version>.dist-info/ contains LICENSE,
NOTICES.txt, and HYPER_API_OSS_disclosure.txt, so more than one candidate
matches and head -1 takes whichever the traversal reaches first. Replaying the
step locally against the real macOS wheel returned NOTICES.txt, not
LICENSE. Since each platform runner traverses its own archive independently, the
LICENSE-HYPERD shipped inside the npm packages may be a notices file, and may
differ between darwin-arm64, linux-x64-gnu, and win32-x64-msvc.

The find expression itself is untouched by this PR, but the archive it reads is
new, so the outcome is attributable here rather than pre-existing. It is a
compliance-artifact nit, not a break: a legitimate legal notice still ships, and
the heredoc fallback means a total miss is non-fatal.

Suggested fix (deliberately left out of this PR — a two-pass preference, so
LICENSE wins deterministically and NOTICE* remains the fallback):

LICENSE_FILE=$(find hyperd-raw -iname "LICENSE*" | head -1)
if [[ -z "$LICENSE_FILE" ]]; then
  LICENSE_FILE=$(find hyperd-raw -iname "NOTICE*" | head -1)
fi

I held it back for two reasons: it is orthogonal to the wheel migration and would
re-run the full matrix on an already-green PR, and my tooling blocks edits to
GitHub Actions workflow files pending human review. Worth folding into the same
follow-up that considers having the workflow read hyperd-version.toml directly.