diff --git a/docs/release-notes/0.1.3.md b/docs/release-notes/0.1.3.md new file mode 100644 index 0000000..4dd40c2 --- /dev/null +++ b/docs/release-notes/0.1.3.md @@ -0,0 +1,72 @@ +# ComputeArena CLI 0.1.3 + +This release fixes benchmarking with BaseRT 0.2.5. On 0.1.2, every default +run with the released BaseRT 0.2.5 ended, after the benchmark had completed, +with `BaseRT returned incompatible headline measurement metadata`, and no +report was saved. Nothing else changes. + +## Install or upgrade + +On macOS or Linux: + +```sh +curl -LsSf https://computearena.ai/install.sh | sh +``` + +The same command upgrades an existing installation. Saved reports, the +installation signing key, login sessions, model receipts, and installed +runtimes are kept. Release archives and checksums are also available below for +manual installation. + +BaseRT 0.2.5 itself needs no change. If BaseRT is not installed yet, or is +older than 0.2.5, `computearena basert install` fetches the current release. + +## What changed + +- **Runs with BaseRT 0.2.5 complete and submit.** BaseRT 0.2.5 runs each + prefill size at its own context capacity and records the largest one the + sweep used as the run's context size: 16384 for the default sweep, which + reaches PP16384. ComputeArena 0.1.2 expected the headline's 4096 there and + refused the run once it had finished. The client now expects the value + BaseRT 0.2.5 records. The measurement is unchanged: PP512 and TG128 still + run first in a freshly loaded model with a 4K reservation, every workload's + capacity, order, and repetitions are still checked before signing, and the + reports still normalize to `computearena-throughput/3`. computearena.ai + applies the same rule. + +## Reports and compatibility + +0.1.2 could not save a BaseRT 0.2.5 report, so there is nothing to resubmit +from it: run the benchmark again with 0.1.3. Reports saved by 0.1.0 to 0.1.2 +with BaseRT 0.2.4 or llama.cpp are unaffected and remain submittable when +they contain the full default sweep. Benchmarks already published on +computearena.ai are not changed. + +A BaseRT harness built from the development branch before the 0.2.5 release +reports version 0.2.5 but records the headline's capacity instead. 0.1.3 does +not sign its runs, and computearena.ai does not accept reports from it; +replace it with `computearena basert install`. BaseRT 0.2.4 keeps working as +before: its runs use the older protocol, the report records that, and the +client says so before the run. + +## Supported systems + +Prebuilt ComputeArena binaries are available for: + +- macOS on Apple Silicon; +- Linux x86_64 with glibc 2.31 or newer; and +- Linux arm64 with glibc 2.31 or newer. + +Windows and Intel Macs are not supported in this release. ComputeArena does not +bundle an inference runtime or model; install BaseRT or llama.cpp and obtain a +compatible model before benchmarking. Prebuilt BaseRT runtimes exist for macOS +on Apple Silicon and for Linux arm64 with CUDA; on Linux x86-64, use llama.cpp +or a BaseRT benchmark harness you built yourself. + +The macOS binary is not signed with an Apple Developer ID. The release archive +is checksum-verified and carries a Sigstore keyless signature from the release +workflow. Browser-downloaded copies may need their quarantine attribute removed +as described in the README. + +Accuracy benchmarks, MLX, vLLM, Windows, and Intel macOS support are not part +of v0.1.3.