fix(developer-rust): bound the pool by free space - #95
Merged
Merged
Conversation
A dedicated 512G cache volume with several concurrent build sessions still reached 96% with both prune units exiting 0, because the `auto` ceiling (a sixth, 85G) sat under the day's working set and the hourly guard only ever pruned back to that ceiling. The guard now runs every five minutes and, below `rust_cache_prune_free_floor` (15%), evicts the least-recently-built workspaces until `rust_cache_prune_free_target` (25%) is free, past the ceiling if need be. On a pool with a filesystem of its own, `auto` now means no ceiling and free space bounds it. No run evicts a workspace whose cargo build-dir lock is held, and the prune holds that lock through the delete. If the pool alone cannot reach the target and Docker's data root is on the same filesystem, the guard runs `docker builder prune` and then `docker image prune` as the user, never with sudo (`rust_cache_prune_docker`, `--no-docker`). hyperi-doctor also stops reporting a worktree or submodule checkout as not a git checkout, since its `.git` is a file there.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The pooled Rust build cache could still fill its disk. A dedicated 512G cache volume with several concurrent agent build sessions hit 96% while both prune units exited 0.
Why
autoset the ceiling at a sixth of the volume (85G), under a working set of around 310G. Each nightly prune evicted live workspaces and the rebuild refilled the disk.What changes
rust_cache_prune_free_floor(15%, was 20%) the guard drops the least-recently-built workspaces untilrust_cache_prune_free_target(25%) is free, past the ceiling if need be. New--free-target.OnBootSec/OnUnitActiveSec, launchdStartInterval300).rust_cache_prune_guard_scheduleis gone..cargo-build-lock(.cargo-lockin 1.91) in each profile directory for the whole build (src/cargo/core/compiler/layout.rs). The prune takes it non-blocking and holds it through the delete.automeans no ceiling when the pool's st_dev differs from$HOME's. An explicit size still wins.docker builder prune -f --filter until=72h, thendocker image prune -af --filter until=168h. As the user, never sudo, neverdaemon.json.rust_cache_prune_docker/--no-dockerswitch it off..gitis a file there). That failed a doctor test in any worktree.How it was verified
tools/ci/run-tests.shandhyperi-ci checkpass: 97 pytest, syntax check, shellcheck.docker. Eight mutants, one per design point, each fail a test.systemd-analyze verifyclean on the rendered guard units.--checkon a real dedicated-volume pool detected the separate filesystem, kept a workspace a runningcargo checkhad locked, and listed the docker commands it would run.What a user sees
autopool ceiling on a dedicated cache volume.Existing hosts pick it up with
./install.sh --tags rust-cachefrom an updated checkout.