fix(gc): report live arena bytes after collection (#7879)

[proggeramlug]

Closes #7879

What changed

• Publish a header-inclusive arena live-allocation census from the existing full/non-moving sweep walk.
• Derive the copying-minor census from the previous live count minus from-space plus copied and promoted survivors, preserving promotions from earlier cycles without another object walk.
• Advance the census between collections from bump high-water growth and general/old free-list consumption, without adding work to the generated inline allocator.
• Route process.memoryUsage().heapUsed, compatible V8 telemetry, and both sides of major-GC pacing through the live metric.
• Keep reserved capacity and allocation high-water as separate fragmentation diagnostics; diagnostic snapshots now expose total_live_allocated_bytes alongside total_in_use_bytes.

Reproduction and tests

The regression test applied unchanged to clean main reports:

heapUsed=1919928 high_water=1919928 heapTotal=2097152

A tiny rooted survivor therefore charges the whole partially dead block. With this branch, the focused accounting suite proves:

• one tiny survivor leaves at least half a block between high-water and heapUsed;
• bump allocation and exact old-hole reuse advance the census byte-for-byte;
• copying minors preserve the census through survivor copies, promotion, and a later cycle after promotion.

Validation:

• cargo test -p perry-runtime --lib gc::tests::heap_accounting: 3 passed
• RUST_TEST_THREADS=1 cargo test -p perry-runtime --lib: 2,128 passed, 4 ignored
• cargo check -p perry-runtime --features diagnostics: passed
• TLS policy, address classification, test registration, rustfmt, diff, and file-size gates: passed

Quiet-host A/B

Seven repeats per probe on the locked M1 mini, with fresh 60-second quiet qualification for each arm (5.8% active for base, 4.8% for fix). Both arms used the same compiler and stdlib; only libperry_runtime.a differed. All 14 probes passed their Node oracle and produced byte-identical stdout.

• heapUsed fell by 59.2% to 99.99% across all 14 probes; heapTotal was unchanged in every probe.
• 12_large_live_set, which promotes a large cohort and then releases most of it: 51,457,736 -> 4,462,040 bytes (-91.3%), with identical GC counters, -0.05% wall, and -0.09% RSS.
• Collector counters and timing were unchanged on 13 probes.
• 13_large_eden_survivors performed one additional GC step: wall +7.5% (920.6 -> 989.9 ms), RSS -5.9% (205.6 -> 193.4 MB). This is the explicit pacing tradeoff of using live allocation rather than fragmentation as the major-collection signal.

The repository ratchet pin is already stale on the clean base. Its --check fails current-main cells including the known 14_grow_then_churn counter shift; the fix arm leaves those same #14 cells and turns every heap-used comparison into an improvement. No ratchet artifact is changed in this PR.

Summary by CodeRabbit

• Bug Fixes

  • Improved heap usage reporting to count live objects more accurately.
  • heapUsed now excludes reclaimed and reusable memory, while heapTotal remains unchanged.
  • Improved garbage-collection pacing and V8-compatible telemetry.
  • Updated accounting across minor collections, full collections, promotions, and reused free space.

• Diagnostics

  • Arena telemetry now includes total live allocated bytes.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: cf3cadf7-edba-4eed-b971-dee839a4019a

📥 Commits

Reviewing files that changed from the base of the PR and between 9ca8b4f and 83e29d4.

📒 Files selected for processing (12)

• changelog.d/7886-live-heap-accounting.md
• crates/perry-runtime/src/arena/mod.rs
• crates/perry-runtime/src/arena/stats.rs
• crates/perry-runtime/src/arena/walk.rs
• crates/perry-runtime/src/gc/copying.rs
• crates/perry-runtime/src/gc/cycle.rs
• crates/perry-runtime/src/gc/old_free.rs
• crates/perry-runtime/src/gc/oldgen.rs
• crates/perry-runtime/src/gc/policy.rs
• crates/perry-runtime/src/gc/telemetry.rs
• crates/perry-runtime/src/gc/tests/heap_accounting.rs
• crates/perry-runtime/src/gc/tests/mod.rs

---

📝 Walkthrough

Walkthrough

Heap accounting now uses header-inclusive live-object censuses for heapUsed, telemetry, and major-GC pacing. Arena high-water and reserved capacity remain separate metrics. Copying and sweeping collections publish live-byte counts, with tests covering dead bytes, hole reuse, and promotions.

Changes

Live heap accounting

Layer / File(s)	Summary
Arena live-byte accounting crates/perry-runtime/src/arena/*	Arena statistics record post-collection live bytes and derive subsequent allocation and hole-reuse changes. Heap usage uses this value, while reserved capacity remains separate.
GC census production crates/perry-runtime/src/gc/copying.rs, crates/perry-runtime/src/gc/cycle.rs, crates/perry-runtime/src/gc/old_free.rs, crates/perry-runtime/src/gc/oldgen.rs	Copying collections and old-generation sweeps calculate live arena bytes and publish censuses. Retained safety-window stubs do not inflate the live-byte total.
Telemetry and pacing integration crates/perry-runtime/src/gc/policy.rs, crates/perry-runtime/src/gc/telemetry.rs	Major-GC pacing and diagnostic telemetry use live allocated arena bytes.
Heap accounting validation crates/perry-runtime/src/gc/tests/heap_accounting.rs, crates/perry-runtime/src/gc/tests/mod.rs, changelog.d/7886-live-heap-accounting.md	Tests cover dead high-water bytes, old-generation hole reuse, copying collections, and promotions. The changelog documents the new accounting model.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant GCCollection
  participant OldgenSweep
  participant ArenaStats
  participant GCPacing
  GCCollection->>OldgenSweep: sweep and count live arena bytes
  OldgenSweep->>ArenaStats: publish arena_live_bytes
  GCCollection->>ArenaStats: record post-collection live census
  ArenaStats->>GCPacing: provide live allocated arena bytes

Loading

Possibly related PRs

• PerryTS/perry#7443: Both changes refine old-generation hole accounting and its interaction with heap metrics.
• PerryTS/perry#7613: Both changes modify GC sweep census data and SweepTraceStats propagation.
• PerryTS/perry#7869: Both changes modify post-collection occupancy readings in GC cycle and policy code.

Suggested reviewers: thehypnoo

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly identifies the main change: reporting live arena bytes after garbage collection.
Description check	✅ Passed	The description covers the change, linked issue, tests, validation results, and performance impact, although it uses non-template headings.
Linked Issues check	✅ Passed	The changes address the linked issue objectives for live accounting, telemetry, GC pacing, diagnostics, and regression coverage.
Out of Scope Changes check	✅ Passed	The documented code, telemetry, diagnostics, tests, and changelog changes are all related to the linked heap-accounting objectives.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch fix/7879-live-heap-accounting

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}