Skip to content

Harden eval-sensitive tests: retry racy group kill, hermetic worktree fixture - #146

Merged
w4ffl35 merged 1 commit into
masterfrom
fix/rsi-eval-harness-flakes
Oct 13, 2026
Merged

w4ffl35 merged 1 commit into
masterfrom
fix/rsi-eval-harness-flakes

Conversation

@w4ffl35

@w4ffl35 w4ffl35 commented Oct 13, 2026

Copy link
Copy Markdown
Member

Two more RSI baseline blockers found by reading the fleet eval artifacts (run ...00abea81 got to 118/194 files):\n\n1. muse-executor group-kill EPERM: kill(-pgid) can throw for a transient group under load (one strike in a loaded eval sandbox, 10/10 clean container re-runs). Retry briefly with backoff; the muse-is-dead invariant below still fails loudly if the worker survives a signalled group.\n2. run-tests-script worktree test requires the ambient checkout to be a git repo (git worktree add), but eval snapshots extract files without .git — deterministic failure when reached. The fixture now builds its own throwaway parent repo (git init) with an untracked node_modules symlink, preserving the exact walk-up scenario and no longer mutating the real repo's worktree list.\n\nBoth verified locally (5/5 and 12/12) plus tsc clean; full-suite container repro in progress.

@w4ffl35
w4ffl35 force-pushed the fix/rsi-eval-harness-flakes branch 2 times, most recently from d42ee49 to c2c5d34 Compare October 13, 2026 09:12
@w4ffl35
w4ffl35 force-pushed the fix/rsi-eval-harness-flakes branch from c2c5d34 to 75382a0 Compare October 13, 2026 09:12
@w4ffl35

w4ffl35 commented Oct 13, 2026

Copy link
Copy Markdown
Member Author

Note: PR title/body edits are failing (GitHub GraphQL projects error), so the title is stale — the branch now also includes fix 3 (openshell-provider reap TOCTOU: treat a stat-vanished-between-probe-and-read as terminal, same as ESRCH). Commit message is accurate. All gates green locally.

@w4ffl35
w4ffl35 merged commit b2ed8f5 into master Oct 13, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant