Repository navigation
fix(testing): test-judge gives up when the judge exits 0 with no usable result, and re-logs deleted files on every Stop #6621
Description
Two test-judge problems on Windows (melo-desk-001, 2026-10-06 and 2026-10-07), seen in ~/.claude/plugins/data/testing-melodic-software/test-judge.log.
1. The judge exits 0 with no usable result
The Stop hook reported: test judge: 1 test not judged after 2 failed attempts. The log shows:
2026-10-07T22:46:11Z attempt failed: 0614dba8d46a0f02a22340109b2f4f57: judge exited 0 with no usable result
2026-10-07T22:46:11Z not judged, the judge failed for: Provisioning.Tests.ps1: stops the leader and exits when its parent, the task
2026-10-07T22:46:39Z attempt failed: 0614dba8d46a0f02a22340109b2f4f57: judge exited 0 with no usable result
2026-10-07T22:46:40Z judge not run after 2 failed attempts for: Provisioning.Tests.ps1: stops the leader and exits when its parent, the task
The log has 12 no usable result lines over two days: 5 on 10-06 and 7 on 10-07. The judge's raw output is not kept, so the cause can't be determined from the log. In the logged test name, a Pester It description from melodic-software/provisioning, the name ends at a comma (...its parent, the task). That may be truncation of a name containing a comma, or only how the log line is written. This is not verified.
Expected:
- keep the judge's raw output, or an excerpt, for any attempt that produces no usable result, so the failure can be diagnosed
- if the test-name extraction cuts at a comma, fix it
2. Files deleted from the session are re-logged on every Stop
A file the session edited lives in a worktree that was later removed. After that, every Stop logs skipped: <path> no longer exists again for the same set of files. The current log has 2,257 such lines, and the same 11 paths repeat at 22:18, 22:20 and 22:46. pending/ is empty, so the entries come from session state, not the queue.
Expected: drop a missing file from the session's set once, on the first skip, so it is not re-checked or re-logged on later Stops.
Found during the 2026-10-07 C: disk incident cleanup, where many worktrees were removed after their PRs merged.
Activity
github-actions commentedon Oct 7, 2026
Thanks for the detailed report. Understood as two separate test-judge Stop-hook bugs: (1) when the judge process exits 0 but produces no usable result, the raw output isn't retained, so the two logged attempts give no way to diagnose why parsing failed, and the test name in the log may be truncated at a comma; (2) once a session-tracked file no longer exists (e.g. its worktree was removed), the hook re-logs a "no longer exists" skip for that same path on every subsequent Stop instead of dropping it from the session's set after the first skip, producing thousands of repeated log lines.
Both reproduction steps and expected behavior are clearly stated, so no missing info. Related but not duplicate: #6387 (test-judge comment-only check/reuse key edge cases) and the now-closed #6226 (test-judge Stop hook log flooding) cover adjacent but distinct symptoms.
Automated triage. A maintainer reviews it.
kyle-sexton commentedon Oct 11, 2026
Draft PR: #7091 (branch
fix/6621-test-judge-no-result-and-missing-files).verdicts/<pkey>/<sid>/unusable-<epoch>-<pid>.txt, andtest-judge.logquotes the start of the result and of stderr.'',`"or"", or a backslash-escaped quote elsewhere. On a Pester file, the scanner reproduced the exact shape of the logged name (...its parent, the task). That is fixed now. I could not read the real provisioning test name from here, so it is not proven that this caused all 12 failures. The kept output will show the cause next time.scripts/affected-tests.sh --base origin/main --runpassed all 10 shell suites, and the one node suite passed when run on its own. New tests cover each of the three fixes.For a human: review the PR, then mark it ready when you're satisfied.
Generated by Claude Code
kyle-sexton commentedon Oct 11, 2026
Handoff note for whoever picks this up next. The overnight Claude run has stopped, and Kyle is moving issue work to Cursor.
fix/6621-test-judge-no-result-and-missing-filesWhere a branch is listed it is pushed, so it can be taken over as is. A draft PR shows a red
ci-statusby design until it is marked ready.Generated by Claude Code