Repository navigation
Re-arm tripwire flake: onCommandComplete fires before console.read() re-arm lands, so a synchronous handler-count assert is racy #694
Description
Summary
Quarkus CI flaked once on the aesh 3.18.6 / readline 3.18.7 bump: AeshLauncherApiTest.accumulatedOutputAcrossCommands failed with the re-arm tripwire reporting only 1 handler replacement since the previous completion (expected >= 2) — even though the REPL was alive, parked in openBlocking, and rearmedSinceLastCompletion=true.
Verdict after analysis: false positive from a race in the Quarkus test harness, not a missed re-arm and not a 3.18.6 regression. Filing here to record the analysis and to ask whether upstream wants a deterministic ordering guarantee (see below).
Where / when
- Update aesh extension to aesh 3.18.6 and aesh-readline 3.18.7 quarkusio/quarkus#57716 (bump to aesh 3.18.6 / readline 3.18.7), commit
c37c159, run40009618863, jobJVM Integration Tests - JDK 21 - Previous BOM (3.18.4 / 3.18.5) was green across all aesh suites; this was a single failure out of 23 tests in the class
- Failure excerpt (0.525 s, i.e. no hang):
java.lang.AssertionError: Command 'hello --name=Second' completed but readline was not re-armed afterwards: 1 handler replacement(s) since last completion (expected >= 2 or console close). Diagnostics (re-arm tripwire): replAlive=true, replState=WAITING, queuedSignals=0, readerDeath=null, lastArmNanos=197132876545, closeNanos=0, rearmedSinceLastCompletion=true, threads=19 [aesh-test-repl(WAITING): Unsafe#park <- ... <- StreamConnection#openBlocking <- ReadlineConsole#accept ... aesh-stream-reader(RUNNABLE): ProcessManager#drainWhilePending <- ProcessManager#requestDrain <- ProcessManager#execute <- ReadlineConsole#processLine]
The follow-up rerun error in the same fork (REPL did not start... replAlive=false TERMINATED) is cascade fallout, not a second bug.
Analysis
Completion-to-signal order in 3.18.6 (CommandJob.finish(), CommandJob.java:420-428):
manager.processFinished(job)— clearsactiveJob, fires pipeline events,requestDrain(). The post-completion re-arm (console.read(),ProcessManager.java:461) runs inside that drain turn: inline on the worker thread if it wins the turn, or deferred to another thread.- Only then
executionListener.onCommandComplete(...)— the signal the Quarkus harness (AeshLauncherImpl.execute()) waits on before asserting the handler count synchronously.
So whether the current command's re-arm has landed at assert time is a thread race: healthy runs show a delta of 2 (submit stub + fast inline re-arm) or 1 (stub only, re-arm still in flight — it lands moments later, as proven here by WAITING in openBlocking plus rearmed=true at diagnose time). The >= 2 synchronous assertion is racy by construction.
Not a 3.18.6 regression: git diff v3.18.4..v3.18.6 shows no ProcessManager/readline read-path change (only context/completion/cursor-listener work), and the readline v3.18.5..v3.18.7 diff touches no handler logic. A timing shift merely exposed the race.
Proposed plan
- Harness-side fix in Quarkus (no upstream change required): replace the immediate assert in
AeshLauncherImpl.execute()with a bounded settle wait (up to ~2 s, short poll) forarmCount - baseline >= 2or console close; fail with the existing message + diagnostics only on timeout. Keep the threshold at 2 — relaxing to>= 1would mask a genuine Second command over piped streams never completes (3.18.1 regression) #634-style skip, since the submit stub alone already yields 1. - Verify by looping
AeshLauncherApiTestlocally (~20x) on 3.18.6, then via CI. - Optional upstream question: would you accept a guarantee that
onCommandCompletefires strictly after the post-completionconsole.read()re-arm lands (or a dedicated re-arm hook), so downstream harnesses can assert re-arm deterministically instead of polling? Happy to implement the harness-side wait regardless; this would just remove the need for it.
Activity
stalep commentedon Oct 24, 2026
Verified the mechanism from this side — the analysis is correct.
CommandJob.finish()callsmanager.processFinished(job)(which only requests the drain turn) and then firesonCommandCompletesynchronously, while the re-arm (console.read()) lands whenever the drain turn runs, possibly on another thread. So at listener time the re-arm provably may not have landed: racy by construction, not a regression. (Also checked: no scheduling/read-path changes inaesh/.../consolesince v3.18.5 that could have introduced this — only the #689 context selection and a test-only seam.)On the plan:
>= 2threshold — relaxing to>= 1would indeed mask a Second command over piped streams never completes (3.18.1 regression) #634-style skip, since the submit stub alone already yields 1. Expect recurrence on loaded runners until it's in; with a race this tight, no code change on either side is needed to flip it.On the upstream question (ordering guarantee): I'd advise against it, for one contractual reason and one architectural one.
CommandExecutionListenerdocuments "Called after each command finishes execution, before the prompt is redisplayed." A guarantee that the listener fires strictly after re-arm lands would contradict the documented contract — it would need a contract change, not just an implementation tweak.The race is inherent: an async completion signal vs. a subsequent re-arm that may run on another thread cannot be made deterministic for a synchronous observer without a rendezvous. The bounded wait belongs in the asserter, which is exactly plan item 1.
If a second consumer ever needs determinism (so far this is test-instrumentation-only — production consumers are all result-oriented, not re-arm-oriented), I'd rather add an explicit, opt-in re-arm signal than reorder the existing event. Not needed now.