fix(watch): coalesce rebuilds triggered during a burst of file changes - #14202
fix(watch): coalesce rebuilds triggered during a burst of file changes#14202glours wants to merge 3 commits into
Conversation
compose watch triggered one rebuild per debounced batch, so a burst of edits arriving faster than the debounce window (e.g. from a coding agent editing several files) queued up several rebuilds in a row instead of a single consolidated one. Rebuilds now run asynchronously through a small scheduler that coalesces requests received while a rebuild is in flight into one trailing rebuild, without adding latency to an isolated edit. This also closes a race where a plain restart could run concurrently with an asynchronous rebuild of the same service. Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
a840592 to
3e3f0eb
Compare
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
ndeloof
left a comment
There was a problem hiding this comment.
Solid work overall: the scheduler is minimal and correct (decision taken synchronously under the lock, so requests racing a run's completion are neither lost nor doubled), the defer ordering in watchEvents guarantees no rebuild goroutine outlives the function, and the test suite is exemplary — deterministic channel-driven fake with the no-reentrancy invariant asserted inside the fake itself, and the assertNoRebuildStarts justification is exactly right. -race passes locally on the new tests.
One behavioral edge worth an explicit decision before merge, then small things:
[medium] A sync+restart change arriving while a rebuild of the same service is ACTIVE is silently lost.
Timeline: batch 1 triggers rebuild(A); while the image builds, batch 2 carries a sync+restart change for A. The sync lands in the doomed old container (the documented tradeoff), the restart is skipped by InFlightOrPending, and the new container comes from a build context snapshotted before the change. Net effect: the edit is silently dropped until the user touches the file again. Before this PR the synchronous rebuild made batch 2 converge; and the burst-editing regime this PR targets makes the overlap window common, not exotic.
The skip is fully correct for a pending rebuild (it will re-read the context, so the change is included and the restart is redundant). It is only the active case that loses data. Suggested fix: distinguish the two — skip on pending as today, and on active either requeue the restart to run after the current rebuild completes, or requeue a trailing scheduler.Request([service]) (converges via context re-read, at the cost of an extra — usually cache-warm — rebuild in the rare overlap).
[low] After Ctrl-C, run() still drains the pending set with a cancelled ctx, so the user log gets an extra "Build failed. Error: context canceled". A ctx.Err() check before each drain iteration would make shutdown silent.
[low] _ = s.rebuild(services) is actually fine — s.rebuild reports everything to options.LogTo itself — but that contract deserves a one-line comment on the discard, or the next reader will read it as a swallowed error.
[question] s.rebuild (build/create/start) now runs concurrently with the event loop's own uses of the same *types.Project (restart, syncer, exec hooks). Worth double-checking none of those paths mutates the project in place — the scheduler serializes rebuilds against each other, but not against the loop.
[nit] The comment on TestHandleWatchBatch_SkipsRestartForServiceWithRebuildInFlight references "#14051 (follow-up)" — that's the TTY renderer PR; stale reference.
[nit] In TestRebuildScheduler_NoConcurrentRebuilds, the local max shadows the builtin.
A rebuild whose build context was snapshotted before a newer change to one of its own services is doomed: its output will be replaced by the trailing rebuild anyway, and meanwhile a sync+restart change landing in that window was silently lost — the sync went to the doomed container, the restart was skipped, and the replacement image predated the edit. The scheduler now gives each run its own cancellable context and upholds one invariant: every request ends up covered by a rebuild whose snapshot postdates it. A request naming an actively-rebuilt service interrupts the run and folds its whole active set back into pending (interruption kills the other services' rebuild too), so the trailing rebuild redoes everything on a fresh snapshot. Requests for other services keep coalescing without interrupting. handleWatchBatch drops a restart only when the rebuild is pending (that one converges on its own) and folds it into a Request when the rebuild is in flight — recreate + start is the restart intent, converged. The run loop also stops draining once the watch context is cancelled, which silences the spurious "Build failed: context canceled" on Ctrl-C. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
|
Concrete, tested proposal for the [medium] finding above: glours#124 — targets this PR's branch, so merging it updates this PR. Design in one line: a rebuild made stale by a newer request for one of its own services is interrupted (per-run cancellable context) and its whole active set folds back into pending, upholding the invariant that every request ends up covered by a rebuild whose context snapshot postdates it. Also turns restart-during-in-flight into a Request (converged restart intent) and stops the drain on watch shutdown ([low] Ctrl-C noise). |
ndeloof
left a comment
There was a problem hiding this comment.
All findings from my earlier review are addressed at this head — the [medium] sync+restart convergence loss (a rebuild made stale by a newer request is now interrupted, its active set folded back into pending, and a restart landing on an in-flight rebuild folds into a Request), the shutdown-drain noise, and the stale test reference. I validated the merged change locally before it landed here (full pkg/compose suite with -race, lint clean), and CI is green on this head. The scheduler invariant — every request ends up covered by a rebuild whose context snapshot postdates it — is exactly the right contract for watch. LGTM.
What I did
compose watch triggered one rebuild per debounced batch, so a burst of edits arriving faster than the debounce window (e.g. from a coding agent editing several files) queued up several rebuilds in a row instead of a single consolidated one.
Rebuilds now run asynchronously through a small scheduler that coalesces requests received while a rebuild is in flight into one trailing rebuild, without adding latency to an isolated edit. This also closes a race where a plain restart could run concurrently with an asynchronous rebuild of the same service.
Related issue
N/A
(not mandatory) A picture of a cute animal, if possible in relation to what you did
