ice: a failed agent must not pin its wake-up deadline in the past - #2
Open
lannbot wants to merge 1 commit into
Open
ice: a failed agent must not pin its wake-up deadline in the past#2lannbot wants to merge 1 commit into
lannbot wants to merge 1 commit into
Conversation
contact() updates last_checking_time only after running checks, and its Failed early-return runs none -- so once an agent fails, the deadline poll_timeout() derives from it (last_checking_time + interval) freezes at the last real check, permanently in the past. handle_timeout() then lands back in the same early-return without advancing anything: a driver that re-polls after handling sees the same expired deadline forever. Measured live at ~4.5M loop passes per second, hard enough to starve the driver's whole runtime -- the teardown cascade that would end the failed connection never got to run (found from wosh via polymorph-iroh's freeze drill: SIGSTOP a peer 45s, ICE fails on consent expiry, host wedges solid). Advance the checking clock in the Failed early-return, exactly as the force_candidate_contact reset above already does for the one-shot form of the same disease (issue webrtc-rs#88). A failed agent then ticks at the ordinary check cadence until closed or restarted. The regression assertion extends the issue-88 test: after failure, handle_timeout at a time well past the creation-time deadline must not leave an expired deadline armed -- 'well past' because that is where a frozen clock is distinguishable from a fresh one.
This was referenced Aug 14, 2026
lannbot
pushed a commit
to lann/wosh
that referenced
this pull request
Aug 14, 2026
The wedge the drill kept finding is now contained where the host can contain it: polymorph-webrtc-datachannels' wasmtime host runs every webrtc peer-connection driver on a dedicated reactor pool instead of the embedder's runtime (its #158/#159; the rev pin here carries it). A driver that wedges -- the rtc-ice frozen-deadline spin, or whatever ships next -- costs a pool thread instead of the whole component, and then SELF-HEALS: the still-running owner observes the failure, drops the connection, and the driver's own closing check ends the loop. Verified against the deliberately-unfixed webrtc pin: through the 45s freeze the listener stays responsive, the zombie parks on schedule, fresh dials are accepted mid-wedge, the spinning thread goes quiet within seconds, and the thawed client resumes cleanly. With that and the polymorph-iroh epoch guard both in the pinned chain, browser-freeze joins check. The rtc-ice root fix (lann/rtc#2) remains worth landing but no longer gates anything here. The gate also grows one retry on the post-thaw keystroke: the click/type can race the renderer's first layout after 45s of not being scheduled and land outside xterm entirely -- a harness hazard (observed once in ~10 runs), not a session one; the session's own delivery is what the marker then proves.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
contact()updateslast_checking_timeonly after actually running checks — and itsFailedearly-return runs none. Once an agent fails (ICE consent expiry on a vanished peer), the deadlinepoll_timeout()derives from it (last_checking_time + interval) freezes at the last real check, permanently in the past.handle_timeout()then lands back in the same early-return without advancing anything, so a driver that re-polls after handling sees the same expired deadline forever.Observed live (from wosh, via polymorph-iroh's freeze drill — SIGSTOP a connected peer 45s, the phone-in-a-pocket case): the
webrtcevent loop spun on the frozen deadline at ~4.5M passes/s (the deadline receding at exactly 1s/s under instrumentation), hard enough to starve its whole tokio runtime — including the teardown cascade that would otherwise have ended the failed connection, so the spin was permanent.This is the periodic twin of the one-shot form already fixed in webrtc-rs#88: the
force_candidate_contactreset at the top ofcontact()exists precisely so "a failed/settled agent does not keep askingpoll_timeoutfor an immediate wake-up" — but the periodic arm has the same disease throughlast_checking_time.The fix advances the checking clock in the
Failedearly-return, so a failed agent ticks at the ordinary check cadence until closed or restarted (matching howDisconnectedbehaves). The regression assertion extends the webrtc-rs#88 test: after failure,handle_timeoutat a time well past the creation-time deadline must not leave an expired deadline armed — "well past", because that is where a frozen clock is distinguishable from a fresh one. Verified to bite: red without the fix, green with it;cargo test -p rtc-ice49/49.End-to-end verification: with this fix alone (completely stock
webrtcdriver), the wosh freeze drill un-wedges — the listener's QUIC idle timeout fires on schedule, the session parks, and fresh dials are accepted mid-freeze. Context: lann/wosh#63 (the drill and the hunt), polymorph-components/polymorph-iroh#78 (the sibling bug the same drill found).One note on consumption:
lann/webrtcpins itsrtcsubmodule at the head ofsctp-reset-undelivered-data(#1). This branch is based onmaster, so after merge the submodule wants a ref that carries both — the companion driver-side patch (a forward-progress floor for the next timer bug of this class) bumps the submodule and can be re-pointed wherever you land this.