Skip to content

endpoint: event-driven wakeups replace the bounded-polling pump - #44

Merged
lann merged 2 commits into
mainfrom
event-driven-pump
Aug 11, 2026
Merged

endpoint: event-driven wakeups replace the bounded-polling pump#44
lann merged 2 commits into
mainfrom
event-driven-pump

Conversation

@lann

@lann lann commented Aug 11, 2026

Copy link
Copy Markdown
Collaborator

With host-jco retired (#40), the guest's portable workaround for its scheduler defect — bounded clock polling (5 ms method quantum / 10 ms pump tick) — protected nothing and cost roughly 5× handshake latency on every host (#10's header, re-measured below).

What changed

Wake-ups are event-driven in both directions, riding wit-bindgen's inter-task-wakeup channel — a guest-internal unit stream whose write resumes a task parked in waitable-set.wait; both remaining hosts deliver it (the websocket sibling's guest-provider already uses the feature):

  • Methods → pump: every mutation needing a flush kicks the pump (State::kick_pump, a pure-Rust select arm): the dial's first flight, stream writes (complete or partial), FIN/RESET_STREAM/STOP_SENDING, datagrams, queued signals, connection and endpoint closes, new relays/channels, and — new — flow-control credit from consumed reads, which previously waited out the tick and could stall the sender.
  • Pump → methods: wait_until parks a waker in State::waiters instead of sleeping a quantum; the pump wakes the waiters after every drain that progressed, on signal-inbox pushes, and unconditionally on its tick.
  • The 10 ms tick stays: noq's timers need servicing, and the tick turn's unconditional wake bounds deadline observation (relay-open timeout, signaling deadline) and any missed wake edge at one tick.
  • The never-cancel-an-import teardown discipline is unchanged; every break path sets terminal state before the pump's final wake, so no waiter can park past the pump's death.

Co-dependence: websocket pin bump

Going event-driven exposed a latent Nagle stall in the websocket sibling's wasmtime host: back-to-back small sends (STREAM then FIN, no longer batched into one tick flush) sat out the peer's delayed ACK — relay-wire echo roundtrips of 27–50 ms. Fixed upstream in polymorph-components/polymorph-websocket#45 (TCP_NODELAY, matching browser behavior); WEBSOCKET_PIN bumps to that merge commit. The two changes cannot land separately without leaving main worse: the pump change alone regresses relay roundtrips 23 → ~47 ms, and the pin bump alone asserts nothing.

Measured (this machine, medians; target/bench/report.tsv)

row metric before after
endpoint-relay handshake / roundtrip 16 / 23 ms 4 / 0 ms
endpoint-udp handshake / roundtrip 21 / 17 ms 1 / 0 ms
endpoint-webrtc handshake / roundtrip 14 / 23 ms 1 / 0 ms
spike-relay (baseline) handshake 4 ms 2–4 ms
bulk relay / udp / webrtc throughput 7.7 / 49.1 / 38.1 MB/s 27.8 / 85.1 / 49.1 MB/s

The bulk gains come from prompt flow-control credit plus the nodelay fix. The deltic exam's relay roundtrip also moved (382 → 59 ms), untracked by any budget.

Budgets

Absolute ceilings tighten 2000 → 250 ms (still order-of-magnitude guards, CI-safe). The issue asked for the win to be asserted rather than prose: an absolute ceiling can't do that without flaking (old world was ~16–21 ms), so the bench now asserts the spike-to-endpoint handshake delta (handshake_tax_ms ≤ 10 ms) — same wire, relay, and run, so runner noise cancels; the old world's +12–17 ms tax fails it, and a reintroduced quantum-stacking or Nagle-class stall on the handshake path fails it too.

Verified

just check, just probes (5/5), just matrix (12 pairings), just exam-deltic (5/5, including the #10 concurrency rows: accept parked across a handshake and woken), just bench (all budgets hold, handshake_tax_ms 0).

Fixes #42

lann added 2 commits August 10, 2026 20:49
With host-jco retired (#40), the portable workaround for its scheduler
defect - bounded clock polling (5 ms method quantum / 10 ms pump tick)
- protected nothing and cost ~5x handshake latency on every host.

Wake-ups are now event-driven in both directions, riding wit-bindgen's
inter-task-wakeup channel (both hosts deliver it): resource methods
kick the pump to flush their mutations (first flight, stream writes,
FIN/RESET/STOP_SENDING, datagrams, signals, closes, flow-control
credit) and park on wakers in State::waiters; the pump wakes the
waiters after every drain that progressed, on signal-inbox pushes, and
unconditionally on its 10 ms tick, which stays for noq's timers and
now also bounds deadline re-checks and any missed wake edge. The
never-cancel-an-import teardown discipline is unchanged; the kick
future is guest-local.

Going event-driven exposed a latent Nagle stall in the websocket
sibling's wasmtime host: back-to-back small sends (STREAM then FIN,
no longer batched by the tick) sat out the peer's delayed ACK, putting
relay-wire echo roundtrips at 27-50 ms. Fixed upstream
(polymorph-components/polymorph-websocket#45, TCP_NODELAY); the
websocket pin bumps to that commit here. The two changes are
co-dependent: the pump change without the pin bump regresses relay
roundtrips 23 -> ~47 ms.

Bench on this machine (medians): endpoint handshakes 16/21/14 ms
(relay/udp/webrtc) -> 1-4 ms, matching the spike; roundtrips 23/17/23
-> 0 ms; bulk throughput 7.7/49.1/38.1 -> 27.8/85.1/49.1 MB/s (prompt
flow-control credit plus nodelay). The budgets tighten accordingly:
absolute ceilings 2000 -> 250 ms, and the spike-to-endpoint handshake
delta is asserted (<= 10 ms) so the polling tax cannot quietly return.

Verified: just check, just probes (5/5), just matrix (12 pairings),
just exam-deltic (5/5), just bench (budgets hold, handshake_tax_ms 0).

Fixes #42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

endpoint: revert the bounded-polling pump to event-driven wakeups (the jco-era workaround is now pure cost)

1 participant