Repository navigation
Peer-fleet × hub-fleet: the layered merge — ports, not rebases (the convergence evidence and the ranked port list) #1803
Description
The thesis
The hub-fleet (this lane: one door, shards, shared state, #1760's contract) and the peer-fleet (main's #1407 lineage: machines as mutual servers, heartbeats, live health) are layers of one system, not competing implementations. A peer's hub can itself be sharded; the shard fleet is the capacity layer the peer fleet lacks, and the peer fleet is the multi-machine layer the hub fleet never attempted. The merge is a layering, not a replacement.
The evidence is the strongest kind: two lines independently converged on the same architecture. This lane built resumable cursors, replay buffers, honest closes, census lines, and client-side gap detection through the stream-chain saga; main's #1770 built epoch:seq ids, bounded replay, reset cursors, per-listener fan-in, and forced resync through an equivalent saga on the two-machine fleet. Same war, same contract, two implementations with different strengths.
Execution doctrine: PORTS, NOT REBASES
The Oct-13 rebase merged two product generations in one 84K-line motion and broke the panel (dead surfaces, mixed generations). Doctrine for this lane: architectural wins cross as ported commits, one at a time, each behind the headless probe suite, each landing green before the next. No whole-branch merges of either lineage until the port list below is done — after that, the trees converge naturally and a merge is boring.
The port list (ranked)
P1 — Engine-side sequencing (epoch:seq, bounded replay, explicit reset). Main's #1770 S-slices put the sequence at the SOURCE (the global bus stamps <epoch>:<seq>, replay buffers honor cursors, unservable cursors get an explicit reset frame). This lane does sequencing in the frontdoor's ring — which only sees what the frontdoor saw and dies with frontdoor restarts. Source-side sequencing survives every middle-hop death; the epoch component makes engine restarts a natural resync boundary. Taking this obsoletes the frontdoor ring and demotes the frontdoor to a dumb pipe — simpler, fewer seams, exactly #1802's spirit. Port: the engine overlay's global-bus id assignment + the replay buffer + the reset contract; then the frontdoor drops its ring and passes ids through.
P2 — The union of liveness fixes. Main's side-finding: engines hold subscriptions never released on abort (18 connects, 0 disconnects, 3.7GB resident). This lane's #1798: FIN-less corpse sockets wedge the fan-out emit loop. These are the same defect seen from opposite ends — the complete fix is both: subscription release on abort (main's slice) + bounded non-blocking fan-out writes + the FIN/keepalive discipline (this lane: _fin_close is live; kernel keepalives are #1802 refinement 2). Either half alone leaves a stall mode.
P3 — Burst-volume cuts at the source. Patch bodies off the wire, shell-output throttling, 256KB frame ceilings (main's #1770 scope). This lane solved the flood receiver-side (member-queue delta coalescing — keep it); main cuts it emitter-side. Both, compounded: the wire stays thin AND the queues stay small.
P4 — Timestamped heartbeats carrying the current seq. Main's shape makes lag measurable, not just detectable. Cheap port; immediate observability win (the freshness/lag dashboard becomes possible).
P5 — Linear frame splitting in the app reader (main's #1779 / S8). The quadratic partial-buffer rescan class this lane skirted. Mechanical port.
P6 — The ownership map as a live feed with last-good retention (main's #1774 / S3). A peer's sessions survive a failed refresh instead of routing to the local engine. This lane's DB-poll is the hub-side equivalent; the feed model composes with #1802 refinement 1 (deterministic placement): the feed becomes a cache of the hash function, retention makes it flap-proof.
What each side keeps
This lane keeps (main lacks these): the one-door gateway, intra-machine sharding, P2's shared state rows, the panel's single-server posture (#1784, telemetry-asserted), reply directory-injection, the watchdog battery, the telemetry channel, the harness API.
Main keeps (this lane adopts): everything in the port list above, plus the peer-machine UX surfaces (fleet headers, session-status, health indicators) — which render against real data on the layered system where they rendered dead on the Oct-13 naive merge.
Verification
Each port lands behind: the headless probe suite green, the wire contract asserted (the preamble-bug lesson: opening frames are a versioned, probe-asserted contract), telemetry lines greppable per port, and one soak window with the census quiet. The P1 port additionally requires the lossless-resume E2E from main's #1770 acceptance (kill the link mid-turn, reconnect, transcript equals fresh load) — run against THIS lane's sharded hub, not just peer-to-peer.
Honest notes
- P1 is the largest and the most load-bearing: it re-derives the frontdoor's core role. Do it first and alone; everything after it gets simpler.
- The port order is deliberate: sequencing before liveness before volume, because each makes the next observable.
- After P1-P6, the trees have the same wire contracts and the same event model — a generation merge at that point is boring, which is the goal.
Activity