Huddles overhaul: drain-safe rooms, bounded send latency, typed relay errors, live route switching - #6585
Draft
tlongwell-block wants to merge 6 commits into
Draft
Huddles overhaul: drain-safe rooms, bounded send latency, typed relay errors, live route switching#6585tlongwell-block wants to merge 6 commits into
tlongwell-block wants to merge 6 commits into
Conversation
Co-authored-by: Max <d8473ee32b973aa31a21a65adddcc4b69cc2a8a4dee8121ecd51926e0cddbc02@buzz.block.builderlab.xyz> Signed-off-by: Max <d8473ee32b973aa31a21a65adddcc4b69cc2a8a4dee8121ecd51926e0cddbc02@buzz.block.builderlab.xyz>
tlongwell-block
force-pushed
the
huddles/overhaul
branch
from
August 22, 2026 23:56
8445c68 to
536d3c2
Compare
A relay drain (SIGTERM or owner-drain) cancelled every local owner client at once, so the last teardown emptied the room and took the archive + 48103 path: every pod restart ended every huddle it owned. The same path also ended a huddle the instant its last client blipped, so a reconnect found the channel archived. The leave path now distinguishes a drain-driven teardown (the owner draining token fired, or `shutting_down` / `owners.is_draining()` is set, covering single-pod mode where there is no mesh token) and only releases the lease and drops the empty room, leaving the channel alive for rejoiners to re-acquire through Redis. Ordinary last-leaver departures no longer end the room atomically. `remove_peer_and_check_idle` captures an `IdleGeneration` (the admission count) under the guard lock; a spawned task waits `ROOM_EMPTY_GRACE` and calls `end_if_idle`, which sets `ended` only if the room is still empty and no admission happened in between. A rejoin during the window fences the stale observation even if the rejoiner has already left again, and two leavers who both observed empty cannot both archive. Owner-loss or a drain that begins mid-window aborts the end; owner-loss wins a tie with the timer. The fence only holds if the rejoiner lands on the same `Room`. A pre-last leaver's `cleanup_if_empty` (or a failed pre-admission join) could run after the last leaver's idle observation and evict the empty room from the manager; the rejoiner would then get a fresh `Room` and the stale grace task would archive underneath it. The idle observation now sets `idle_hold` on the room and `cleanup_if_empty` refuses to evict while it is set. The hold survives `end_if_idle` so the room stays registered while `archive_channel` is in flight — a joiner in that gap meets `ended` on the same room instead of admitting into a fresh one whose pre-join DB check raced the archive. The hold clears on admission or via the fenced `release_idle_hold`, called after the archive resolves (success or rollback) and on the drain/abort paths that abandon the window. Tests: room idle generation (last-leaver-only, end-once, rejoin fences stale), manager eviction (concurrent-leaver ordering pins the room and the rejoiner fences the stale end; released hold permits eviction, stale release does not; ended room stays pinned and refuses joiners until the archive resolves; failed archive reopens and releases), and paused-time grace outcomes (holds through window, rejoin, no-rejoin ends, owner-loss aborts, mid-window drain aborts). Co-authored-by: Meli <5aaa86bce934fc3445fc254aab560a40923f10252f92107e665073dede0e04d3@buzz.block.builderlab.xyz> Signed-off-by: Meli <5aaa86bce934fc3445fc254aab560a40923f10252f92107e665073dede0e04d3@buzz.block.builderlab.xyz>
Render reply events already present in the bounded channel window while the complete thread query loads. Keep the window data as placeholder-only so it does not delay the authoritative request. Co-authored-by: Wren <5217c5c2f7bfb4333e46d17c98a9255a52dadee18dcd43a43536b95e6776dfa0@buzz.block.builderlab.xyz> Signed-off-by: Wren <5217c5c2f7bfb4333e46d17c98a9255a52dadee18dcd43a43536b95e6776dfa0@buzz.block.builderlab.xyz>
Preserve typed relay admission codes through the audio connection boundary so UI copy and reconnect policy can act on production handshake failures. Move all active huddle playback consumers when the output route changes: remote peers, agent TTS, and STT acoustic-coupling policy. Co-authored-by: Max <d8473ee32b973aa31a21a65adddcc4b69cc2a8a4dee8121ecd51926e0cddbc02@buzz.block.builderlab.xyz> Signed-off-by: Max <d8473ee32b973aa31a21a65adddcc4b69cc2a8a4dee8121ecd51926e0cddbc02@buzz.block.builderlab.xyz>
tlongwell-block
force-pushed
the
huddles/overhaul
branch
from
August 23, 2026 00:16
536d3c2 to
4cff8ea
Compare
Squash of 4baccd5..b92854104 (Perci's agent-voice lane). Relay derives the publisher role from verified NIP-OA auth + active bot membership and elects one publisher seat per agent identity, same-pod and cross-pod; rosters and ordered deltas carry the role; desktop suppresses only the authoritative publisher's local fallback and drops the utterance on a typed election loss. Integration note: the lane's standalone `AudioConnectError` enum was folded into commit 6's typed `AudioRelayConnectError` as `is_lost_election()` (code == "duplicate_identity") so the audio handshake has one error type. `relay_api.rs` crossed the desktop 1000-line ratchet with the new tests, so its `tests` module moved to `relay_api_tests.rs` via `#[path]`, matching `stt.rs`/`models.rs`. Co-authored-by: Perci <5a968df9a7494b4e019b9ecf739e088ba61097b4312124e9a88ae5b42e3f5f3e@buzz.block.builderlab.xyz> Signed-off-by: Perci <5a968df9a7494b4e019b9ecf739e088ba61097b4312124e9a88ae5b42e3f5f3e@buzz.block.builderlab.xyz>
An unexpected audio-relay disconnect used to emit
`huddle-audio-disconnected`, and HuddleContext.tsx answered with a
7-attempt loop (0/100/250/500/1000/2000/2000 ms, ~5.85 s total) that ended
in a silent `leaveHuddle()`. A relay deploy or a Wi-Fi roam that lasts
longer than that dropped the user out of the huddle with no explanation.
The audio pipeline task now awaits `reconnect::after_unexpected_disconnect`
in place of the event. `reconnect::run` redials for at least
`RECONNECT_WINDOW` (60 s) from the disconnect, with 250 ms → 5 s exponential
backoff jittered into [0.5, 1.0]; a `huddle_relay_draining` refusal redials
at the 250 ms floor because a replacement pod is coming. Huddle identity
(`is_current_huddle`) is re-checked after every dial and every sleep, so a
leave or a replacement huddle always wins and a socket won for the old
huddle is cancelled, never installed. Exhausting the window sets
`AudioLink::Lost` — still joined, visibly without audio — instead of leaving.
Progress is a new `HuddleState::audio_link` (`Live | Reconnecting{attempt,
draining} | Lost`), serde-tagged on `status`. It is a field, not a
`HuddlePhase` variant: `Connected | Active` is the liveness gate for
STT/TTS/agent voice, and a reconnecting phase would have stopped them
mid-blip. `claim_reconnect` flips `Live → Reconnecting` under the lock and
is the single guard against duplicate disconnect signals.
The `reconnect_huddle_audio` Tauri command and the React loop are deleted;
HuddleBar shows a muted pulsing "Reconnecting…" (or the relay-restart
variant) and a destructive "Couldn't reconnect audio" banner from the new
`audioLinkNotice`. Type mirrors in HuddleBar and the e2e bridge carry the
field.
`after_unexpected_disconnect` returns a boxed `Send` future because the call
graph is recursive (pipeline task → reconnect → `connect_audio_relay` →
pipeline task).
`connect_audio_relay` spawns the replacement pipeline before its handles
reach the reconnect loop. If that pipeline dies in the gap, its own
disconnect callback is refused by `claim_reconnect` (the link is already
`Reconnecting`), so the loop treats an already-cancelled token as a failed
dial and redials — `Live` never holds a dead sender.
Tests (paused tokio clock, injectable dial, each proven to fail when its
guard is removed): recovers after a 35 s outage without leaving, draining
redials promptly and is visible in state, leave during backoff exits the
loop, leave during an in-flight dial cancels the fresh socket, replacement
huddle is left untouched, a replacement pipeline that dies before install
is redialed not installed, duplicate signals do not start a second loop,
exhaustion marks Lost and terminates, not-live huddle is ignored, backoff
caps and jitters, `AudioLink` serialization. Node tests cover the banner
copy.
Co-authored-by: Meli <5aaa86bce934fc3445fc254aab560a40923f10252f92107e665073dede0e04d3@buzz.block.builderlab.xyz>
Signed-off-by: Meli <5aaa86bce934fc3445fc254aab560a40923f10252f92107e665073dede0e04d3@buzz.block.builderlab.xyz>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Huddles overhaul
One PR, bug-first, each commit independently green and revertible. Lead: Meli; implementation Max / Wren / Perci / Meli; independent review Mari (every commit cleared at ≥ 9/10 on minimalness / elegance / correctness before integration).
Plan:
PLANS/HUDDLES_OVERHAUL_HANDOFF_2026_08_21.md+PLANS/HUDDLES_OVERHAUL_PLAN_DELTA_2026_08_22.md(in the Buzz workspace). Thread: #buzz-conversational-agents.Integrated
fix(desktop): bound huddle audio send latencyfix(relay): never auto-end a huddle on drain; hold empty rooms 20sRoom; owner-loss or mid-window drain aborts).feat(desktop): seed huddle thread repliesplaceholderDatafrom the channel window so a new transcript root renders instantly while its authoritative fetch starts immediately. Projects path unchanged.feat(desktop): polish huddle audio controlsAudioRelayConnectError { code, message }across the Rust boundary (reconnect can act onhuddle_relay_draining); per-code friendly error copy; live output-device switching reaches peer playout, active TTS (mid-utterance), and STT coupling; one-time PTT hint.feat(huddles): move audio reconnect into Rust with a 60 s windowhuddle_relay_draining→ 250 ms redial, huddle identity re-checked after every dial and sleep; exhaustion →AudioLink::Lost(still joined, visible), never a silent leave. `HuddleState.audio_link: Livefeat(huddles): elect one agent voice publisherduplicate_identityelection loss.relay_api.rstests moved torelay_api_tests.rs(desktop 1000-line ratchet).Pending (own branches, not yet cleared)
Dropped with receipts
4baccd539(mounted-hook[A]→[B,A], add-root, invalidate/refetch all retain A). Not shipping a fix for a bug we cannot reproduce; symptom stays on the live-pass watch list.Known flake (pre-existing, not touched here)
api::mesh_demo::tests::demo_join_forwarded_arm_round_trips_echofails with 504-vs-200 under CPU contention (observed on parent4baccd539and on this branch while other builds ran; passes when the box is quiet).mesh_demo.rs/mesh_boot.rsreference no audio-room code.Verification
Per-commit receipts are in the thread. At PR head: pre-push lanes (
rust-tests,desktop-check,desktop-typecheck,desktop-test,desktop-tauri-checks,file-size-check,branch-skew) all green. Live two-client pass (GUIDES/HUDDLES_LOCAL_LIVE_PASS_2026_08_22.md) pending two desktop operators; staging hasBUZZ_HUDDLE_AUDIO_AVAILABLE=false.