Conversation
RecognizeStream counts retries over the lifetime of the stream and only resets on a FINAL_TRANSCRIPT. A caller who never speaks never produces one, so providers that close an idle socket on a fixed interval (Cartesia sends 1001 Idle timeout every ~3 minutes) exhaust max_retry after a few clean reconnects and the stream gives up for good. An attempt that ran longer than the connect timeout had connected, so its failure is not consecutive with the previous one: reset the count before handling the error. The budget now means consecutive failures in a row, independent of whether anyone was talking. Fixes #7208.
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
| if time.time() - last_start_time > self._conn_options.timeout: | ||
| self._num_retries = 0 |
There was a problem hiding this comment.
🔴 Connection timeouts retry forever
A handshake timeout after conn_options.timeout resets _num_retries despite never connecting. Repeated handshake timeouts never exhaust max_retry, so fallback never starts.
Learn more
Elapsed attempt time cannot distinguish a healthy connection from a handshake that consumed its entire timeout. Several streams enforce the same option around connection establishment, such as SpeechStream._connect_ws. Such a timeout returns after approximately conn_options.timeout, which satisfies this reset and turns the next failure into attempt zero again. The public options also permit timeout=0, making any nonzero-duration retryable failure reset the budget immediately.
Example: With max_retry=3 and timeout=10, xAI repeatedly fails its WebSocket handshake at 10 seconds. Every failure resets _num_retries to zero, so the stream retries indefinitely instead of failing after four attempts.
Recommended fix: Reset the budget only from explicit evidence that the attempt established a usable provider connection. Add a connection-acquired signal or timestamp to RecognizeStream, and have each streaming plugin set it after its handshake succeeds. Do not infer connection success from total _run() duration.
Was this helpful? React with 👍 or 👎 to provide feedback.
There was a problem hiding this comment.
I think this comment is valid
There was a problem hiding this comment.
maybe use elapsed > 2 * self._conn_options.timeout?
3 participants
Fixes #7208. Supersedes #7207.
Problem
RecognizeStream._main_taskgives a streammax_retry(default 3) reconnect attempts, and the only thing that resets_num_retriesis aFINAL_TRANSCRIPT. A caller who never speaks never produces one, so a provider that recycles an idle socket on a fixed interval spends the whole budget on ordinary, successful reconnects. Cartesia closes an idle socket with1001 Idle timeoutevery ~3 minutes; after four of them the stream raisesfailed to recognize speech after 3 attemptsand emitsrecoverable=False, even though every reconnect in between succeeded.#7207 proposes resetting on any provider event. That covers Deepgram, which emits usage events every 5 s while audio flows, but not a provider that stays quiet during silence: Cartesia delivers nothing at all until someone speaks, so under #7207 the silent-caller case still dies. Its own description notes this.
Since #6418 the session tolerates unrecoverable STT errors and
_STTPipelinerecreates the stream, so onmainthis costs a socket rather than the call. It still burns one ofmax_unrecoverable_errorsper ~12 quiet minutes, and that count only resets on a user transcript, so a long silent call dies at ~36 minutes.Reproduced against real Cartesia with
cue-clidriving a dispatchable agent started withRoomOptions(audio_input=False)onmain:Fix
An attempt that ran longer than
conn_options.timeouthad connected, so its failure is not consecutive with the previous one: reset_num_retriesbefore handling the error.max_retrynow means "consecutive failures in a row" regardless of whether anyone was talking. No new option; reuses the existing connect timeout. TheFINAL_TRANSCRIPTreset is left in place.Verification
Same
cue-cliscenario on this branch:Every retry is attempt 0 again and every error is
recoverable=True; the stream never gave up and the session's unrecoverable-error count was never touched.Unit tests in
tests/test_stt_base.py: a flapping stream whose connections stay up for 180 s survives 9 drops withmax_retry=3, and one whose connections drop after 1 s still gives up aftermax_retry + 1attempts.mypy -p livekit.agentsclean.JS counterpart: livekit/agents-js#2494 (which also ports #6418).