Skip to content

fix(stt): keep providers available after fallback stream close - #7296

Open
rosetta-livekit-bot[bot] wants to merge 1 commit into
mainfrom
herald-grafting-hankered
Open

rosetta-livekit-bot[bot] wants to merge 1 commit into
mainfrom
herald-grafting-hankered

Conversation

@rosetta-livekit-bot

@rosetta-livekit-bot rosetta-livekit-bot Bot commented Sep 15, 2026

Copy link
Copy Markdown
Contributor

Summary

Ports livekit/agents-js#2381 to the Python STT fallback adapter.

  • isolate provider failures and recovery ownership across concurrent fallback streams
  • transfer serialized recovery probes between live streams and stop them safely on teardown
  • preserve fallback/retry behavior after premature EOF while ignoring late events after closure
  • bound child and probe cleanup, guard late batch recovery, and redact provider exception details

Testing

  • uv run pytest tests/test_stt_fallback.py --unit -q (36 passed)
  • make check
  • uv build --package livekit-agents
  • make unit-tests attempted twice: the first run reached 3486 passed before Docker-backed room fixtures failed because Docker is unavailable and configured inference credentials returned 401; with LiveKit credentials unset, the run reached 3090 passed before existing concurrent OpenAI realtime tests closed their shared event loop (9 errors).
  • cue-cli runtime validation attempted in voice mode, but the configured Cue LiveKit endpoint rejected the API key with 401 before an agent could join.

No Changeset was added because this repository does not use Changesets.

Source diff coverage
Source file Classification Target coverage
.changeset/clean-fallback-teardown.md Not applicable The Python repository does not use Changesets; package versions are managed by release workflows.
agents/etc/agents.api.md Not applicable This generated TypeScript API Extractor report has no checked-in Python counterpart.
agents/src/stt/fallback_adapter.test.ts Adapted Ported the modified regressions to tests/test_stt_fallback.py using Python async stream controls and aclose().
agents/src/stt/fallback_adapter.ts Adapted Ported to livekit-agents/livekit/agents/stt/fallback_adapter.py; JS abort controllers, queues, and promises map to asyncio tasks, channels, bounded cancellation, and recovery waiters.
agents/src/stt/fallback_adapter_lifecycle.test.ts Adapted Ported lifecycle, concurrency, recovery-transfer, cleanup, retry, EOF, and safe-logging regressions to tests/test_stt_fallback.py. AgentTask handoff behavior is exercised through direct concurrent stream ownership because Python reuses its STT pipeline across compatible handoffs.
agents/src/stt/stt.ts Not applicable Python RecognizeStream.__anext__() already rethrows each child task exception, providing the stream-local terminal failure state that the source adds as _failed.

Ported from livekit/agents-js#2381

Original PR description

Summary

Treat a closed parent queue during FallbackSpeechStream teardown as a local lifecycle event. This prevents a late child transcript from marking a healthy STT provider unavailable.

Closes #2349.

Testing

  • vitest run agents/src/stt
  • tsc -p agents/tsconfig.typecheck.json
  • ESLint and Prettier on changed files

@rosetta-livekit-bot
rosetta-livekit-bot Bot requested a review from a team as a code owner September 15, 2026 19:23

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 1 potential issue.

Devin Review

Comment on lines +583 to +585
await asyncio.sleep(0)
if self._closing:
return

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 Failover drops incoming audio frames

When audio arrives during a provider switch, _forward_input_task still sends it to the failed main_stream. Push failures are discarded, so the fallback provider misses that speech.

Learn more

The failed child remains in main_stream while the failure path yields to the event loop. The concurrent input-forwarding task can run during that yield and push new frames into the failed or closing child. Its exception handler discards the push failure, and no replay buffer exists for the next provider.

Example: The primary stream fails, then a caller pushes a frame containing the start of “hello” while this path yields. The primary rejects the frame. The secondary starts afterward and receives only the remaining audio, producing an incomplete transcript.

Recommended fix: Clear main_stream before the first yield after child termination. Serialize provider replacement with input forwarding, or buffer frames received while no live child is assigned and drain them into the replacement stream.

Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Realtime metrics collection throws error when token_count is None

0 participants