fix(livekit-agents): propagate prewarm through the STT fallback and stream adapters - #6583
Open
fxhxdxd wants to merge 1 commit into
Open
fix(livekit-agents): propagate prewarm through the STT fallback and stream adapters#6583fxhxdxd wants to merge 1 commit into
fxhxdxd wants to merge 1 commit into
Conversation
…tream adapters `stt.FallbackAdapter` and `stt.StreamAdapter` both inherit the no-op `STT.prewarm()`, so the prewarm issued by `AgentActivity` never reached the wrapped instances. Providers that implement prewarming (currently the mistralai realtime STT, which warms its connection pool) were therefore never warmed when used behind either adapter. Both hops matter: `FallbackAdapter` transparently wraps non-streaming STTs in a `StreamAdapter`, so prewarm has to survive the adapter *and* the wrapper to reach the provider. `tts.FallbackAdapter` and `tts.StreamAdapter` already forward it; this brings the STT path in line. Only the primary instance is prewarmed, matching the TTS adapter, since the remaining instances are not expected to serve traffic unless it fails.
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #6582, where I flagged the same gap on the STT side and offered to cover it separately.
Problem
Both
stt.FallbackAdapterandstt.StreamAdapterinherit the no-opSTT.prewarm()and neither forwards it, so the prewarm issued byAgentActivity(voice/agent_activity.py:608,724) never reaches the wrapped instances. Providers that implement prewarming — currently the mistralai realtime STT, which warms its connection pool — are therefore never warmed behind either adapter.The TTS equivalents already forward it (
tts/fallback_adapter.py:126-128,tts/stream_adapter.py:68-69), so this brings the STT path in line.Why both hops
They interact rather than being two independent nits.
FallbackAdaptertransparently wraps non-streaming STTs in aStreamAdapter(stt/fallback_adapter.py:74-76), so on that path prewarm has to survive the adapter and the wrapper to reach the provider — fixing onlyFallbackAdapterleaves it swallowed one level down.test_prewarm_reaches_non_streaming_primarycovers exactly that chain and still fails with only theFallbackAdapterhalf applied.As in #6582, only the primary instance is prewarmed: the remaining instances are not expected to serve traffic unless it fails.
Tests
New
tests/test_stt_prewarm.py:test_fallback_adapter_prewarms_primary_stt— primary warmed, fallbacks stay cold.test_stream_adapter_forwards_prewarm—StreamAdapterforwards to the wrapped STT.test_prewarm_reaches_non_streaming_primary— prewarm survives both hops when the primary is non-streaming and gets auto-wrapped.All three fail on
main. With only theFallbackAdapterfix, the third still fails, which is what motivates touchingStreamAdaptertoo.ruff check,ruff format --checkandmypy --strictare clean on the touched files, andtest_stt_prewarm.py,test_stt_fallback.py,test_stt_base.pyandtest_tts_fallback.pypass together (20 passed).