fix(stt): send the turn commit when a flush yields no buffered audio - #6598
Open
chenghao-mou wants to merge 3 commits into
Open
fix(stt): send the turn commit when a flush yields no buffered audio#6598chenghao-mou wants to merge 3 commits into
chenghao-mou wants to merge 3 commits into
Conversation
Scribe v2 realtime only finalizes a transcript once it receives a commit, so a VAD-driven end of turn left the agent waiting on the server's own (very late) commit. Send an empty commit=True chunk on flush. Fixes AGT-2264 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
AudioByteStream.flush() returns no frames when the buffer is empty, so a Finalize gated on the per-frame loop is skipped and has_ended leaks into the next utterance. Hoist it out of the loop, matching rtzr and the elevenlabs fix. stt_v2 gets the same dedent for its metrics flush. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
chenghao-mou
changed the title
Drives the real SpeechStream._run loop against a fake socket and asserts the commit/Finalize goes out after a flush, both when the repack buffer is empty (the regression) and when a partial chunk is still pending (ordering). Verified both fail with the fix reverted. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
AudioByteStream.flush()returns[]when its buffer happens to be empty, roughly 1 in 5 flushes with 10ms input frames repacked into 50ms chunks. Any finalize gated onfor frame in frames:is then skipped, andhas_endedleaks into the next utterance so the message fires mid-way through the following turn.elevenlabs: Scribe v2 realtime never finalized at all without a
commit, so send an emptycommit: truechunk, hoisted out of the frame loop.deepgram: hoist
Finalizeout of the frame loop (stt_v2gets the same dedent for its metrics flush).Tests: both plugins get coverage that drives the real
SpeechStream._runloop against a fake socket, asserting the commit lands after a flush with an empty repack buffer (the regression) and after the leftover audio when a partial chunk is pending (ordering). Both fail with the fix reverted.Scope: the default
stt_nodeonly callspush_frame, and the pipeline commits a turn by pushing silence, so nothing flushes a streaming STT stream in a normalAgentSession. Verified by patchingRecognizeStream.flush/end_inputacross the unit suite: 0 calls from the voice pipeline. This hardens directstt.stream()+flush()use; it does not on its own fix AGT-2264.Gladia has the same shape and is deliberately left alone:
stop_recordingis session-terminating there (seegladia/stt.py:388), so making it fire reliably per turn would be worse than the current miss.Related to AGT-2264 (#4087)
🤖 Generated with Claude Code