Bug Description
The liveavatar plugin's WebSocket event handler (avatar.py's _recv_task) only recognizes 4 event types: session.state_updated, agent.speak_started, agent.speak_ended, agent.speak_interrupted. During a real session, the LiveAvatar server sends different event types that all fall through to the "unhandled" branch and are silently ignored:
agent.audio_buffer_appended (repeated many times per response)
agent.audio_buffer_committed
agent.state_updated
agent.audio_buffer_cleared
Only the very first "connected" message on session start used the schema the plugin expects (session.state_updated); every event during the actual conversation used this different, unrecognized schema instead.
As a result, _handle_agent_speak_started and _handle_agent_speak_ended never fire during a real conversation, so self._avatar_speaking / self._avatar_interrupted never update, and QueueAudioOutput.notify_playback_started() / notify_playback_finished() are never called from server acknowledgment. This most directly affects _on_clear_buffer's barge-in logic, which checks if self._avatar_speaking: before sending agent.interrupt — since that flag never becomes True, interrupting the avatar mid-response may not correctly signal LiveAvatar's server to stop.
Expected Behavior
The plugin should recognize whatever event schema the LiveAvatar server currently sends, so _avatar_speaking and playback-start/finish notifications correctly reflect the avatar's real state, and barge-in/interruption works as intended.
Reproduction Steps
1. Set up a LiveKit agent using livekit.plugins.liveavatar.AvatarSession (is_sandbox=True), following the standard pattern (AgentSession with any STT/LLM/TTS, await avatar.start(session, room), then await session.start(agent=..., room=...)).
2. Set the livekit.plugins.liveavatar logger to DEBUG.
3. Connect via the Agents Playground and have a short back-and-forth conversation.
4. Observe the log — every event past the initial "connected" message logs as Unhandled LiveAvatar event: agent.audio_buffer_appended / agent.audio_buffer_committed / agent.state_updated / agent.audio_buffer_cleared.
Operating System
Operating System: macOS 26.6.2 (Darwin 25.6.0, arm64)
Models Used
STT: Deepgram (default model) LLM: google/gemini-3-flash via LiveKit Inference TTS: cartesia/sonic-3 via LiveKit Inference Avatar: HeyGen LiveAvatar, LITE mode, sandbox
Package Versions
- livekit-plugins-liveavatar: 1.8.0
- livekit-agents: 1.8.0
Session/Room/Call IDs
room: console-320ff264
job_id: AJ_ogpRGmyexCVC
Proposed Solution
Update _recv_task's event dispatch in avatar.py to also handle agent.audio_buffer_appended, agent.audio_buffer_committed, agent.audio_buffer_cleared, and agent.state_updated, mapping them to the equivalent _avatar_speaking/playback-notification behavior currently tied to agent.speak_started/agent.speak_ended. Since the exact intended semantics of the new event names aren't publicly documented, this likely needs input from LiveKit/HeyGen on the current protocol spec before the mapping can be written correctly.
Additional Context
Discovered while investigating a separate, still-unconfirmed report that the avatar appears static (no movement) during a live conversation despite the avatar correctly being told it's in a "listening" state (via agent.start_listening, which is triggered client-side independent of this bug). This event-handling gap doesn't confirm or rule out that issue, but is a real, separate compatibility gap worth fixing regardless.
Screenshots and Recordings
No response
Bug Description
The liveavatar plugin's WebSocket event handler (
avatar.py's _recv_task) only recognizes 4 event types:session.state_updated, agent.speak_started, agent.speak_ended, agent.speak_interrupted. During a real session, the LiveAvatar server sends different event types that all fall through to the "unhandled" branch and are silently ignored:Only the very first "connected" message on session start used the schema the plugin expects (session.state_updated); every event during the actual conversation used this different, unrecognized schema instead.
As a result, _handle_agent_speak_started and _handle_agent_speak_ended never fire during a real conversation, so self._avatar_speaking / self._avatar_interrupted never update, and QueueAudioOutput.notify_playback_started() / notify_playback_finished() are never called from server acknowledgment. This most directly affects _on_clear_buffer's barge-in logic, which checks if self._avatar_speaking: before sending agent.interrupt — since that flag never becomes True, interrupting the avatar mid-response may not correctly signal LiveAvatar's server to stop.
Expected Behavior
The plugin should recognize whatever event schema the LiveAvatar server currently sends, so _avatar_speaking and playback-start/finish notifications correctly reflect the avatar's real state, and barge-in/interruption works as intended.
Reproduction Steps
Operating System
Operating System: macOS 26.6.2 (Darwin 25.6.0, arm64)
Models Used
STT: Deepgram (default model) LLM: google/gemini-3-flash via LiveKit Inference TTS: cartesia/sonic-3 via LiveKit Inference Avatar: HeyGen LiveAvatar, LITE mode, sandbox
Package Versions
Session/Room/Call IDs
room: console-320ff264
job_id: AJ_ogpRGmyexCVC
Proposed Solution
Additional Context
Discovered while investigating a separate, still-unconfirmed report that the avatar appears static (no movement) during a live conversation despite the avatar correctly being told it's in a "listening" state (via agent.start_listening, which is triggered client-side independent of this bug). This event-handling gap doesn't confirm or rule out that issue, but is a real, separate compatibility gap worth fixing regardless.
Screenshots and Recordings
No response