feat: add StepFun speech and chat plugin by robinzsx · Pull Request #7977 · livekit/agents

robinzsx · 2026-10-20T08:37:13Z

Summary

Adds the official StepFun plugin for LiveKit Agents with support for:

StepAudio streaming WebSocket ASR
StepAudio batch HTTP+SSE ASR
StepAudio streaming TTS
StepFun OpenAI-compatible chat LLM
Global and mainland-China API clusters
English and Chinese language configuration
Provider lifecycle, reconnect, and error handling
Comprehensive STT, TTS, and LLM tests

Highlights

Speech-to-text

Streaming model: stepaudio-2.5-asr-stream
Batch model: stepaudio-3-asr-max
Streaming WebSocket endpointing
Batch HTTP+SSE recognition
Local VAD compatibility through StreamAdapter
Proper handling of provider transcript deltas, final events, usage, and
session identifiers

Text-to-speech

Streaming StepAudio TTS over WebSocket
One-shot/chunked synthesis support
Voice catalogue support
Reconnect and stale socket recovery
WebSocket close-code reporting

LLM

OpenAI-compatible StepFun chat integration
Supports:
step-5-preview
step-3.7-flash
step-3.5-flash
Supports documented reasoning_effort values for
step-5-preview and step-3.7-flash
Optional livekit-plugins-openai dependency

Regional clusters

Supports:

global → api.stepfun.ai
china → api.stepfun.com

The cluster can be configured through STEPFUN_CLUSTER or the constructor
arguments.

Testing

Validated locally with:

156 StepFun plugin tests passed
Ruff lint passed
Ruff format check passed
Mypy passed for livekit.plugins.stepfun
uv lock --check passed
git diff --check passed
Branch rebased cleanly onto current upstream/main

A complete runnable demo and recipe to build a voice agent using this plugin is available in my public repo:
https://github.com/robinzsx/livekit-plugins-stepfun

You can follow the README in the demo repo to test the end-to-end voice agent workflow.

Checklist

Added plugin implementation
Added plugin tests
Added package workspace registration
Updated lockfile
Ran lint, formatting, type checking, and targeted tests

CLAassistant · 2026-10-21T03:10:39Z

All committers have signed the CLA.

Add WebSocket and batch SSE speech recognition, streaming and HTTP SSE speech synthesis, and an optional OpenAI-compatible chat wrapper. Support regional endpoints, provider error handling, and conversational stream lifecycle management with regression coverage.

The streaming protocol never says which audio a final covers, so a reconnect cannot know what to resend: resending everything repeats speech the server already transcribed, and resending nothing drops the speech it had not. Retrying on a guess trades one failure for the other, so audio is consumed without being retained and a connection lost after any of it was uploaded — or while a turn is still pending — is reported as non-retryable. Recorded audio belongs on the batch recognize() route until the server can identify what a final covers. Also drop the redundant pool.remove on a failed session handshake, since the pool's context manager already drops the connection, and stop the keepalive's ping failure from vanishing with its task.

robinzsx · 2026-10-22T03:03:57Z

Status update: all review findings are addressed.

The test_google_long_decode.py unit test failure is unrelated to this PR
TestRequestShaping.test_sampling_params_stay_dropped_across_calls fails at
teardown, not in its body, and nothing in this branch touches the Google plugin

robinzsx requested a review from a team as a code owner October 20, 2026 08:37