Update Cerebras models and default - #6593
Conversation
| "llama3.1-8b", | ||
| "llama-3.3-70b", | ||
| "llama-4-scout-17b-16e-instruct", | ||
| "llama-4-maverick-17b-128e-instruct", |
There was a problem hiding this comment.
🔴 Cerebras helper still defaults to a retired model
The Cerebras convenience constructor still defaults to a model (with_cerebras at livekit-plugins/livekit-plugins-openai/livekit/plugins/openai/llm.py:267) that this change removes from the supported catalog, so calling it without naming a model targets a model that no longer exists.
Impact: Anyone using the built-in Cerebras helper without specifying a model gets failed requests until they manually pick a model.
Removed model still referenced as default in with_cerebras
This PR deletes "llama-4-scout-17b-16e-instruct" (and other models) from CerebrasChatModels in livekit-plugins/livekit-plugins-openai/livekit/plugins/openai/models.py:110-114, leaving only gpt-oss-120b, zai-glm-4.7, and gemma-4-31b. However OpenAILLM.with_cerebras() at livekit-plugins/livekit-plugins-openai/livekit/plugins/openai/llm.py:267 still uses model: str | CerebrasChatModels = "llama-4-scout-17b-16e-instruct" as its default. Since the type is str | CerebrasChatModels, mypy will not catch this, but at runtime the retired model id is sent to the Cerebras API and requests fail. The default should be updated to one of the currently-available models (e.g. gpt-oss-120b), consistent with the dedicated plugin's new default.
Prompt for agents
The PR removes llama-4-scout-17b-16e-instruct and other models from the CerebrasChatModels literal, leaving only gpt-oss-120b, zai-glm-4.7 and gemma-4-31b. But OpenAILLM.with_cerebras() in livekit-plugins/livekit-plugins-openai/livekit/plugins/openai/llm.py (around line 267) still defaults model to "llama-4-scout-17b-16e-instruct", which no longer exists in the Cerebras catalog. Update that default to a currently-supported model such as "gpt-oss-120b" so that calling with_cerebras() without an explicit model does not send a retired model id to the API.
Was this helpful? React with 👍 or 👎 to provide feedback.
Summary
gpt-oss-120b,zai-glm-4.7, andgemma-4-31b.Why
The previous model ID was retired or the fixed catalog was missing a currently available Cerebras model. This could produce failed requests, stale autocomplete, or misleading examples.
Validation