Skip to content

Update Cerebras models and default - #6593

Open
ryanl-cerebras wants to merge 2 commits into
livekit:mainfrom
ryanl-cerebras:agent/update-cerebras-model-ids
Open

Update Cerebras models and default#6593
ryanl-cerebras wants to merge 2 commits into
livekit:mainfrom
ryanl-cerebras:agent/update-cerebras-model-ids

Conversation

@ryanl-cerebras

Copy link
Copy Markdown

Summary

  • Adds zai-glm-4.7 and gemma-4-31b to both Cerebras model types and changes the dedicated plugin default to gpt-oss-120b.
  • The current public catalog is gpt-oss-120b, zai-glm-4.7, and gemma-4-31b.

Why

The previous model ID was retired or the fixed catalog was missing a currently available Cerebras model. This could produce failed requests, stale autocomplete, or misleading examples.

Validation

@CLAassistant

CLAassistant commented Jul 28, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@ryanl-cerebras
ryanl-cerebras marked this pull request as ready for review July 29, 2026 15:29
@ryanl-cerebras
ryanl-cerebras requested a review from a team as a code owner July 29, 2026 15:29

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 1 potential issue.

View 2 additional findings in Devin Review.

Open in Devin Review

"llama3.1-8b",
"llama-3.3-70b",
"llama-4-scout-17b-16e-instruct",
"llama-4-maverick-17b-128e-instruct",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 Cerebras helper still defaults to a retired model

The Cerebras convenience constructor still defaults to a model (with_cerebras at livekit-plugins/livekit-plugins-openai/livekit/plugins/openai/llm.py:267) that this change removes from the supported catalog, so calling it without naming a model targets a model that no longer exists.
Impact: Anyone using the built-in Cerebras helper without specifying a model gets failed requests until they manually pick a model.

Removed model still referenced as default in with_cerebras

This PR deletes "llama-4-scout-17b-16e-instruct" (and other models) from CerebrasChatModels in livekit-plugins/livekit-plugins-openai/livekit/plugins/openai/models.py:110-114, leaving only gpt-oss-120b, zai-glm-4.7, and gemma-4-31b. However OpenAILLM.with_cerebras() at livekit-plugins/livekit-plugins-openai/livekit/plugins/openai/llm.py:267 still uses model: str | CerebrasChatModels = "llama-4-scout-17b-16e-instruct" as its default. Since the type is str | CerebrasChatModels, mypy will not catch this, but at runtime the retired model id is sent to the Cerebras API and requests fail. The default should be updated to one of the currently-available models (e.g. gpt-oss-120b), consistent with the dedicated plugin's new default.

Prompt for agents
The PR removes llama-4-scout-17b-16e-instruct and other models from the CerebrasChatModels literal, leaving only gpt-oss-120b, zai-glm-4.7 and gemma-4-31b. But OpenAILLM.with_cerebras() in livekit-plugins/livekit-plugins-openai/livekit/plugins/openai/llm.py (around line 267) still defaults model to "llama-4-scout-17b-16e-instruct", which no longer exists in the Cerebras catalog. Update that default to a currently-supported model such as "gpt-oss-120b" so that calling with_cerebras() without an explicit model does not send a retired model id to the API.
Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants