Skip to content

fix(models): fallback to application config default context size in /v1/models/capabilities (#12202) - #12216

Open
PINYOPATTANAWASANPORN wants to merge 1 commit into
mudler:masterfrom
PINYOPATTANAWASANPORN:fix/capabilities-default-context-size-12202
Open

PINYOPATTANAWASANPORN wants to merge 1 commit into
mudler:masterfrom
PINYOPATTANAWASANPORN:fix/capabilities-default-context-size-12202

Conversation

@PINYOPATTANAWASANPORN

Copy link
Copy Markdown

Description

This PR fixes #12202.

When a model has no explicit context_size configured in its YAML configuration, /v1/models/capabilities was unconditionally defaulting to DefaultContextSize (4096), ignoring the application-wide default context size configured under Settings -> Performance -> Default Context Size (appConfig.ContextSize).

Key Changes:

  • In ListModelCapabilitiesEndpoint (core/http/endpoints/openai/list_capabilities.go), if a model's cfg.ContextSize is unset/zero and appConfig.ContextSize > 0, fall back to appConfig.ContextSize so client capability discovery respects the configured default.
  • Added comprehensive unit test in core/http/endpoints/openai/list_capabilities_test.go asserting that when appConf.ContextSize is configured, models with unconfigured context size report appConf.ContextSize.

Notes for Reviewers

  • Surgical 38-line diff (+16 / 0 lines) with normalized LF line endings.
  • Backward compatible: if appConfig.ContextSize is 0 or unset, backend.EffectiveRequestContextSize(cfg) cleanly retains the 4096 default fallback.

Signed commits

  • Yes, I signed my commits.
  • Documentation updated (docs/content/) for user-facing changes, or not applicable

…ler#12202)

Honor appConfig.ContextSize in /v1/models/capabilities when model context_size is unset.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

/v1/models/capabilities -- wrong default for context_size

1 participant