Skip to content

[OMEGA-522] fix(providers): forward reasoning_mode in base AIProvider (fixes #402) - #403

Open
watchme35 wants to merge 5 commits into
singnet:mainfrom
watchme35:fix/reasoning-mode-openaiapi
Open

watchme35 wants to merge 5 commits into
singnet:mainfrom
watchme35:fix/reasoning-mode-openaiapi

Conversation

@watchme35

@watchme35 watchme35 commented Oct 20, 2026 •

Copy link
Copy Markdown

Description

Fixes #402.

The base AIProvider.convert_request in providers/lib_llm_ext.py built the request body from model, messages, max_tokens, tools and tool_choice only, silently dropping request.reasoning_mode. As a result, the OpenAIAPI provider (and any provider delegating to the base implementation) sent no reasoning hint to OpenAI-compatible backends such as llama.cpp server, vLLM, or Strata.

Thinking-capable models then ran at the backend's default thinking level. Thinking tokens share the output budget, so long thinking chains exhaust it: finish_reason=length with empty content, and the user sees the "reasoning exceeded the token limit" message - which advises lowering reasoningMode, a setting that had no effect on this provider.

The fix forwards the mode as extra_body.reasoning_effort, the parameter these servers accept ("none" disables thinking). Plain OpenAI chat-completions endpoints ignore unknown fields, so existing users are unaffected. The OpenAI, OpenRouter and ASIonE providers already map the mode themselves:

  • OpenRouter merges the base extra_body with its own reasoning object (both keys are consistent);
  • ASIonE replaces extra_body with its thinking budget exactly as before.

Three unit tests are added for the base provider: forwarding medium and none, and omitting extra_body entirely when reasoning_mode is unset.

Review follow-up (thanks @timur-ashkenov): with the shipped default reasoningMode: medium, the base provider sends reasoning_effort to every model, including non-thinking models that reject the parameter (non-reasoning OpenAI models answer 400: "reasoning.effort is not supported with this model"). Added a supportsReasoningEffort config flag (default true): set to false, the parameter is omitted entirely, so such deployments keep the pre-PR request shape. OpenRouter and ASIOne are unaffected - they send their own reasoning parameters as before. Two more unit tests cover the omitted and sent cases.

How Has This Been Tested?

Checklist

  • PR contains autogenerated code
  • Self-review completed
  • Test scenarios above are passed with the version of the code from PR

The base AIProvider.convert_request in providers/lib_llm_ext.py ignored
request.reasoning_mode, so the OpenAIAPI provider sent no reasoning hint
to OpenAI-compatible backends (llama.cpp server, vLLM, Strata, ...).
Thinking-capable models then ran at the server's default thinking level
and could exhaust the shared output budget, producing finish_reason=length
with empty content and the misleading "reasoning exceeded the token limit"
reply, which advises lowering a setting that had no effect.

Forward the mode as extra_body.reasoning_effort, the parameter these
servers accept ("none" disables thinking). Plain OpenAI chat-completions
endpoints ignore unknown fields. The OpenAI, OpenRouter and ASIonE
providers already map the mode themselves and are unaffected:
OpenRouter merges the base extra_body with its own reasoning object,
ASIonE replaces it with its thinking budget as before.

Adds unit tests for the base provider (forward medium/none, omit when
reasoning_mode is unset).

Fixes singnet#402
@aquashade aquashade changed the title fix(providers): forward reasoning_mode in base AIProvider (fixes #402) [OMEGA-522] fix(providers): forward reasoning_mode in base AIProvider (fixes #402) Oct 21, 2026
Comment thread providers/lib_llm_ext.py
# OpenAI-compatible servers (llama.cpp, vLLM, Strata, ...) accept
# reasoning_effort; "none" disables thinking. Plain OpenAI chat
# completions endpoints ignore unknown fields.
result["extra_body"] = {"reasoning_effort": str(reasoning).lower()}

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@watchme35 With the default reasoningMode: medium, this now sends reasoning_effort to every model using the base provider, including non-thinking models. Could we preserve compatibility by allowing this parameter to be omitted for unsupported models, and add coverage for that case?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch — thanks for the review. Added a supportsReasoningEffort config flag (default true): set to false, the base provider omits reasoning_effort entirely, so deployments pointing OpenAIAPI at models that reject the parameter (non-reasoning OpenAI models answer 400: reasoning.effort is not supported with this model) keep working exactly as before this PR. OpenRouter and ASIOne are unaffected — they send their own reasoning parameters as before. Added unit tests for both the omitted and sent cases; full unit suite passes (147).

…fort

With the default reasoningMode: medium, the base provider sends
reasoning_effort to every model, including non-thinking models that
reject the parameter (non-reasoning OpenAI models answer 400). Add a
supportsReasoningEffort config flag (default true): set to false, the
parameter is omitted, keeping the pre-PR request shape for those
deployments. OpenRouter and ASIOne are unaffected. Two unit tests
cover the omitted and sent cases.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[OMEGA-522] OpenAIAPI provider drops reasoningMode from the request body (base AIProvider.convert_request ignores request.reasoning_mode)

3 participants