Repository navigation
Conversation
The base AIProvider.convert_request in providers/lib_llm_ext.py ignored
request.reasoning_mode, so the OpenAIAPI provider sent no reasoning hint
to OpenAI-compatible backends (llama.cpp server, vLLM, Strata, ...).
Thinking-capable models then ran at the server's default thinking level
and could exhaust the shared output budget, producing finish_reason=length
with empty content and the misleading "reasoning exceeded the token limit"
reply, which advises lowering a setting that had no effect.
Forward the mode as extra_body.reasoning_effort, the parameter these
servers accept ("none" disables thinking). Plain OpenAI chat-completions
endpoints ignore unknown fields. The OpenAI, OpenRouter and ASIonE
providers already map the mode themselves and are unaffected:
OpenRouter merges the base extra_body with its own reasoning object,
ASIonE replaces it with its thinking budget as before.
Adds unit tests for the base provider (forward medium/none, omit when
reasoning_mode is unset).
Fixes singnet#402
| # OpenAI-compatible servers (llama.cpp, vLLM, Strata, ...) accept | ||
| # reasoning_effort; "none" disables thinking. Plain OpenAI chat | ||
| # completions endpoints ignore unknown fields. | ||
| result["extra_body"] = {"reasoning_effort": str(reasoning).lower()} |
There was a problem hiding this comment.
@watchme35 With the default reasoningMode: medium, this now sends reasoning_effort to every model using the base provider, including non-thinking models. Could we preserve compatibility by allowing this parameter to be omitted for unsupported models, and add coverage for that case?
There was a problem hiding this comment.
Good catch — thanks for the review. Added a supportsReasoningEffort config flag (default true): set to false, the base provider omits reasoning_effort entirely, so deployments pointing OpenAIAPI at models that reject the parameter (non-reasoning OpenAI models answer 400: reasoning.effort is not supported with this model) keep working exactly as before this PR. OpenRouter and ASIOne are unaffected — they send their own reasoning parameters as before. Added unit tests for both the omitted and sent cases; full unit suite passes (147).
…fort With the default reasoningMode: medium, the base provider sends reasoning_effort to every model, including non-thinking models that reject the parameter (non-reasoning OpenAI models answer 400). Add a supportsReasoningEffort config flag (default true): set to false, the parameter is omitted, keeping the pre-PR request shape for those deployments. OpenRouter and ASIOne are unaffected. Two unit tests cover the omitted and sent cases.
Description
Fixes #402.
The base
AIProvider.convert_requestinproviders/lib_llm_ext.pybuilt the request body frommodel,messages,max_tokens,toolsandtool_choiceonly, silently droppingrequest.reasoning_mode. As a result, theOpenAIAPIprovider (and any provider delegating to the base implementation) sent no reasoning hint to OpenAI-compatible backends such as llama.cpp server, vLLM, or Strata.Thinking-capable models then ran at the backend's default thinking level. Thinking tokens share the output budget, so long thinking chains exhaust it:
finish_reason=lengthwith empty content, and the user sees the "reasoning exceeded the token limit" message - which advises loweringreasoningMode, a setting that had no effect on this provider.The fix forwards the mode as
extra_body.reasoning_effort, the parameter these servers accept ("none"disables thinking). Plain OpenAI chat-completions endpoints ignore unknown fields, so existing users are unaffected. TheOpenAI,OpenRouterandASIonEproviders already map the mode themselves:extra_bodywith its ownreasoningobject (both keys are consistent);extra_bodywith its thinking budget exactly as before.Three unit tests are added for the base provider: forwarding
mediumandnone, and omittingextra_bodyentirely whenreasoning_modeis unset.Review follow-up (thanks @timur-ashkenov): with the shipped default
reasoningMode: medium, the base provider sendsreasoning_effortto every model, including non-thinking models that reject the parameter (non-reasoning OpenAI models answer 400: "reasoning.effort is not supported with this model"). Added asupportsReasoningEffortconfig flag (defaulttrue): set tofalse, the parameter is omitted entirely, so such deployments keep the pre-PR request shape. OpenRouter and ASIOne are unaffected - they send their own reasoning parameters as before. Two more unit tests cover the omitted and sent cases.How Has This Been Tested?
main@fccc74dwith the patch applied:pytest Autotests/unit-> 147 passed (142 pre-existing + 5 new).extra_body={"reasoning_effort": ...}) has been running in a production Omega v0.1.20 deployment (Docker,OpenAIAPIprovider against a local llama.cpp/Strata proxy) since 2026-10-14, where it resolved the empty-response failures described in [OMEGA-522] OpenAIAPI provider drops reasoningMode from the request body (base AIProvider.convert_request ignores request.reasoning_mode) #402.Checklist