sglang backend: pass through thinking_budget + require_reasoning

[pos-ei-don]

Motivation

sglang's raw Engine.async_generate() API — which backend/python/sglang/backend.py calls
directly, bypassing sglang's own OpenAI-compatible HTTP server — supports a precise,
tokenizer-derived reasoning-length budget via
sampling_params["custom_params"]["thinking_budget"] plus require_reasoning=True, gated
behind sglang's --enable-strict-thinking engine arg (already passable today via this
backend's existing generic engine_args: passthrough). Neither of the two request-side pieces
was reachable through LocalAI:

• _build_sampling_params() only builds from a fixed proto-field mapping (temperature, top_p,
  …) — there was no way to add a custom_params key at all.
• _predict() never passed require_reasoning to async_generate(), so even with a budget
  set, sglang's strict-thinking grammar backend wouldn't know a request is inside a reasoning
  block.

This came up wiring a self-hosted Qwen3.6-35B-A3B onto sglang: without this, the only way to
bound reasoning length is reasoning_effort (a blunt LocalAI-side workaround, does not reach
the backend at all today) or an unbounded/hardcoded-token-ID mechanism
(Qwen3ThinkingBudgetLogitProcessor) that's wrong for current Qwen3.x vocabularies.

Change

• LoadModel reads a new model-level thinking_budget option, using the exact same mechanism
  already used for tool_parser/reasoning_parser (per-model YAML options:, not a new proto
  field). _build_sampling_params() adds it as custom_params.thinking_budget on every request
  when configured — mirroring how sglang's own --preferred-sampling-params is a server-wide
  default rather than a per-request choice.
• _new_reasoning_parser() already inspects the rendered prompt to detect whether the chat
  template pre-opened a reasoning block (Qwen3-style templates append <think> to the prompt
  instead of the model emitting it) — that's exactly the signal sglang's own OpenAI server
  derives per-template to set require_reasoning. This backend has no template manager, so it
  now returns that signal alongside the parser, and _predict forwards it to
  async_generate(require_reasoning=...).

Scope note

require_reasoning is derived only from the existing prompt-suffix heuristic, not sglang's
full per-template _get_reasoning_from_request decision tree (there are model-family special
cases there, e.g. minimax-m3, hunyuan). This backend has no template-manager equivalent to
evaluate that tree against, and the prompt-suffix check is the one heuristic already validated
in this file's own tests. Happy to extend if that gap matters for a concrete model.

Verification

• Verified against a real production stack (NVFP4 quantization, GB10/sm_121,
  Qwen3.6-35B-A3B, --enable-strict-thinking --grammar-backend xgrammar) via a raw
  Engine.async_generate() call bypassing this backend entirely: custom_params: {"thinking_budget": 300} + require_reasoning=True → 301 reasoning tokens, clean
  finish_reason: stop, ~27s.
• Not yet verified end-to-end through this backend's own gRPC path — no local CUDA/sglang
  environment available here to load a real model through LocalAI itself.
• backend/python/sglang/test.py: fixed the two existing call sites for
  _new_reasoning_parser()'s now-tuple return, and added two new tests
  (test_thinking_budget_added_to_sampling_params_as_custom_params,
  test_no_thinking_budget_means_no_custom_params_key) covering the new
  _build_sampling_params() behavior. Could not run the suite locally (needs sglang/torch);
  happy to address CI feedback.

Related

Complements sgl-project/sglang#40634 (a preferred_sampling_params/custom_params merge bug
found while investigating this — fixed upstream in sglang itself, independent of this change).

Dependency note (added 2026-09-22)

This backend does not load at all against sglang ≥ 0.5.20 without #12155
(ServerArgs moved from a dataclass to a msgspec.Struct there; the
dataclasses.fields(ServerArgs) validation in backend/python/sglang/backend.py
raises TypeError on model load, unconditionally). This PR was developed and tested
against sglang 0.5.20, so it implicitly depends on #12155 (or an equivalent local patch)
being applied first. Rebasing onto #12155 once that merges, or noting the dependency
explicitly, would avoid a maintainer hitting a load failure unrelated to this change.

End-to-end verification (added 2026-09-22)

Verified through this backend's own gRPC path this time (not bypassed): a real LocalAI +
sglang 0.5.20 stack on the same production hardware (NVFP4, GB10/sm_121, Qwen3.6-35B-A3B),
model YAML option thinking_budget:N, request-level reasoning_effort. Confirmed the
sampling_params passthrough works and the budget is honored (e.g. thinking_budget:256 →
~245 reasoning tokens; thinking_budget:128 → ~123). One operational caveat found while
doing this: options: (this backend's model-level config, including thinking_budget) is
read once when the backend process loads a model and is not re-read by LocalAI's
/models/reload (config-only reload) or even a per-model backend restart — only a full
LocalAI process restart re-reads the model YAML and picks up a new options: value. Not a
bug in this PR, but worth a doc note for anyone tuning thinking_budget iteratively: a
value change needs a full container/process restart to take effect, not just a model
reload.