feat(reasoning): per-model reasoning toggle — /think soft-switch (Category #1)

[nsgds]

Summary

A first, deliberately narrow step toward per-model reasoning control.

Scope — read this first: this implements exactly one toggle mechanism, the /think soft-switch (Category 1), which is only relevant to a small set of models — the Nemotron-VL family today (Qwen3 / Hunyuan use the same /think style). Every other model, and every other provider, is left byte-for-byte unchanged. It is intentionally a foundation, not a complete reasoning-control system.

Why a per-model mechanism at all: Odysseus has no per-model settings today — supports_tools, thinking_style, refresh config, etc. are all per-endpoint. But whether reasoning can be toggled (and how) is a property of the model, and one endpoint can serve several models that differ. So this adds a minimal per-model store — ModelEndpoint.reasoning_modes, a JSON map {model_id: "on"|"off"} (absent = "auto" = leave the model's default) — with the /think directive as its first consumer. The logic sits behind a single boundary, src/reasoning_control.py: when a model's preference is on, /think is injected into the latest user turn; auto/unknown models emit nothing, so a directive is never sent to a model that wouldn't understand it.

What this does NOT do (yet): the ecosystem has several other reasoning-toggle mechanisms. They're catalogued in reasoning_control.py as future extension points but are not implemented here:

• Category 2: system-prompt directive (detailed thinking on/off) — Llama-Nemotron (incl. nim-nano)
• Category 3: chat_template_kwargs.enable_thinking — Qwen3, DeepSeek-V3.1/3.2 self-host, GLM, Granite
• Category 4: native think bool — Ollama
• Category 5: structured {type: enabled|disabled} object — Anthropic, GLM/Z.ai, DeepSeek/Kimi/Cohere hosted
• graded reasoning_effort — a separate setting, not a binary toggle — OpenAI o-series/GPT-5, Grok, Mistral

So this is the smallest useful slice: the per-model store + one mechanism, structured so the rest can be added without reshaping.

Design was discussed and endorsed on #2737. It's opened as a shape-validation prototype (per that discussion) — not seeking to merge ahead of #2739: it currently uses a standalone reasoning_modes preference, and the "which family / does it reason?" lookup is isolated behind one function so it can re-key onto #2739's CAP_REASONING + capability vocabulary once that lands.

It complements the recently-merged #2449: that router handles reasoning output (response-side); this is the request-side on/off. They're in the same file (src/llm_core.py) but non-overlapping regions — our 9-line call block sits before the provider branches; the harmony router is downstream in the stream loop.

No UI in this PR: reasoning_modes is set via the model-endpoints API (PATCH /api/model-endpoints/{id}), like supports_tools today; a UI surface is a deliberate separate follow-up.

Target branch

• This PR targets dev, not main.

Linked Issue

Closes #4146

Part of #315

Parent tracker: #2737

Related: #4146, #2737 / #2739 (design discussion + the capability schema this will build on), #2449 (the merged response-side router this complements), and #2905 — the reported "I disabled Reasoning but the model still thinks and stays slow" bug, which this addresses for /think-style models (the display toggle only hides reasoning; this actually stops it).

Type of Change

• New feature (non-breaking)

Checklist

• I searched open issues and PRs — this is not a duplicate.
• This PR targets dev
• Changes are scoped to the above — one new module + a per-model column + minimal wiring; no unrelated changes.
• I actually ran it end-to-end (toggled /think on a live Nemotron-VL model and confirmed the reasoning block appears/disappears).

How to Test

1. Restart so the additive reasoning_modes migration runs (nullable column on model_endpoints).
2. Turn reasoning on for a Nemotron-VL model: PATCH /api/model-endpoints/{id} with body {"reasoning_modes": {"<model-id>": "on"}}.
3. Start a chat with that model → /think is injected into your message → the reasoning/thinking block appears.
4. Set it to "off" (or omit it) → no reasoning. Confirm other models/providers are unaffected — default auto sends nothing.
5. Unit tests: python -m pytest tests/test_reasoning_control.py — covers the directive mapping (only /think-dialect models, only on on), the no-leak guard (other models/providers get nothing), message injection (string + multimodal), and degrade-to-auto.

Visual / UI changes

No UI or CSS — backend + API only (reasoning_modes is configured via the model-endpoints API, like supports_tools). The visible effect is that a model set to on now surfaces its reasoning where it previously didn't.

• No new UI patterns; no emoji; no new color/spacing values.

[RaresKeY]

Thanks for opening #3031. I think the prototype is useful, and the clean boundary is:

vendor-specific reported shape -> canonical capability/control evidence -> runtime preference -> request mutation

My read is that #2739 should own the first two parts: capability/control evidence, source, confidence, and endpoint-scoped model identity. #3031 should own the last two parts: stored runtime preference (auto / on / off) and the request mutation that turns that preference into /think, think: false, enable_thinking, budgets, effort fields, etc.

We added a small #2739 schema piece for this: canonical reasoning control mechanisms and values. The important distinction is:

• CAP_REASONING: model-level capability. This model supports or produces reasoning.
• REASONING_CONTROL_* mechanism: endpoint/provider/engine control evidence. This serving path accepts a specific control shape, normalized as one of reasoning_message_directive, reasoning_system_directive, reasoning_template_kwarg, reasoning_native_bool, reasoning_structured_object, reasoning_budget, or reasoning_effort.
• REASONING_CONTROL_VALUE_*: provider-supported values for a mechanism, normalized as on, off, or auto. Here auto means the provider explicitly supports adaptive/dynamic/vendor-decided reasoning, not merely that Odysseus omitted a control.
• Odysseus runtime preference: user intent, also expressible as auto / on / off, but this belongs in feat(reasoning): per-model reasoning toggle — /think soft-switch (Category #1) #3031 and later gets resolved into request mutation.

So I think #3031 is the right follow-up direction, but it should stay draft until #2739 lands/rebases. After that, the main alignment changes I’d ask for are:

• key dispatch off feat(models): define capability schema and readers #2739 capability/control evidence instead of model-name substring matching;
• resolve settings by endpoint/stable model identity, not URL-only lookup, since one base URL can have multiple endpoint rows/keys/owners;
• keep runtime auto / on / off as preference state, not capability metadata;
• use feat(models): define capability schema and readers #2739’s provider-supported control value auto only when the endpoint explicitly reports or documents adaptive/dynamic/vendor-decided reasoning;
• keep the first mergeable slice narrow: either explicitly “default-off /think enablement” or implement both /think and /no_think with tests;
• add regression coverage for same-base-url endpoints and no-leak behavior to generic providers.

I do not think #2739 should absorb the toggle implementation. It should provide the evidence layer that #3031 consumes.

[nsgds]

Agreed on the split — #2739 owns the evidence (capability + control mechanism/value), #3031 owns the stored preference and the request mutation. Keeping it draft until #2739 lands/rebases, and I'll re-key the dispatch onto the control evidence then.

What's in now (doesn't depend on #2739):

• Preference vs capability (your asks 3/4): on/off/auto is stored as user intent only — ModelEndpoint.reasoning_modes is a {model_id: "on"|"off"} map, set via PATCH, which drops anything that isn't on/off. auto means "leave the model's default", not a provider-advertised adaptive mode; the adaptive/value semantics stay on the feat(models): define capability schema and readers #2739 side.
• Scope (your ask 5): narrowed and labeled explicitly as default-off /think enablement — only on injects /think for the Nemotron-VL family; off/auto inject nothing. /no_think (the off-direction for default-on families like Qwen3) is intentionally not in this slice; it's flagged in-code as the next increment.
• Tests (your ask 6): added same-base-url coverage — two endpoint rows on one base_url, preference read from the row that actually serves each model, and a model neither serves resolves to auto (no stray preference). Plus the existing check that non-/think models never receive the directive.

Deferred to post-#2739 (both isolated to one function, marked in-code):

• Dispatch (your ask 1): still a model-name match (_is_think_directive_model); the TODO(#2739) at that constant swaps it to REASONING_CONTROL_reasoning_message_directive once that evidence is wired at runtime.
• Identity (your ask 2): preference is resolved via the URL+model lookup the stream layer can see today (it has url+model, not the endpoint id). Same-base-url collisions are disambiguated by which row serves the model, but resolving by stable endpoint/model identity is deferred until feat(models): define capability schema and readers #2739's evidence is threaded through.

Agreed too that #2739 shouldn't absorb the toggle — this stays the consumer of that evidence.

[RaresKeY]

Now that #2739 has merged, please rebase this branch onto current dev, complete the two alignment items already discussed in this thread, and address the three additional issues below before moving it out of draft. I checked the latest head on a clean current-dev rebase; it applies mechanically, but it still relies on model-name/URL matching instead of canonical control evidence and stable endpoint/model identities.

Findings

issue (runtime): Apply the reasoning preference to non-streaming requests

• Problem: Preference resolution and /think injection run only in _stream_llm_inner(). llm_call() and the normal llm_call_async() path build payloads from unchanged messages, while public POST /api/chat calls llm_call_async().

• Impact: The same stored endpoint/model preference works in streaming chat but is silently ignored by non-streaming chat and the many sync/async consumers. An offline request capture with mode on sent hello instead of /think hello.

• Ask: Move preference resolution and message mutation into a shared preparation boundary used by sync, async, and streaming builders, before sync/async cache-key calculation. Add payload-parity tests for all three paths and a cache-transition regression.

• Location: src/llm_core.py:1762, src/llm_core.py:1924, src/llm_core.py:2159, routes/chat_routes.py:504

issue (security): Redact endpoint credentials from reasoning-control error logs

• Problem: reasoning_mode_for() catches lookup or preference-decode failures and logs endpoint_url verbatim, bypassing the repository's core.log_safety.redact_url() contract for configured URLs.

• Impact: With DEBUG logging enabled, a reasoning lookup failure can persist URL userinfo or query credentials in console/application logs. An offline probe confirmed both classes of fake marker were emitted.

• Ask: Redact or omit the endpoint URL before logging it, and add an error-path regression proving URL userinfo, query, and fragment data never reach the log record.

• Location: src/reasoning_control.py:129

issue (correctness): Match an actual directive token when checking idempotency

• Problem: inject_directive() treats any substring match as an already-present directive. A prompt containing /thinker is left unchanged even though it has no /think control token; multimodal text uses the same check.

• Impact: An enabled preference can silently fail for otherwise ordinary prompts that happen to contain the character sequence.

• Ask: Recognize an exact/leading directive token instead of an arbitrary substring, and cover false positives in both string and multimodal content.

• Location: src/reasoning_control.py:103 and src/reasoning_control.py:106

Validation

• Focused combined-tree suites passed, including a 256-test model/route set; changed modules also compiled.
• Offline payload and logging probes reproduced both P2s.
• The full isolated suite reported 4,671 passed, 5 skipped, and the same 12 unrelated runner/baseline failures seen outside this change.
• Not run: live Nemotron/vLLM, full app/browser, existing-database upgrade, or GitHub CI on a post-#2739 public head.

PR Hygiene

The raw head and its checks predate #2739. After rebasing and addressing the alignment work and findings above, please refresh CI and request another review.