Skip to content

Add per-chat reasoning effort and verbosity controls #3971

Description

@uhhgoat

Prerequisites

  • I searched open issues and pull requests and did not find this exact UI/runtime-control proposal.
  • I searched discussions and did not find an active discussion covering this proposal.
  • This is a concrete, actionable feature request.

Area

Chat / Agent

Problem / Motivation

Odysseus can connect to hosted and local models that expose reasoning, thinking, or response-style controls, but users do not currently have a compact per-chat way to set reasoning effort or final-answer verbosity from the composer.

Existing related work covers adjacent pieces:

This proposal is the user-facing control layer: let users keep Auto defaults, optionally choose a supported reasoning/thinking level for the selected provider/model, and separately control final-answer verbosity where the provider supports it.

Without this, model comparisons are noisy: hosted reasoning models may run with unknown reasoning budget, local thinking models may think when not desired, and simple requests can produce overly verbose answers even when the user wants a concise final response.

Proposed Solution

Add two compact per-chat controls in the composer toolbar near the existing Agent/Chat, shell access, web search, and more-tools controls:

  • Reasoning: Auto, Off / None, Low, Medium, High, XHigh / Max where supported.
  • Verbosity: Auto, Low, Medium, High where supported.

Auto should preserve current behavior and omit provider-specific fields.

Use provider/model capability gating:

Provider mapping should be conservative and provider-specific:

  • OpenAI Responses-style reasoning models: map reasoning to reasoning.effort; map verbosity to text.verbosity.
  • Anthropic Claude extended thinking: map supported controls to the provider's thinking object/budget/adaptive-effort shape; do not expose unsupported verbosity unless implemented as an Odysseus instruction.
  • Google Gemini: map supported thinking controls to thinkingConfig, such as thinkingLevel or thinkingBudget where supported by the selected model/API version.
  • Ollama native and Ollama OpenAI-compatible thinking models: map binary thinking to native think where supported; keep it separate from graded reasoning_effort.
  • Self-hosted Qwen, DeepSeek, GLM, Granite, and similar models through vLLM, SGLang, llama.cpp, or compatible servers: add support only where backend-specific knobs are available, such as chat-template kwargs, enable_thinking, documented request extras, or model-specific directives.
  • xAI/Grok, Mistral, Cohere, Kimi, Z.ai/GLM, DeepSeek hosted, and similar providers: expose graded effort only after provider docs or deterministic capability metadata prove the compatible request shape.
  • Providers without native support: leave controls hidden/disabled, keep Auto, or optionally use an Odysseus-level response-style instruction for verbosity only, clearly marked as an instruction rather than a provider parameter.

Suggested first PR scope:

  1. Add composer controls with Auto defaults.
  2. Add request fields to /api/chat_stream.
  3. Thread settings through direct chat and agent mode.
  4. Add helper functions for normalizing and gating reasoning/verbosity settings.
  5. Apply the first provider payload mapping to one verified provider path, plus Ollama think only if the existing code path makes that safe.
  6. Add focused payload tests proving unsupported endpoints do not receive unsupported fields.
  7. Keep durable storage, Compare metadata, and broader provider support as follow-up if needed.

Alternatives Considered

  • Put controls only in Settings: less convenient for per-task evals and quick model comparisons.
  • Add only a binary thinking toggle: useful for local thinking models but does not cover graded effort or verbosity.
  • Send OpenAI-style fields to every OpenAI-compatible endpoint: unsafe because many local/proxy endpoints reject unknown fields.
  • Use prompt instructions for all providers: useful as a fallback for verbosity, but not equivalent to real provider-side reasoning-budget controls.

Prior Art / Related Issues

Are you willing to implement this?

Yes. I can open a PR.

Metadata

Metadata

Assignees

No one assigned

    Labels

    ready for reviewDescription complete — ready for maintainer review

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions