Skip to content

Expose reasoning controls for Anthropic/Opus models #690

@PierreLeGuen

Description

@PierreLeGuen

Problem

NEAR AI Cloud exposes Anthropic Opus models, but reasoning controls are not currently effective for them through our OpenAI-compatible API surface.

Prod behavior observed:

  • openai/gpt-5.5 rejects an invalid reasoning_effort, which confirms GPT/OpenAI-compatible requests forward and validate the field upstream.
  • anthropic/claude-opus-4-7 accepts both valid and invalid reasoning_effort values and returns normally.
  • anthropic/claude-opus-4-7 also accepts an Anthropic-style thinking object through /v1/chat/completions, but returns normally with no evidence that the field is forwarded.

This means Opus appears to run with default behavior only, even when callers ask for an explicit reasoning/thinking level.

What is ours to fix

The OpenAI-compatible backend preserves flattened extra params, but the Anthropic backend rebuilds an Anthropic Messages request and only maps known fields. It currently does not map reasoning-related inputs from the Cloud API request into Anthropic's request shape.

Relevant code areas:

  • crates/inference_providers/src/external/anthropic/mod.rs
  • crates/inference_providers/src/external/anthropic/converter.rs
  • crates/api/src/models.rs
  • crates/services/src/responses/models.rs

Expected behavior

For Anthropic-backed models that support thinking/reasoning, callers should have a supported way to request it, and unsupported/invalid values should not be silently ignored.

Possible acceptance criteria:

  • Define the supported public input shape for Anthropic reasoning controls on Cloud API.
  • Map supported values into the Anthropic Messages request thinking config where applicable.
  • Reject or clearly ignore unsupported fields/values instead of silently accepting them.
  • Add coverage for Opus/Anthropic request serialization so valid thinking config is forwarded and invalid values are handled intentionally.

Related note

GPT + function tools on OpenAI /chat/completions is a separate upstream API limitation. This issue is specifically about the Anthropic/Opus adapter dropping or ignoring reasoning controls.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions