Skip to content

fix(providers): self-heal temperature rejection on the OpenAI-compatible path - #1246

Merged
warren618 merged 1 commit into
HKUDS:mainfrom
lorenzozanee:fix/openai-compat-temperature-self-heal
Aug 29, 2026
Merged

fix(providers): self-heal temperature rejection on the OpenAI-compatible path#1246
warren618 merged 1 commit into
HKUDS:mainfrom
lorenzozanee:fix/openai-compat-temperature-self-heal

Conversation

@lorenzozanee

Copy link
Copy Markdown
Contributor

Summary

  • Extend the Anthropic temperature self-heal (classify the 400, retry once, remember the model) to the generic OpenAI-compatible branch (ChatOpenAIWithReasoning), so requests to Anthropic models through OPENAI_BASE_URL recover from "temperature is deprecated for this model." automatically.
  • No change for models that accept temperature — the configured value keeps being sent.

Why

Next-generation Claude models (claude-opus-4-8, claude-sonnet-5, ...) reject any request carrying temperature with HTTP 400. At v0.1.14 the native Anthropic Messages path detects the error, retries without the field and remembers the model, but the generic OpenAI-compatible branch still sends temperature unconditionally, so running LANGCHAIN_PROVIDER=openai with OPENAI_BASE_URL=https://api.anthropic.com/v1/ fails on every call.

Closes #1223

Changes

  • ChatOpenAIWithReasoning._remember_temperature_unsupported classifies the 400 with the existing _is_anthropic_temperature_unsupported_error detector, logs once, and records the model in _ANTHROPIC_TEMPERATURE_UNSUPPORTED — the same set the native path uses.
  • _generate / _agenerate retry once without the field; _stream / _astream do the same (the 400 is raised before the first chunk, so the retry cannot duplicate output). The async stream_usage fallback moves to a named _astream_with_usage_fallback to mirror the sync _stream layering — no behavior change.
  • _get_request_payload omits temperature up front for remembered models, so later calls skip the doomed request.

Test Plan

  • Existing tests pass (pytest --ignore=agent/tests/e2e_backtest --tb=short -q — verified for the touched components: agent/tests/test_llm.py, agent/tests/test_provider_header_isolation.py, agent/tests/test_env_schema.py)
  • New tests added: mocked-error regression tests in agent/tests/test_llm.py covering invoke self-heal and the remembered-model fast path, supported-model preservation, stream self-heal, and async invoke; all fail on the pre-fix code with the verbatim 400
  • Tested manually (provider-dependent behavior verified with mocked 400/SSE responses via httpx.MockTransport, no live credentials)

Checklist

…ble path

Next-generation Claude models reject any request carrying `temperature`
with HTTP 400 "`temperature` is deprecated for this model." (issue
HKUDS#1223). The native Anthropic Messages path already classifies the error,
retries once without the field and remembers the model, but the generic
OpenAI-compatible branch still sent `temperature` unconditionally, so
LANGCHAIN_PROVIDER=openai with OPENAI_BASE_URL pointed at Anthropic's
compatibility endpoint failed on every call.

Extend the same self-heal to ChatOpenAIWithReasoning: reuse
_is_anthropic_temperature_unsupported_error and the process-wide
_ANTHROPIC_TEMPERATURE_UNSUPPORTED set, retry sync/async generate and
both stream paths once without the field, and omit temperature up front
for remembered models in _get_request_payload. _astream's stream_usage
fallback moves to a named _astream_with_usage_fallback to mirror the
sync layering; behavior is unchanged.

Models that accept `temperature` are unaffected. Regression tests feed
the verbatim 400 body through mocked endpoints and fail on the
pre-fix code.

Signed-off-by: lorenzozanee <wyz0707@proton.me>
@warren618
warren618 merged commit c403a74 into HKUDS:main Aug 29, 2026
9 checks passed
warren618 added a commit that referenced this pull request Aug 29, 2026
…se the two #1223 side fixes

Hardens the retry #1246 adds: it restarted the whole stream on the
temperature 400 because that error arrives before the first chunk. True, but
that is a property of the provider, not of this code — the retry now re-raises
once a chunk has been emitted, so a mid-stream error matching the detector can
never replay already-emitted output.

Also lands the two fixes promised on #1223 and not covered by that PR:

- ANTHROPIC_API_KEY joins env_schema. It was read through a bare os.getenv, so
  provider doctor and preflight could not tell a missing key from a working
  one and it surfaced as an opaque 401 at the first call.
- .env.example states that pointing OPENAI_BASE_URL at Anthropic's
  compatibility endpoint is not a supported route.
@warren618

Copy link
Copy Markdown
Collaborator

Merged, and I was wrong to decline this direction on #1223.

My argument there was that the generic OpenAI-compatible branch is "deliberately provider-agnostic" and teaching it Anthropic-specific error prose would couple it to every provider quirk it currently stays clean of. That is not true of the branch as it stands: _STREAM_USAGE_UNSUPPORTED in the same file is already a classify-the-error, retry-once, remember-the-model self-heal for a provider quirk, inside ChatOpenAIWithReasoning, landed for #1224. Same class, same file, same branch. You said as much in your checklist and you were right. Sorry for the round trip.

One hardening pushed on top (3e27d25a). The stream retry restarted the whole stream on the 400, with a comment that the error arrives before the first chunk so output cannot duplicate. That is true, but it is a property of the provider, not of this code — the retry now re-raises once a chunk has been emitted, so a mid-stream error matching the detector can never replay already-emitted output.

The same commit lands the two things I promised on #1223 and had not done: ANTHROPIC_API_KEY joins env_schema (it was read through a bare os.getenv, so provider doctor and preflight could not tell a missing key from a working one), and .env.example now states that pointing OPENAI_BASE_URL at Anthropic's compatibility endpoint is not a supported route.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

temperature sent unconditionally on the OpenAI-compatible path to Anthropic (native-path fix at v0.1.14 doesn't cover it)

2 participants