Skip to content

fix(langchain): sanitize anthropic cache markers on fallback retries#37867

Draft
Hunter Lovell (hntrl) wants to merge 1 commit into
masterfrom
hunter/fix-model-fallback-cache-control-leak
Draft

fix(langchain): sanitize anthropic cache markers on fallback retries#37867
Hunter Lovell (hntrl) wants to merge 1 commit into
masterfrom
hunter/fix-model-fallback-cache-control-leak

Conversation

@hntrl
Copy link
Copy Markdown
Member

Summary

Fixes #33709.

When AnthropicPromptCachingMiddleware runs before ModelFallbackMiddleware, Anthropic-specific cache_control markers can leak into fallback attempts targeting non-Anthropic models. This change makes fallback retries sanitize those markers only on non-primary attempts, preserving primary-call behavior and avoiding API changes.

Related: langchain-ai/deepagentsjs#551.

Changes

libs/langchain_v1

  • Added a private fallback sanitizer in model_fallback.py to remove Anthropic cache_control markers from fallback requests.
  • Sanitization covers model_settings, system/content message blocks, and tool payloads (BaseTool.extras plus dict-style tools/extras).
  • Applied sanitization in both sync and async fallback retry paths (wrap_model_call and awrap_model_call) while leaving the primary attempt unchanged.

libs/langchain_v1 tests

  • Added sync and async regression tests in test_model_fallback.py that simulate primary failure and assert fallback calls only succeed when cache markers are stripped.
  • Verified non-cache settings (for example temperature and top_p) are preserved on fallback.
  • Preserved existing fallback behavior coverage (ordering, exhaustion, and error propagation).

@github-actions github-actions Bot added fix For PRs that implement a fix internal langchain `langchain` package issues & PRs size: M 200-499 LOC labels Jun 2, 2026
@hntrl Hunter Lovell (hntrl) marked this pull request as draft June 2, 2026 22:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

fix For PRs that implement a fix internal langchain `langchain` package issues & PRs size: M 200-499 LOC

Projects

None yet

Development

Successfully merging this pull request may close these issues.

AnthropicPromptCachingMiddleware breaks model fallback with cache_control param

1 participant