✨ feat(anthropic): add prompt caching model validation utilities #20069

zhaoxinn-chang · 2025-10-13T04:44:10Z

Add Anthropic Prompt Caching Model Validation Function

Description

This PR adds a utility function is_anthropic_prompt_caching_supported_model() to the Anthropic LLM integration, following the same pattern as the existing is_bedrock_prompt_caching_supported_model() function in the Bedrock Converse integration.

Motivation:

Provides a standardized way to validate which Claude models support prompt caching
Improves developer experience by centralizing model support information
Helps developers correctly enable prompt caching to reduce costs by up to 90% and latency by up to 85%
Maintains consistency across LlamaIndex integrations

Implementation:

Added ANTHROPIC_PROMPT_CACHING_SUPPORTED_MODELS tuple constant containing all supported models
Added is_anthropic_prompt_caching_supported_model() function with comprehensive docstring
Added 14 unit tests covering all supported/unsupported models and edge cases
Follows the exact pattern established in llama-index-llms-bedrock-converse

Supported Models:

Claude 4 family (Opus 4, Opus 4.1, Sonnet 4, Sonnet 4.5)
Claude 3.7 (Sonnet 3.7)
Claude 3.5 (Sonnet 3.5, Haiku 3.5)
Claude 3 (Haiku 3, Opus 3)

Reference: https://docs.claude.com/en/docs/build-with-claude/prompt-caching

Fixes # (N/A - Enhancement)

New Package?

Yes
No

Version Bump?

Yes - Bumped from 0.9.3 to 0.9.4
No

Type of Change

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to not work as expected)
This change requires a documentation update

How Has This Been Tested?

This change is covered by comprehensive unit testing.

I added new unit tests to cover this change
I believe this change is already covered by existing unit tests

Test Details:

Created tests/test_anthropic_utils.py with 14 test cases
Tests cover:
- All supported Claude model families (4.x, 3.7, 3.5, 3.x)
- Unsupported models (Claude 2.x, Instant)
- Invalid/unknown model names
- Case sensitivity
- Constant validation
All tests pass: pytest tests/test_anthropic_utils.py -v ✅

Test Output:

============================= test session starts ==============================
collected 14 items

tests/test_anthropic_utils.py::TestAnthropicPromptCachingSupport::test_claude_4_1_opus_supported PASSED
tests/test_anthropic_utils.py::TestAnthropicPromptCachingSupport::test_claude_4_opus_supported PASSED
tests/test_anthropic_utils.py::TestAnthropicPromptCachingSupport::test_claude_4_5_sonnet_supported PASSED
tests/test_anthropic_utils.py::TestAnthropicPromptCachingSupport::test_claude_4_sonnet_supported PASSED
tests/test_anthropic_utils.py::TestAnthropicPromptCachingSupport::test_claude_3_7_sonnet_supported PASSED
tests/test_anthropic_utils.py::TestAnthropicPromptCachingSupport::test_claude_3_5_sonnet_supported PASSED
tests/test_anthropic_utils.py::TestAnthropicPromptCachingSupport::test_claude_3_5_haiku_supported PASSED
tests/test_anthropic_utils.py::TestAnthropicPromptCachingSupport::test_claude_3_haiku_supported PASSED
tests/test_anthropic_utils.py::TestAnthropicPromptCachingSupport::test_claude_3_opus_deprecated_but_supported PASSED
tests/test_anthropic_utils.py::TestAnthropicPromptCachingSupport::test_claude_2_not_supported PASSED
tests/test_anthropic_utils.py::TestAnthropicPromptCachingSupport::test_claude_instant_not_supported PASSED
tests/test_anthropic_utils.py::TestAnthropicPromptCachingSupport::test_invalid_model_not_supported PASSED
tests/test_anthropic_utils.py::TestAnthropicPromptCachingSupport::test_constant_contains_all_supported_models PASSED
tests/test_anthropic_utils.py::TestAnthropicPromptCachingSupport::test_case_sensitivity PASSED

============================== 14 passed ==============================

Suggested Checklist:

I have performed a self-review of my own code
I have commented my code, particularly in hard-to-understand areas
I have made corresponding changes to the documentation
I have added Google Colab support for the newly added notebooks. (N/A - no notebooks)
My changes generate no new warnings
I have added tests that prove my fix is effective or that my feature works
New and existing unit tests pass locally with my changes
I ran uv run make format; uv run make lint to appease the lint gods (will run before final submission)

Files Changed

llama_index/llms/anthropic/utils.py
- Added ANTHROPIC_PROMPT_CACHING_SUPPORTED_MODELS constant (32 model identifiers)
- Added is_anthropic_prompt_caching_supported_model() function
- Added comprehensive docstring with examples
tests/test_anthropic_utils.py (NEW)
- Created comprehensive test suite
- 14 test cases covering all scenarios
- Follows pytest best practices
pyproject.toml
- Bumped version from 0.9.3 to 0.9.4

Example Usage

from llama_index.llms.anthropic.utils import is_anthropic_prompt_caching_supported_model

# Validate model support before enabling caching
model = "claude-sonnet-4-20250514"
if is_anthropic_prompt_caching_supported_model(model):
    # Enable prompt caching for supported models
    llm = Anthropic(model=model, cache_idx=-1)
    print(f"✅ Prompt caching enabled for {model}")
else:
    # Use without caching for unsupported models
    llm = Anthropic(model=model)
    print(f"⚠️ Prompt caching not supported for {model}")

Consistency with Existing Code

This implementation follows the exact pattern used in llama-index-llms-bedrock-converse:

Function naming convention: is_{provider}_prompt_caching_supported_model()
Constant naming: {PROVIDER}_PROMPT_CACHING_SUPPORTED_MODELS
Return type: Simple boolean check
Documentation style: Comprehensive docstring with examples

Additional Notes

We're happy to make any adjustments based on maintainer feedback. Thank you for considering this contribution to improve the LlamaIndex ecosystem!

Add validation utility function for Anthropic prompt caching support, following the same pattern as Bedrock Converse integration. Changes: - Add ANTHROPIC_PROMPT_CACHING_SUPPORTED_MODELS constant with 32 model identifiers - Add is_anthropic_prompt_caching_supported_model() function with comprehensive docstring - Add 14 unit tests covering all supported/unsupported models and edge cases - Bump version from 0.9.3 to 0.9.4 - Apply ruff linter fixes Benefits: - Provides standardized model validation for prompt caching - Helps developers reduce costs by up to 90% and latency by up to 85% - Maintains consistency across LlamaIndex integrations - Single source of truth for model support information All tests pass successfully (14/14).

AstraBert

This PR might be useful, but we should probably add this check for prompt caching support in the main code for the LLM :)

AstraBert · 2025-10-13T11:12:33Z

llama-index-integrations/llms/llama-index-llms-anthropic/llama_index/llms/anthropic/utils.py

+def is_anthropic_prompt_caching_supported_model(model: str) -> bool:
+    """
+    Check if the given Anthropic model supports prompt caching.
+
+    Prompt caching allows you to cache frequently used context (like system prompts,
+    long documents, or tool definitions) to reduce costs by up to 90% and latency
+    by up to 85% for supported models.
+
+    For more information, see:
+    https://docs.claude.com/en/docs/build-with-claude/prompt-caching
+
+    Args:
+        model: The Anthropic model identifier (e.g., 'claude-sonnet-4-20250514')
+
+    Returns:
+        bool: True if the model supports prompt caching, False otherwise
+
+    Examples:
+        >>> is_anthropic_prompt_caching_supported_model('claude-sonnet-4-20250514')
+        True
+        >>> is_anthropic_prompt_caching_supported_model('claude-2.1')
+        False
+        >>> is_anthropic_prompt_caching_supported_model('claude-3-5-haiku-20241022')
+        True
+
+    """


As much as I appreciate detailed docstrings, I think we can make this a little less detailed :))

dosubot bot added the size:L This PR changes 100-499 lines, ignoring generated files. label Oct 13, 2025

zhaoxinn-chang force-pushed the main branch from db22fd8 to e553d7d Compare October 13, 2025 04:56

AstraBert requested changes Oct 13, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

✨ feat(anthropic): add prompt caching model validation utilities #20069

✨ feat(anthropic): add prompt caching model validation utilities #20069

zhaoxinn-chang commented Oct 13, 2025 •

edited

Loading

Uh oh!

AstraBert left a comment

Uh oh!

AstraBert Oct 13, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

✨ feat(anthropic): add prompt caching model validation utilities #20069

Are you sure you want to change the base?

✨ feat(anthropic): add prompt caching model validation utilities #20069

Conversation

zhaoxinn-chang commented Oct 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Add Anthropic Prompt Caching Model Validation Function

Description

New Package?

Version Bump?

Type of Change

How Has This Been Tested?

Suggested Checklist:

Files Changed

Example Usage

Consistency with Existing Code

Additional Notes

Uh oh!

AstraBert left a comment

Choose a reason for hiding this comment

Uh oh!

AstraBert Oct 13, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

zhaoxinn-chang commented Oct 13, 2025 •

edited

Loading