Skip to content

feat(bedrock): add prompt cache TTL support#4581

Open
imdigitalashish wants to merge 4 commits intopydantic:mainfrom
imdigitalashish:feat/bedrock-cache-ttl-4096
Open

feat(bedrock): add prompt cache TTL support#4581
imdigitalashish wants to merge 4 commits intopydantic:mainfrom
imdigitalashish:feat/bedrock-cache-ttl-4096

Conversation

@imdigitalashish
Copy link
Copy Markdown

@imdigitalashish imdigitalashish commented Mar 7, 2026

Add configurable TTL ('5m' or '1h') for Bedrock prompt caching. Previously all cache settings were boolean-only. Now bedrock_cache_instructions, bedrock_cache_tool_definitions, and bedrock_cache_messages accept bool | Literal['5m', '1h'], and CachePoint(ttl=...) is forwarded to the Bedrock API.

Closes #4096

  • Closes #

Pre-Review Checklist

  • Any AI generated code has been reviewed line-by-line by the human PR author, who stands by it.
  • No breaking changes in accordance with the version policy.
  • Linting and type checking pass per make format and make typecheck.
  • PR title is fit for the release changelog.

Pre-Merge Checklist

  • New tests for any fix or new behavior, maintaining 100% coverage.
  • Updated documentation for new features and behaviors, including docstrings for API docs.

Summary by cubic

Adds TTL support to Amazon Bedrock prompt caching so you can set '5m' or '1h' instead of only on/off. TTL applies to system instructions, tool definitions, the last user message, and manual CachePoint. Closes #4096.

  • New Features

    • bedrock_cache_instructions, bedrock_cache_tool_definitions, and bedrock_cache_messages accept bool | '5m' | '1h'; TTL is sent only when a string is provided, otherwise omitted. CachePoint(ttl='5m'|'1h') is forwarded to Bedrock. Updated CachePoint.ttl docs to include Bedrock, clarify TTL omission for Anthropic via Bedrock, and add the missing docstring for the cache_point parameter.
  • Dependencies

    • Upgrade boto3 and boto3-stubs to 1.42.63.

Written for commit 89018eb. Summary will update on new commits.

@github-actions github-actions bot added size: M Medium PR (101-500 weighted lines) feature New feature request, or PR implementing a feature (enhancement) labels Mar 7, 2026
devin-ai-integration[bot]

This comment was marked as resolved.

Add configurable TTL ('5m' or '1h') for Bedrock prompt caching.
Previously all cache settings were boolean-only. Now
`bedrock_cache_instructions`, `bedrock_cache_tool_definitions`, and
`bedrock_cache_messages` accept `bool | Literal['5m', '1h']`, and
`CachePoint(ttl=...)` is forwarded to the Bedrock API.

Closes pydantic#4096
@imdigitalashish imdigitalashish force-pushed the feat/bedrock-cache-ttl-4096 branch from 71753fd to 876615d Compare March 7, 2026 08:41
Copy link
Copy Markdown
Contributor

@devin-ai-integration devin-ai-integration bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 1 new potential issue.

View 4 additional findings in Devin Review.

Open in Devin Review

@rand-davis-ssc
Copy link
Copy Markdown

I'd love to see this merged - this would directly help reduce costs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

feature New feature request, or PR implementing a feature (enhancement) size: M Medium PR (101-500 weighted lines)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support configuration around the prompt cache TTL inBedrockModelSettings

2 participants