feat: add prompt caching schema support for system/developer messages#230
feat: add prompt caching schema support for system/developer messages#230dcox761 wants to merge 1 commit intoaws-samples:mainfrom
Conversation
Add CacheControl model and cache_control field on TextContent to support
OpenAI-compatible prompt caching hints (e.g., cache_control: {type: 'ephemeral'}).
Changes:
- schema.py: Add CacheControl model; add cache_control field to TextContent;
change SystemMessage.content and DeveloperMessage.content from str to
str | list[TextContent] to accept structured content with cache hints
- bedrock.py: Update _parse_system_prompts to handle list-format content
with cache_control, emitting Bedrock cachePoint blocks; update
_parse_content_parts to emit cachePoint for user message content parts
This allows clients to mark specific content blocks for caching using the
standard OpenAI cache_control field, which is translated to Bedrock's
cachePoint tagged union format.
Ref: https://docs.aws.amazon.com/bedrock/latest/userguide/prompt-caching.html
|
Thanks for the PR! The I had a question about the The Could you share more about the use case you have in mind? For example:
Understanding the real-world scenario would help us evaluate the best approach. If there is a valid use case, we'd want to make sure the Again, the |
*Issue #229 *
Description of changes:
Add CacheControl model and cache_control field on TextContent to support OpenAI-compatible prompt caching hints (e.g., cache_control: {type: 'ephemeral'}).
Changes:
This allows clients to mark specific content blocks for caching using the standard OpenAI cache_control field, which is translated to Bedrock's cachePoint tagged union format.
Ref: https://docs.aws.amazon.com/bedrock/latest/userguide/prompt-caching.html
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.