Feature Request: Bedrock-Compatible PromptCaching Middleware

[Oscar-Umana]

Checked other resources

• This is a bug, not a usage question. For questions, please use the LangChain Forum (https://forum.langchain.com/).
• I added a very descriptive title to this issue.
• I searched the LangChain.js documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.
• I am sure that this is a bug in LangChain.js rather than my code.
• The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).

Example Code

Sample usage:

import { ChatBedrockConverse } from '@langchain/aws';
import { createAgent, bedrockPromptCachingMiddleware } from 'langchain';

// Example usage:
const model = new ChatBedrockConverse({
  model: 'anthropic.claude-3-5-sonnet-20241022-v2:0',
  region: 'us-east-1'
});

const agent = createAgent({
  model,
  middleware: [
    bedrockPromptCachingMiddleware({
      enableCaching: true,
      minMessagesToCache: 10
    })
  ],
});

// Invoke the agent
const result = await agent.invoke({
  messages: [{ role: 'user', content: 'Hello!' }]
});

Error Message and Stack Trace (if applicable)

No response

Description

Summary:
Requesting support for Bedrock models in LangChain's promptCaching middleware, similar to the implementation for Anthropic found in libs/langchain/src/agents/middleware/provider/anthropic/promptCaching.ts.

Details:

• Ensure cache control logic and API structure are compatible with Bedrock endpoints and formats.
• Matching feature set to Anthropic:
  • Configurable minMessagesToCache
  • Runtime and middleware options for enableCaching
  • Behavior control for unsupported models
• Consider Bedrock-specific API behaviors and cache headers.

Benefits:

• Cost and latency reduction for repetitive prompts, long system messages, or extensive conversation histories on Bedrock.
• Enables prompt caching best practices across more providers.

References:

• Existing Anthropic middleware: promptCaching.ts
• Anthropic prompt caching docs

Suggested implementation notes:

• Structure middleware API and error handling similar to Anthropic version.
• Document differences and caveats between Anthropic and Bedrock behavior.
• Add examples and usage guidance for Bedrock prompt caching in documentation.

System Info

N/A