From 12cb52c3352130354180c323b217f79d9d522dda Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" Date: Thu, 2 Apr 2026 16:28:41 +0000 Subject: [PATCH] docs: add enable_prompt_caching to AWS Bedrock LLM settings Updates documentation for pipecat PR #4219 which added prompt caching support to AWSBedrockLLMService. Added the new enable_prompt_caching parameter to the Settings table with description and AWS docs link. --- api-reference/server/services/llm/aws.mdx | 25 ++++++++++++----------- 1 file changed, 13 insertions(+), 12 deletions(-) diff --git a/api-reference/server/services/llm/aws.mdx b/api-reference/server/services/llm/aws.mdx index b7af10037..46152be77 100644 --- a/api-reference/server/services/llm/aws.mdx +++ b/api-reference/server/services/llm/aws.mdx @@ -128,18 +128,19 @@ Before using AWS Bedrock LLM services, you need: Runtime-configurable settings passed via the `settings` constructor argument using `AWSBedrockLLMService.Settings(...)`. These can be updated mid-conversation with `LLMUpdateSettingsFrame`. See [Service Settings](/pipecat/fundamentals/service-settings) for details. -| Parameter | Type | Default | Description | -| --------------------------------- | ----------- | ----------- | -------------------------------------------------------------------------------------------------- | -| `model` | `str` | `None` | AWS Bedrock model identifier. _(Inherited from base settings.)_ | -| `system_instruction` | `str` | `None` | System instruction/prompt for the model. _(Inherited from base settings.)_ | -| `max_tokens` | `int` | `NOT_GIVEN` | Maximum number of tokens to generate. | -| `temperature` | `float` | `NOT_GIVEN` | Sampling temperature (0.0 to 1.0). Lower values are more focused, higher values are more creative. | -| `top_p` | `float` | `NOT_GIVEN` | Top-p (nucleus) sampling (0.0 to 1.0). Controls diversity of output. | -| `top_k` | `int` | `NOT_GIVEN` | Top-k sampling parameter. | -| `seed` | `int` | `NOT_GIVEN` | Random seed for deterministic outputs. | -| `stop_sequences` | `List[str]` | `NOT_GIVEN` | List of strings that stop generation when encountered. | -| `latency` | `str` | `NOT_GIVEN` | Performance mode: `"standard"` or `"optimized"`. | -| `additional_model_request_fields` | `dict` | `NOT_GIVEN` | Additional model-specific parameters passed directly to the API. | +| Parameter | Type | Default | Description | +| --------------------------------- | ----------- | ----------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `model` | `str` | `None` | AWS Bedrock model identifier. _(Inherited from base settings.)_ | +| `system_instruction` | `str` | `None` | System instruction/prompt for the model. _(Inherited from base settings.)_ | +| `max_tokens` | `int` | `NOT_GIVEN` | Maximum number of tokens to generate. | +| `temperature` | `float` | `NOT_GIVEN` | Sampling temperature (0.0 to 1.0). Lower values are more focused, higher values are more creative. | +| `top_p` | `float` | `NOT_GIVEN` | Top-p (nucleus) sampling (0.0 to 1.0). Controls diversity of output. | +| `top_k` | `int` | `NOT_GIVEN` | Top-k sampling parameter. | +| `seed` | `int` | `NOT_GIVEN` | Random seed for deterministic outputs. | +| `stop_sequences` | `List[str]` | `NOT_GIVEN` | List of strings that stop generation when encountered. | +| `latency` | `str` | `NOT_GIVEN` | Performance mode: `"standard"` or `"optimized"`. | +| `enable_prompt_caching` | `bool` | `NOT_GIVEN` | Whether to enable prompt caching by adding cachePoint markers to system prompts and tool definitions. Can reduce TTFT by up to 85% for multi-turn conversations. See [AWS docs](https://docs.aws.amazon.com/bedrock/latest/userguide/prompt-caching.html). | +| `additional_model_request_fields` | `dict` | `NOT_GIVEN` | Additional model-specific parameters passed directly to the API. | `NOT_GIVEN` values are omitted from the inference config, letting the Bedrock