Skip to content

Commit 156b484

Browse files
github-actions[bot]github-actions[bot]
committed
docs: add enable_prompt_caching to AWS Bedrock LLM settings
Updates documentation for pipecat PR #4219 which added prompt caching support to AWSBedrockLLMService. Added the new enable_prompt_caching parameter to the Settings table with description and AWS docs link.
1 parent e92c38d commit 156b484

1 file changed

Lines changed: 13 additions & 12 deletions

File tree

‎server/services/llm/aws.mdx‎

Lines changed: 13 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -128,18 +128,19 @@ Before using AWS Bedrock LLM services, you need:
128128

129129
Runtime-configurable settings passed via the `settings` constructor argument using `AWSBedrockLLMService.Settings(...)`. These can be updated mid-conversation with `LLMUpdateSettingsFrame`. See [Service Settings](/guides/fundamentals/service-settings) for details.
130130

131-
| Parameter | Type | Default | Description |
132-
| --------------------------------- | ----------- | ----------- | -------------------------------------------------------------------------------------------------- |
133-
| `model` | `str` | `None` | AWS Bedrock model identifier. _(Inherited from base settings.)_ |
134-
| `system_instruction` | `str` | `None` | System instruction/prompt for the model. _(Inherited from base settings.)_ |
135-
| `max_tokens` | `int` | `NOT_GIVEN` | Maximum number of tokens to generate. |
136-
| `temperature` | `float` | `NOT_GIVEN` | Sampling temperature (0.0 to 1.0). Lower values are more focused, higher values are more creative. |
137-
| `top_p` | `float` | `NOT_GIVEN` | Top-p (nucleus) sampling (0.0 to 1.0). Controls diversity of output. |
138-
| `top_k` | `int` | `NOT_GIVEN` | Top-k sampling parameter. |
139-
| `seed` | `int` | `NOT_GIVEN` | Random seed for deterministic outputs. |
140-
| `stop_sequences` | `List[str]` | `NOT_GIVEN` | List of strings that stop generation when encountered. |
141-
| `latency` | `str` | `NOT_GIVEN` | Performance mode: `"standard"` or `"optimized"`. |
142-
| `additional_model_request_fields` | `dict` | `NOT_GIVEN` | Additional model-specific parameters passed directly to the API. |
131+
| Parameter | Type | Default | Description |
132+
| --------------------------------- | ----------- | ----------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
133+
| `model` | `str` | `None` | AWS Bedrock model identifier. _(Inherited from base settings.)_ |
134+
| `system_instruction` | `str` | `None` | System instruction/prompt for the model. _(Inherited from base settings.)_ |
135+
| `max_tokens` | `int` | `NOT_GIVEN` | Maximum number of tokens to generate. |
136+
| `temperature` | `float` | `NOT_GIVEN` | Sampling temperature (0.0 to 1.0). Lower values are more focused, higher values are more creative. |
137+
| `top_p` | `float` | `NOT_GIVEN` | Top-p (nucleus) sampling (0.0 to 1.0). Controls diversity of output. |
138+
| `top_k` | `int` | `NOT_GIVEN` | Top-k sampling parameter. |
139+
| `seed` | `int` | `NOT_GIVEN` | Random seed for deterministic outputs. |
140+
| `stop_sequences` | `List[str]` | `NOT_GIVEN` | List of strings that stop generation when encountered. |
141+
| `latency` | `str` | `NOT_GIVEN` | Performance mode: `"standard"` or `"optimized"`. |
142+
| `enable_prompt_caching` | `bool` | `NOT_GIVEN` | Whether to enable prompt caching by adding cachePoint markers to system prompts and tool definitions. Can reduce TTFT by up to 85% for multi-turn conversations. See [AWS docs](https://docs.aws.amazon.com/bedrock/latest/userguide/prompt-caching.html). |
143+
| `additional_model_request_fields` | `dict` | `NOT_GIVEN` | Additional model-specific parameters passed directly to the API. |
143144

144145
<Note>
145146
`NOT_GIVEN` values are omitted from the inference config, letting the Bedrock

0 commit comments

Comments
 (0)