Skip to content

Commit 0db501e

Browse files
Jeadielukekim
andauthored
Improve docs for available Bedrock models (#1462)
* update support bedrock LLMs * fix * add to v1.11 --------- Co-authored-by: Luke Kim <80174+lukekim@users.noreply.github.com>
1 parent e15ef4e commit 0db501e

2 files changed

Lines changed: 61 additions & 16 deletions

File tree

website/docs/components/models/bedrock.md

Lines changed: 30 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -9,16 +9,21 @@ Spice supports large language models hosted on [Amazon Bedrock](https://aws.amaz
99

1010
## Supported Models
1111

12-
Spice supports the following Amazon Nova models:
12+
Spice supports both Amazon's Nova models and models from other providers that are available on AWS bedrock.
1313

14-
| Model ID | Description |
15-
| -------------------------- | ---------------------------------------------- |
16-
| `amazon.nova-micro-v1:0` | Text-only, lowest latency responses |
17-
| `amazon.nova-lite-v1:0` | Multimodal, low-cost with fast processing |
18-
| `amazon.nova-pro-v1:0` | Multimodal, balanced accuracy, speed, and cost |
19-
| `amazon.nova-premier-v1:0` | Multimodal, best for complex tasks |
14+
Providers include:
2015

21-
Cross-region inference profiles (e.g., `us.amazon.nova-lite-v1:0`) are also supported. See the [Amazon Bedrock model IDs documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/model-ids.html) for details.
16+
| Family | Example model IDs |
17+
| ------ | ----------------- |
18+
| Amazon Nova | `amazon.nova-micro-v1:0`, `amazon.nova-lite-v1:0`, `amazon.nova-pro-v1:0`, `amazon.nova-premier-v1:0` |
19+
| Anthropic Claude | `anthropic.claude-3-5-haiku-20241022-v1:0`, `anthropic.claude-sonnet-4-20250514-v1:0` |
20+
| Meta Llama | `meta.llama3-1-70b-instruct-v1:0`, `meta.llama3-2-90b-instruct-v1:0` |
21+
| Mistral | `mistral.mixtral-8x7b-instruct-v0:1`, `mistral.mistral-large-2407-v1:0` |
22+
| Cohere Command | `cohere.command-r-v1:0`, `cohere.command-r-plus-v1:0` |
23+
| AI21 Jamba | `ai21.jamba-1-5-mini-v1:0`, `ai21.jamba-1-5-large-v1:0` |
24+
| DeepSeek | `deepseek.r1-v1:0`, `deepseek.v3.2` |
25+
26+
Cross-region inference profiles (for example, `us.amazon.nova-lite-v1:0` or `us.meta.llama3-1-70b-instruct-v1:0`) are supported. See the [Amazon Bedrock model IDs documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/model-ids.html) for the latest IDs and availability by region.
2227

2328
To request support for additional models, file a [GitHub Issue](https://github.com/spiceai/spiceai/issues).
2429

@@ -101,6 +106,23 @@ models:
101106
aws_region: us-east-1
102107
```
103108

109+
### Inference Profile for Models Without On-Demand Throughput
110+
111+
Some models (for example, several Anthropic/Meta variants) require inference profile IDs:
112+
113+
```yaml
114+
models:
115+
- from: bedrock:us.meta.llama3-1-70b-instruct-v1:0
116+
name: llama31
117+
params:
118+
aws_region: us-east-1
119+
120+
- from: bedrock:us.anthropic.claude-opus-4-6-v1
121+
name: claude-opus-46
122+
params:
123+
aws_region: us-east-1
124+
```
125+
104126
### With Guardrails
105127

106128
```yaml

website/versioned_docs/version-1.11.x/components/models/bedrock.md

Lines changed: 31 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -9,16 +9,21 @@ Spice supports large language models hosted on [Amazon Bedrock](https://aws.amaz
99

1010
## Supported Models
1111

12-
Spice supports the following Amazon Nova models:
12+
Spice supports both Amazon's Nova models and models from other providers that are available on AWS bedrock.
1313

14-
| Model ID | Description |
15-
| -------------------------- | ---------------------------------------------- |
16-
| `amazon.nova-micro-v1:0` | Text-only, lowest latency responses |
17-
| `amazon.nova-lite-v1:0` | Multimodal, low-cost with fast processing |
18-
| `amazon.nova-pro-v1:0` | Multimodal, balanced accuracy, speed, and cost |
19-
| `amazon.nova-premier-v1:0` | Multimodal, best for complex tasks |
14+
Providers include:
2015

21-
Cross-region inference profiles (e.g., `us.amazon.nova-lite-v1:0`) are also supported. See the [Amazon Bedrock model IDs documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/model-ids.html) for details.
16+
| Family | Example model IDs |
17+
| ------ | ----------------- |
18+
| Amazon Nova | `amazon.nova-micro-v1:0`, `amazon.nova-lite-v1:0`, `amazon.nova-pro-v1:0`, `amazon.nova-premier-v1:0` |
19+
| Anthropic Claude | `anthropic.claude-3-5-haiku-20241022-v1:0`, `anthropic.claude-sonnet-4-20250514-v1:0` |
20+
| Meta Llama | `meta.llama3-1-70b-instruct-v1:0`, `meta.llama3-2-90b-instruct-v1:0` |
21+
| Mistral | `mistral.mixtral-8x7b-instruct-v0:1`, `mistral.mistral-large-2407-v1:0` |
22+
| Cohere Command | `cohere.command-r-v1:0`, `cohere.command-r-plus-v1:0` |
23+
| AI21 Jamba | `ai21.jamba-1-5-mini-v1:0`, `ai21.jamba-1-5-large-v1:0` |
24+
| DeepSeek | `deepseek.r1-v1:0`, `deepseek.v3.2` |
25+
26+
Cross-region inference profiles (for example, `us.amazon.nova-lite-v1:0` or `us.meta.llama3-1-70b-instruct-v1:0`) are supported. See the [Amazon Bedrock model IDs documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/model-ids.html) for the latest IDs and availability by region.
2227

2328
To request support for additional models, file a [GitHub Issue](https://github.com/spiceai/spiceai/issues).
2429

@@ -49,6 +54,7 @@ models:
4954
| `aws_access_key_id` | AWS access key ID. If not provided, credentials load from environment variables or IAM roles. | - |
5055
| `aws_secret_access_key` | AWS secret access key. If not provided, credentials load from environment variables or IAM roles. | - |
5156
| `aws_session_token` | AWS session token for temporary credentials. | - |
57+
| `aws_iam_role_source` | IAM role credential source. `auto` uses the default AWS credential chain, `metadata` uses only instance/container metadata (IMDS, ECS, EKS/IRSA), `env` uses only environment variables. | `auto` |
5258

5359
#### Guardrails
5460

@@ -100,6 +106,23 @@ models:
100106
aws_region: us-east-1
101107
```
102108

109+
### Inference Profile for Models Without On-Demand Throughput
110+
111+
Some models (for example, several Anthropic/Meta variants) require inference profile IDs:
112+
113+
```yaml
114+
models:
115+
- from: bedrock:us.meta.llama3-1-70b-instruct-v1:0
116+
name: llama31
117+
params:
118+
aws_region: us-east-1
119+
120+
- from: bedrock:us.anthropic.claude-opus-4-6-v1
121+
name: claude-opus-46
122+
params:
123+
aws_region: us-east-1
124+
```
125+
103126
### With Guardrails
104127

105128
```yaml

0 commit comments

Comments
 (0)