Skip to content

Update Databricks models and embeddings docs with m2m auth #989

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
peasee merged 8 commits into from
May 13, 2025
Merged

Update Databricks models and embeddings docs with m2m auth #989

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
052b91e
Update Databricks models and embeddings docs with m2m auth
ewgenius May 11, 2025
73b80a8
Fix spicepod sample in openai embeddings docs
ewgenius May 11, 2025
5aafd36
Update website/docs/components/embeddings/databricks.md
ewgenius May 12, 2025
4493ba3
Update website/docs/components/embeddings/databricks.md
ewgenius May 12, 2025
a55405e
fix letter casing
ewgenius May 12, 2025
42e3865
Update website/docs/components/embeddings/databricks.md
ewgenius May 12, 2025
bed7547
Update website/docs/components/models/databricks.md
ewgenius May 12, 2025
5728411
Merge branch 'trunk' into evgenii/databricks-models-m2m
lukekim May 13, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
52 changes: 52 additions & 0 deletions website/docs/components/embeddings/databricks.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
---
title: 'Databricks Model Provider'
description: 'Instructions for using Databricks Mosaic AI Models'
sidebar_label: 'Databricks'
sidebar_position: 8
---

To use an embedding model deployed to [Databricks Mosaic AI Model Serving](https://docs.databricks.com/aws/en/machine-learning/model-serving/), specify the model endpoint name prefixed with `databricks:` in the `from` field and include the required parameters in the `params` section.

### Parameters

| Parameter | Description |
| -------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `databricks_endpoint` | The Databricks workspace endpoint, e.g., `dbc-a12cd3e4-56f7.cloud.databricks.com`. |
| `databricks_token` | The Databricks API token to authenticate with the Databricks Models API. Use the [secret replacement syntax](../secret-stores/index.md) to reference a secret, e.g., `${secrets:my_databricks_token}`. |
| `databricks_client_id` | The Databricks Service Principal Client ID. Can't be used with `databricks_token`. |
| `databricks_client_secret` | The Databricks Service Principal Client Secret. Can't be used with `databricks_token`. |

### Example `spicepod.yaml` configuration, using personal access token

To Learn more about how to set up personal access tokens, see [Databricks PAT docs](https://docs.databricks.com/aws/en/dev-tools/auth/pat).
Comment thread
ewgenius marked this conversation as resolved.
Outdated

```yaml
embeddings:
- from: databricks:databricks-gte-large-en
name: gte-large-en
params:
databricks_endpoint: dbc-46470731-42e5.cloud.databricks.com
databricks_token: ${ secrets:SPICE_DATABRICKS_TOKEN }
```

### Example `spicepod.yaml` configuration, using Databricks service principal

Spice supports the M2M OAuth flow with service principal credentials by utilizing the `databricks_client_id` and `databricks_client_secret` parameters. The runtime will automatically refresh the token.

Ensure that you grant your service principal the "Can Query" permission for model serving.
Comment thread
ewgenius marked this conversation as resolved.
Outdated

To Learn more about how to set up the service principal, see [Databricks M2M OAuth docs](https://docs.databricks.com/aws/en/dev-tools/auth/oauth-m2m).

```yaml
embeddings:
- from: databricks:databricks-gte-large-en
name: gte-large-en
params:
databricks_endpoint: dbc-46470731-42e5.cloud.databricks.com
Comment thread
ewgenius marked this conversation as resolved.
Outdated
databricks_client_id: ${secrets:DATABRICKS_CLIENT_ID}
databricks_client_secret: ${secrets:DATABRICKS_CLIENT_SECRET}
```

### Additional Information

Refer to the [Mosaic AI Model Serving documentation](https://docs.databricks.com/aws/en/machine-learning/model-serving/) for more details on available models and configurations.
14 changes: 8 additions & 6 deletions website/docs/components/embeddings/index.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,17 +19,19 @@ Embeddings are used for vector-based and similarity search, like document retrie

Spice supports various model sources and formats to provide embedding components:

| Name | Description | Status | ML Format(s) | LLM Format(s)\* |
| ------------------- | -------------------------------------------- | ----------------- | ------------ | ------------------------------- |
| [`file`][file] | Local filesystem | Release Candidate | ONNX | GGUF, GGML, SafeTensor |
| [`huggingface`][hf] | Models hosted on HuggingFace | Release Candidate | ONNX | GGUF, GGML, SafeTensor |
| [`openai`][openai] | OpenAI (or compatible) LLM endpoint | Release Candidate | - | OpenAI-compatible HTTP endpoint |
| [`azure`][azure] | Azure OpenAI | Alpha | - | OpenAI-compatible HTTP endpoint |
| Name | Description | Status | ML Format(s) | LLM Format(s)\* |
| -------------------------- | -------------------------------------------- | ----------------- | ------------ | ------------------------------- |
| [`file`][file] | Local filesystem | Release Candidate | ONNX | GGUF, GGML, SafeTensor |
| [`huggingface`][hf] | Models hosted on HuggingFace | Release Candidate | ONNX | GGUF, GGML, SafeTensor |
| [`openai`][openai] | OpenAI (or compatible) LLM endpoint | Release Candidate | - | OpenAI-compatible HTTP endpoint |
| [`azure`][azure] | Azure OpenAI | Alpha | - | OpenAI-compatible HTTP endpoint |
| [`databricks`][databricks] | Models deployed to Databricks Mosaic AI | Alpha | - | OpenAI-compatible HTTP endpoint |

[file]: /components/embeddings/local.md
[hf]: /components/embeddings/huggingface.md
[openai]: /components/embeddings/openai.md
[azure]: /components/embeddings/azure.md
[databricks]: /components/embeddings/databricks.md

## Overview

Expand Down
2 changes: 1 addition & 1 deletion website/docs/components/embeddings/openai.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ The following parameters are specific to OpenAI models:
Below is an example configuration in `spicepod.yaml`:

```yaml
models:
embeddings:
- from: openai:text-embedding-3-large
name: xl_embed
params:
Expand Down
36 changes: 29 additions & 7 deletions website/docs/components/models/databricks.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,22 +9,44 @@ To use a language model deployed to [Databricks Mosaic AI Model Serving](https:/

### Parameters

| Parameter | Description |
| --------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `databricks_endpoint` | The Databricks workspace endpoint, e.g., `dbc-a12cd3e4-56f7.cloud.databricks.com`. |
| `databricks_token` | The Databricks API token to authenticate with the Unity Catalog API. Use the [secret replacement syntax](../secret-stores/index.md) to reference a secret, e.g., `${secrets:my_databricks_token}`. |
| Parameter | Description |
| -------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `databricks_endpoint` | The Databricks workspace endpoint, e.g., `dbc-a12cd3e4-56f7.cloud.databricks.com`. |
| `databricks_token` | The Databricks API token to authenticate with the Databricks Models API. Use the [secret replacement syntax](../secret-stores/index.md) to reference a secret, e.g., `${secrets:my_databricks_token}`. |
| `databricks_client_id` | The Databricks Service Principal Client ID. Can't be used with `databricks_token`. |
| `databricks_client_secret` | The Databricks Service Principal Client Secret. Can't be used with `databricks_token`. |

### Example `spicepod.yaml` Configuration
### Example `spicepod.yaml` configuration, using personal access token

To Learn more about how to set up personal access tokens, see [Databricks PAT docs](https://docs.databricks.com/aws/en/dev-tools/auth/pat).

```yaml
models:
- from: databricks:jeadie
name: food
- from: databricks:databricks-llama-4-maverick
name: llama-4-maverick
params:
databricks_endpoint: dbc-46470731-42e5.cloud.databricks.com
databricks_token: ${ secrets:SPICE_DATABRICKS_TOKEN }
```

### Example `spicepod.yaml` configuration, using Databricks service principal

Spice supports the M2M OAuth flow with service principal credentials by utilizing the `databricks_client_id` and `databricks_client_secret` parameters. The runtime will automatically refresh the token.

Ensure that you grant your service principal the "Can Query" permission for model serving.
Comment thread
ewgenius marked this conversation as resolved.
Outdated

To Learn more about how to set up the service principal, see [Databricks M2M OAuth docs](https://docs.databricks.com/aws/en/dev-tools/auth/oauth-m2m).

```yaml
models:
- from: databricks:databricks-llama-4-maverick
name: llama-4-maverick
params:
databricks_endpoint: dbc-46470731-42e5.cloud.databricks.com
databricks_client_id: ${secrets:DATABRICKS_CLIENT_ID}
databricks_client_secret: ${secrets:DATABRICKS_CLIENT_SECRET}
```

### Additional Information

Refer to the [Mosaic AI Model Serving documentation](https://docs.databricks.com/aws/en/machine-learning/model-serving/) for more details on available models and configurations.
Loading