Skip to content

Commit d2a21c4

Browse files
authored
Merge branch 'trunk' into fix/1049-ci-cd-deployment-section
2 parents 7b96943 + 9f293f6 commit d2a21c4

46 files changed

Lines changed: 713 additions & 157 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

website/docs/cli/reference/login.md

Lines changed: 59 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -45,3 +45,62 @@ spice login
4545
```shell
4646
spice login --key <API_KEY>
4747
```
48+
49+
## `spice cloud login`
50+
51+
Authenticate with the Spice Cloud Platform. Running `spice cloud login` without a subcommand opens an interactive method chooser when stdin is a TTY. Non-interactive callers must specify a method explicitly.
52+
53+
### Methods
54+
55+
#### `spice cloud login subscription`
56+
57+
Browser-based OAuth login flow. Automatically opens a browser for authentication.
58+
59+
```shell
60+
spice cloud login subscription
61+
```
62+
63+
Use `--device` to print the URL and one-time code without opening a browser (useful for SSH/headless environments):
64+
65+
```shell
66+
spice cloud login subscription --device
67+
```
68+
69+
#### `spice cloud login pat`
70+
71+
Authenticate with a personal access token.
72+
73+
```shell
74+
spice cloud login pat --token <TOKEN>
75+
```
76+
77+
The token can also be provided via the `SPICE_CLOUD_PAT` environment variable:
78+
79+
```shell
80+
export SPICE_CLOUD_PAT=<TOKEN>
81+
spice cloud login pat
82+
```
83+
84+
#### `spice cloud login api`
85+
86+
Authenticate using OAuth2 client credentials for CI/automation workflows.
87+
88+
```shell
89+
spice cloud login api --client-id <CLIENT_ID> --client-secret <CLIENT_SECRET>
90+
```
91+
92+
Credentials can also be provided via environment variables:
93+
94+
```shell
95+
export SPICE_CLOUD_CLIENT_ID=<CLIENT_ID>
96+
export SPICE_CLOUD_CLIENT_SECRET=<CLIENT_SECRET>
97+
spice cloud login api
98+
```
99+
100+
### Environment Variables
101+
102+
| Variable | Used by | Description |
103+
| --- | --- | --- |
104+
| `SPICE_CLOUD_PAT` | `login pat` | Personal access token |
105+
| `SPICE_CLOUD_CLIENT_ID` | `login api` | OAuth2 client ID |
106+
| `SPICE_CLOUD_CLIENT_SECRET` | `login api` | OAuth2 client secret |

website/docs/components/data-accelerators/cayenne/index.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -418,6 +418,7 @@ Cayenne (via Vortex) supports most Arrow data types with the following considera
418418
- Timestamps (normalized to Microsecond precision)
419419
- Date32 and Date64
420420
- Lists and FixedSizeLists
421+
- Maps
421422
- Structs
422423

423424
### Automatically Converted Types
@@ -433,7 +434,6 @@ The following types require the `unsupported_type_action` parameter:
433434

434435
- `Interval` types
435436
- `Duration` types
436-
- `Map` types
437437
- `FixedSizeBinary`
438438

439439
**`unsupported_type_action` options:**
@@ -504,7 +504,7 @@ Consider the following limitations when using Spice Cayenne acceleration:
504504

505505
- **File Mode Only**: Spice Cayenne only supports `mode: file` and does not support in-memory (`mode: memory`) acceleration.
506506
- **S3 Express Only**: Standard S3 buckets are not supported for remote storage. Only S3 Express One Zone directory buckets are supported.
507-
- **Unsupported Data Types**: `Interval`, `Duration`, `Map`, and `FixedSizeBinary` types require `unsupported_type_action` configuration.
507+
- **Unsupported Data Types**: `Interval`, `Duration`, and `FixedSizeBinary` types require `unsupported_type_action` configuration.
508508
- **No Traditional Indexes**: Spice Cayenne does not support explicit index creation via the `indexes` configuration. Vortex's segment statistics and fast random access encodings provide equivalent or better performance for most point lookup workloads.
509509
- **No MVCC**: Multi-version concurrency control is not yet implemented. Snapshots and time-travel queries are planned for future releases.
510510
- **No File Compaction**: Automatic file compaction to reclaim space from deleted rows is not yet available.

website/docs/components/data-connectors/abfs.md

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -68,9 +68,11 @@ The dataset name cannot be a [reserved keyword](../../reference/spicepod/keyword
6868
| --------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------- |
6969
| `file_format` | Specifies the data format. Required if not inferrable from `from`. Options: `parquet`, `csv`. Refer to [File Formats](./#file-formats) for details. |
7070
| `abfs_account` | Azure storage account name |
71+
| `abfs_container_name` | Azure container name |
7172
| `abfs_sas_string` | SAS (Shared Access Signature) Token to use for authorization |
7273
| `abfs_endpoint` | Storage endpoint, default: `https://{account}.blob.core.windows.net` |
73-
| `abfs_use_emulator` | Use `true` or `false` to connect to a local emulator |
74+
| `abfs_use_emulator` | Use `true` or `false` to connect to a local emulator. Default: `false` |
75+
| `abfs_use_fabric_endpoint` | Use Microsoft Fabric endpoint. Default: `false` |
7476
| `abfs_authority_host` | Alternative authority host, default: `https://login.microsoftonline.com` |
7577
| `abfs_proxy_url` | Proxy URL |
7678
| `abfs_proxy_ca_certificate` | CA certificate for the proxy |
@@ -107,7 +109,7 @@ If none of these are set the connector will default to using a [managed identity
107109

108110
| Parameter name | Description |
109111
| ------------------------------- | -------------------------------------------- |
110-
| `abfs_max_retries` | Maximum retries |
112+
| `abfs_max_retries` | Maximum retries. Default: `3` |
111113
| `abfs_retry_timeout` | Total timeout for retries (e.g., `5s`, `1m`) |
112114
| `abfs_backoff_initial_duration` | Initial retry delay (e.g., `5s`) |
113115
| `abfs_backoff_max_duration` | Maximum retry delay (e.g., `1m`) |

website/docs/components/data-connectors/databricks/index.md

Lines changed: 25 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -74,8 +74,8 @@ Use the [secret replacement syntax](../secret-stores/) to reference a secret, e.
7474
| `client_timeout` | Optional. Specifies timeout for HTTP operations. In `delta_lake` mode, applies to object store operations. In `sql_warehouse` mode, applies per-HTTP-call (statement submit, status poll, chunk fetch) — not total query duration. Default: `30s`. E.g. `client_timeout: 2m` |
7575
| `connect_timeout` | Optional. Timeout for establishing TCP/TLS connections. Applies in `sql_warehouse` mode. Default: `10s`. E.g. `connect_timeout: 15s` |
7676
| `databricks_token` | The Databricks API token to authenticate with the Unity Catalog API. Can't be used with `databricks_client_id` and `databricks_client_secret`. |
77-
| `databricks_client_id` | The Databricks Service Principal Client ID. Can't be used with `databricks_token`. |
78-
| `databricks_client_secret` | The Databricks Service Principal Client Secret. Can't be used with `databricks_token`. |
77+
| `databricks_client_id` | The Databricks OAuth client ID. Used with `databricks_client_secret` for service-principal (M2M) auth, or alone for interactive User-to-Machine (U2M) auth. Can't be used with `databricks_token`. |
78+
| `databricks_client_secret` | The Databricks Service Principal Client Secret. Required for M2M auth; omit for U2M auth. Can't be used with `databricks_token`. |
7979

8080
#### SQL Warehouse tuning
8181

@@ -126,6 +126,29 @@ datasets:
126126
databricks_client_secret: ${secrets:DATABRICKS_CLIENT_SECRET} # service principal client secret
127127
```
128128

129+
### User-to-Machine (U2M) OAuth
130+
131+
Spice supports the User-to-Machine (U2M) OAuth flow for interactive sign-in against Databricks. To use U2M auth, supply only `databricks_client_id` (without `databricks_token` or `databricks_client_secret`).
132+
133+
When U2M auth is configured, the connector defers initialization until first use. On the first query the runtime opens a browser to complete the Databricks OAuth sign-in, then caches and refreshes the resulting token for subsequent requests.
134+
135+
To learn more about how to set up U2M OAuth, see the [Databricks U2M OAuth docs](https://docs.databricks.com/aws/en/dev-tools/auth/oauth-u2m).
136+
137+
:::note
138+
U2M auth is supported with `mode: delta_lake` and `mode: sql_warehouse`. It is not supported with `mode: spark_connect` — use a personal access token or service principal credentials when querying through Spark Connect.
139+
:::
140+
141+
```yaml
142+
datasets:
143+
- from: databricks:spiceai.datasets.my_awesome_table
144+
name: my_awesome_table
145+
params:
146+
mode: sql_warehouse
147+
databricks_endpoint: dbc-a1b2345c-d6e7.cloud.databricks.com
148+
databricks_sql_warehouse_id: 2b4e24cff378fb24
149+
databricks_client_id: ${secrets:DATABRICKS_CLIENT_ID} # OAuth app client id
150+
```
151+
129152
## Delta Lake object store parameters
130153

131154
Configure the connection to the object store when using `mode: delta_lake`. Use the [secret replacement syntax](../secret-stores/) to reference a secret, e.g. `${secrets:aws_access_key_id}`.

website/docs/components/data-connectors/iceberg.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -114,7 +114,7 @@ SELECT COUNT(*) FROM transactions;
114114
| `iceberg_s3_role_arn` | ARN of the IAM role to assume when accessing the S3-compatible endpoint. |
115115
| `iceberg_s3_role_session_name` | Session name to use when assuming the IAM role. |
116116
| `iceberg_s3_iam_role_source` | Optional. IAM role credential source. `auto` (default) uses the default AWS credential chain, `metadata` uses only instance/container metadata (IMDS, ECS, EKS/IRSA), `env` uses only environment variables. |
117-
| `iceberg_s3_connect_timeout` | Connection timeout in seconds for the S3-compatible endpoint. Default: `60` |
117+
| `iceberg_s3_connect_timeout` | Connection timeout in seconds for the S3-compatible endpoint. Default: `60`. **Note:** This parameter is currently accepted but has no effect — it is not consumed by any code path. |
118118
| `iceberg_sigv4_enabled` | Enable SigV4 (AWS Signature Version 4) authentication when connecting to the catalog. Automatically enabled if the URL in `from` is an AWS Glue catalog. Default: `false` |
119119
| `iceberg_signing_region` | Region to use for SigV4 authentication. Extracted from the URL in `from` if not specified. |
120120
| `iceberg_signing_name` | Service name to use for SigV4 authentication. Default: `glue`. |
@@ -264,7 +264,7 @@ INSERT INTO my_table
264264
SELECT * FROM source_table;
265265
```
266266

267-
Inserting into partitioned Iceberg tables is supported. `UPDATE` and `DELETE` operations are not currently supported.
267+
Inserting into partitioned Iceberg tables is supported. `DELETE FROM` is supported via equality delete files. `UPDATE` operations are not currently supported.
268268

269269
Write operations require `s3:PutObject` permission on the target S3 bucket in addition to the read permissions listed above. For more details, see [Data Ingestion](../../features/data-ingestion).
270270

website/docs/components/data-connectors/redshift.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -123,11 +123,11 @@ datasets:
123123
| --------------------------- | ----------------------------------------------------------------------------------------------------------------- |
124124
| `pg_connection_string` | Optional. A [PostgreSQL connection string](https://www.postgresql.org/docs/current/libpq-connect.html#LIBPQ-CONNSTRING). Overrides individual connection parameters when provided. |
125125
| `pg_host` | Hostname or IP address of the Redshift cluster |
126-
| `pg_port` | Port for Redshift (default: 5439) |
126+
| `pg_port` | The PostgreSQL TCP port. Redshift uses port `5439` by default — set this explicitly. |
127127
| `pg_db` | Database name |
128128
| `pg_user` | Username for authentication |
129129
| `pg_pass` | Password for authentication (use secret reference) |
130-
| `pg_sslmode` | SSL mode (e.g., `prefer`, `require`, `verify-ca`, `verify-full`) |
130+
| `pg_sslmode` | SSL mode (`disable`, `allow`, `prefer`, `require`, `verify-ca`, `verify-full`) |
131131
| `pg_sslrootcert` | Optional. Path to a custom root certificate for SSL verification |
132132
| `pg_connection_pool_min_idle` | Optional. The minimum number of idle connections to keep open in the pool. Default is `1`. |
133133
| `connection_pool_size` | Optional. The maximum number of connections in the connection pool. Default is `5`. |

0 commit comments

Comments
 (0)