Merge branch 'trunk' into jeadie/26-05-07/xai-grok-model-retirement

Author: Jeadie

website/docs/components/catalogs/cayenne.md

@@ -38,11 +38,14 @@ Use the `include` field to specify which tables to include from the catalog. The
 
 ## `params`
 
-| Parameter Name                | Description                                           | Default              |
-| ----------------------------- | ----------------------------------------------------- | -------------------- |
-| `cayenne_data_dir`            | Local directory for table data files (Vortex format). | Spice data directory |
-| `cayenne_metadata_dir`        | Local directory for Cayenne SQLite metadata.          | Spice data directory |
-| `cayenne_target_file_size_mb` | Target Vortex file size in MB.                        | `128`                |
+| Parameter Name                   | Description                                                                            | Default      |
+| -------------------------------- | -------------------------------------------------------------------------------------- | ------------ |
+| `cayenne_data_dir`               | Local directory for table data files (Vortex format).                                  | Spice data directory |
+| `cayenne_metadata_dir`           | Local directory for Cayenne SQLite metadata.                                           | Spice data directory |
+| `cayenne_target_file_size_mb`    | Target Vortex file size in MB.                                                         | `128`        |
+| `cayenne_footer_cache_mb`        | Size of the in-memory Vortex footer cache in MB for query performance.                 | `128`        |
+| `cayenne_segment_cache_mb`       | Size of the in-memory Vortex segment cache in MB for caching decompressed data.        | `256`        |
+| `cayenne_compression_strategy`   | Compression algorithm for Vortex files. Options: `btrblocks`, `zstd`.                  | `btrblocks`  |
 
 ## Examples
 

website/docs/components/catalogs/ducklake.md

@@ -70,24 +70,52 @@ The `access` field controls what operations are allowed on the catalog:
 
 ## `params`
 
-| Parameter Name               | Description                                                                                                                                           |
-| ---------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------- |
-| `ducklake_connection_string` | The DuckLake metadata location (e.g., `s3://bucket/path/metadata.ducklake`). If omitted, the value from `from: ducklake:<connection_string>` is used. |
-| `ducklake_name`              | The name to attach the DuckLake catalog as in DuckDB. Default: `ducklake`.                                                                            |
-| `ducklake_open`              | Path to an existing DuckDB file for persistent storage. If not provided, an in-memory DuckDB instance is used.                                        |
+| Parameter Name                     | Description                                                                                                                                           |
+| ---------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------- |
+| `ducklake_connection_string`       | The DuckLake metadata location (e.g., `s3://bucket/path/metadata.ducklake`). If omitted, the value from `from: ducklake:<connection_string>` is used. |
+| `ducklake_name`                    | The name to attach the DuckLake catalog as in DuckDB. Default: `ducklake`.                                                                            |
+| `ducklake_open`                    | Path to an existing DuckDB file for persistent storage. If not provided, an in-memory DuckDB instance is used.                                        |
+| `ducklake_aws_region`              | Optional. The AWS region for S3 storage. Default: `us-east-1` when explicit credentials are provided.                                                 |
+| `ducklake_aws_access_key_id`       | Optional. The AWS access key ID for S3 storage. Must be set together with `ducklake_aws_secret_access_key`.                                           |
+| `ducklake_aws_secret_access_key`   | Optional. The AWS secret access key for S3 storage. Must be set together with `ducklake_aws_access_key_id`.                                           |
+| `ducklake_aws_endpoint`            | Optional. Custom S3-compatible endpoint URL (e.g., for MinIO).                                                                                        |
+| `ducklake_aws_allow_http`          | Optional. Set to `true` to allow HTTP (non-TLS) connections to S3. Default: `false`.                                                                  |
 
 ## Authentication
 
-DuckLake relies on DuckDB's credential resolution for cloud storage access. No Spice-specific authentication parameters are needed.
-
 ### AWS S3
 
-Uses the standard AWS credential chain:
+When no explicit S3 credentials are configured, DuckDB falls back to its built-in credential chain provider:
 
 1. Environment variables (`AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, `AWS_SESSION_TOKEN`)
 2. Shared credentials file (`~/.aws/credentials`)
 3. IAM instance profiles (on EC2/ECS)
 
+To provide explicit S3 credentials, use the `ducklake_aws_*` parameters:
+
+```yaml
+catalogs:
+  - from: ducklake:s3://my-bucket/metadata.ducklake
+    name: my_lakehouse
+    params:
+      ducklake_aws_region: us-west-2
+      ducklake_aws_access_key_id: ${secrets:AWS_ACCESS_KEY_ID}
+      ducklake_aws_secret_access_key: ${secrets:AWS_SECRET_ACCESS_KEY}
+```
+
+For S3-compatible storage (e.g., MinIO), use `ducklake_aws_endpoint`:
+
+```yaml
+catalogs:
+  - from: ducklake:s3://my-bucket/metadata.ducklake
+    name: my_lakehouse
+    params:
+      ducklake_aws_endpoint: http://minio:9000
+      ducklake_aws_access_key_id: ${secrets:MINIO_ACCESS_KEY}
+      ducklake_aws_secret_access_key: ${secrets:MINIO_SECRET_KEY}
+      ducklake_aws_allow_http: true
+```
+
 ## Examples
 
 ### Local DuckLake catalog

website/docs/components/catalogs/postgres.md

@@ -59,7 +59,7 @@ Connection can be configured using a connection string or individual parameters.
 | `pg_user`        | The PostgreSQL username for authentication.                            |
 | `pg_pass`        | The PostgreSQL password for authentication.                            |
 | `pg_sslmode`     | The SSL mode for the connection (e.g. `require`, `prefer`, `disable`). |
-| `pg_sslrootcert` | Path to the SSL root certificate file.                                 |
+| `pg_sslrootcert` | Path to the SSL root certificate file, or inline PEM content.          |
 
 ## Authentication
 

website/docs/components/data-accelerators/arrow/index.md

@@ -47,8 +47,8 @@ datasets:
     acceleration:
       engine: arrow
       primary_key: order_id
-    params:
-      hash_index: enabled
+      params:
+        hash_index: enabled
 ```
 
 See [Hash Index](../../features/data-acceleration/hash-index) for configuration details, supported data types, and performance characteristics.

website/docs/components/data-accelerators/cayenne/index.md

@@ -207,7 +207,7 @@ When a query includes a `WHERE` clause, Spice Cayenne evaluates whether each seg
 For a table with segments containing timestamp ranges `[2024-01-01, 2024-01-15]`, `[2024-01-16, 2024-01-31]`, `[2024-02-01, 2024-02-15]`, a query:
 
 ```sql
-SELECT * FROM events WHERE timestamp > '2024-01-20'
+SELECT * FROM events WHERE timestamp > '2024-01-20';
 ```
 
 Prunes the first segment (max < 2024-01-20) and reads only the second and third segments.

website/docs/components/data-accelerators/duckdb/deployment.md

@@ -59,8 +59,8 @@ DuckDB self-tunes its memory limit based on system memory. For containers, set a
 
 | Parameter                    | Description                                                                                             |
 | ---------------------------- | ------------------------------------------------------------------------------------------------------- |
-| `index_scan_percentage`      | Optimizer hint: fraction of rows below which index scan is preferred over table scan.                   |
-| `index_scan_max_count`       | Optimizer hint: maximum rows for which index scan is preferred.                                         |
+| `duckdb_index_scan_percentage`      | Optimizer hint: fraction of rows below which index scan is preferred over table scan.                   |
+| `duckdb_index_scan_max_count`       | Optimizer hint: maximum rows for which index scan is preferred.                                         |
 | `on_refresh_sort_columns`    | Columns to sort by during refresh. **Caution**: current implementation uses `CREATE OR REPLACE`, which drops constraints and indexes. |
 
 DuckDB supports traditional B-tree / ART indexes via SQL `CREATE INDEX` against the accelerated table. Define them once the dataset schema is stable.
@@ -94,6 +94,6 @@ DuckDB acceleration operations participate in [task history](../../../reference/
 | Slow first startup after restart                       | WAL replay due to ungraceful shutdown.                        | Use graceful shutdown (`SIGTERM`). Subsequent starts will be fast once the checkpoint is clean.             |
 | OOM on refresh                                         | DuckDB memory limit too high for container cgroup.            | Set a `memory_limit` pragma via the connection string.                                                      |
 | Disk fills during large queries                        | Spill directory on undersized volume.                         | Point `runtime.query.temp_directory` at a larger volume; monitor free space.                                |
-| Query uses table scan when an index exists             | `index_scan_percentage` / `index_scan_max_count` too low.     | Tune thresholds; `EXPLAIN` to confirm.                                                                       |
+| Query uses table scan when an index exists             | `duckdb_index_scan_percentage` / `duckdb_index_scan_max_count` too low.     | Tune thresholds; `EXPLAIN` to confirm.                                                                       |
 | Indexes disappear after refresh                        | `on_refresh_sort_columns` triggers `CREATE OR REPLACE`.       | Re-create indexes post-refresh, or avoid sort-column refreshes until the underlying behavior is updated.    |
 | `IO Error: Could not set lock on file`                 | Another process holds a write lock.                           | Ensure single-writer semantics; verify no other Spice instance is using the same file.                      |

website/docs/components/data-accelerators/postgres/deployment.md

@@ -29,7 +29,7 @@ The accelerator uses the same Postgres wire-protocol authentication as the [Post
 | `pg_user`             | Postgres user. Must have `CREATE`, `INSERT`, `UPDATE`, `DELETE`, `SELECT` on the target schema. |
 | `pg_pass`             | Password. Use `${secrets:...}` to resolve from a configured secret store.  |
 | `pg_sslmode`          | TLS mode: `disable` / `prefer` / `require` / `verify-ca` / `verify-full`.  |
-| `pg_sslrootcert`      | CA bundle path for `verify-ca` / `verify-full`.                            |
+| `pg_sslrootcert`      | CA bundle file path for `verify-ca` / `verify-full`.      |
 
 For production, use `pg_sslmode: verify-full` and source passwords from a [secret store](../../secret-stores/). The accelerator sets `application_name` on each connection to the Spice.ai version, which surfaces in `pg_stat_activity` for attribution.
 
@@ -43,7 +43,7 @@ The accelerator creates and writes tables in the configured database. Grant the
 
 | Parameter                 | Default | Description                                                             |
 | ------------------------- | ------- | ----------------------------------------------------------------------- |
-| `connection_pool_min`     | `5`     | Minimum idle connections held by the pool.                              |
+| `pg_connection_pool_min`  | `5`     | Minimum idle connections held by the pool.                              |
 | `connection_pool_size`    | `10`    | Maximum connections the pool will open.                                 |
 
 `connection_pool_min <= connection_pool_size` is enforced at startup; mismatched values are rejected as configuration errors.

website/docs/components/data-accelerators/postgres/index.md

@@ -32,13 +32,13 @@ The connection to PostgreSQL can be configured by providing the following `param
 - `pg_user`: The username to connect with.
 - `pg_pass`: The password to connect with. Use the [secret replacement syntax](../../components/secret-stores) to load the password from a secret store, e.g. `${secrets:my_pg_pass}`.
 - `pg_sslmode`: Optional. Specifies the SSL/TLS behavior for the connection, supported values:
-  - `verify-full`: (default) This mode requires an SSL connection, a valid root certificate, and the server host name to match the one specified in the certificate.
+  - `verify-full`: This mode requires an SSL connection, a valid root certificate, and the server host name to match the one specified in the certificate.
   - `verify-ca`: This mode requires a TLS connection and a valid root certificate.
   - `require`: This mode requires a TLS connection.
-  - `prefer`: This mode will try to establish a secure TLS connection if possible, but will connect insecurely if the server does not support TLS.
+  - `prefer`: (default) This mode will try to establish a secure TLS connection if possible, but will connect insecurely if the server does not support TLS.
   - `disable`: This mode will not attempt to use a TLS connection, even if the server supports it.
   - `allow`: This mode will try a non-TLS connection first, then retry with TLS if the server requires it.
-- `pg_sslrootcert`: Optional parameter specifying the path to a custom PEM certificate that the connector will trust.
+- `pg_sslrootcert`: Optional. Path to a custom PEM certificate file that the connector will trust.
 - `pg_connection_pool_min`: Optional. The minimum number of connections to keep open in the pool, lazily created when requested. Default is `5`.
 - `connection_pool_size`: Optional. The maximum number of connections created in the connection pool. Default is `10`.