You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
docs: Rewrite Spice.ai connector docs to cover Cloud and Spice-to-Spice
The Spice.ai data connector targets two architectures with the same
spice.ai: URI scheme, but the existing docs only covered the managed
Cloud Platform case. Add comprehensive coverage of the Spice → Spice
self-hosted federation pattern (cluster sidecar, edge-to-core,
local development) including:
- All five `from:` URI formats (path/colon/URL-style cloud + colon-prefixed
http/https/grpc+tls for self-hosted)
- The full parameter list: spiceai_api_key, spiceai_token,
spiceai_region, spiceai_endpoint, spiceai_flight_endpoint,
spiceai_tls_ca_certificate_file
- Endpoint resolution precedence (params -> from URI -> region default)
- Authentication via runtime.auth.api-key on the upstream
- TLS with custom CA pinning for cluster-internal certs
- Append-stream CDC between Spice runtimes
- Sidecar deployment patterns and example spicepods
- Topology decision matrix and per-topology troubleshooting
Also corrects parameter names that the previous docs had wrong
(plain `endpoint` -> `spiceai_endpoint`; missing `spiceai_region`).
Verified against the connector source: spice_dataset_path URI
parsing, get_endpoint resolution order, get_credentials handling,
and the full PARAMETERS array.
title: 'Spice.ai Cloud Platform Data Connector Deployment Guide'
2
+
title: 'Spice.ai Data Connector Deployment Guide'
3
3
sidebar_label: 'Deployment Guide'
4
-
description: 'Operating guide for the Spice.ai Cloud Platform connector in production: API keys, Flight endpoints, message sizing, and observability.'
4
+
description: 'Operating guide for the Spice.ai connector in production: API keys, Flight endpoints, message sizing, sidecar topology, and observability.'
5
5
sidebar_position: 10
6
6
pagination_prev: null
7
7
pagination_next: null
@@ -11,80 +11,177 @@ tags:
11
11
- observability
12
12
---
13
13
14
-
Production operating guide for the [Spice.ai Cloud Platform](https://spice.ai) data connector covering authentication, Arrow Flight transport, and operational tuning.
14
+
Production operating guide for the [Spice.ai Data Connector](./index.md), covering both the **Spice → Spice Cloud Platform** and **Spice → Spice (self-hosted / cluster-sidecar)** topologies. The connector uses [Arrow Flight](https://arrow.apache.org/docs/format/Flight.html) over gRPC for both.
| Federate datasets hosted on the managed Spice.ai Cloud Platform | Spice → Spice Cloud |`https://<region>-prod-aws-flight.spiceai.io` (auto) |
21
+
| Per-pod sidecar federating to a heavier upstream Spice runtime | Spice → Spice (sidecar)|`https://upstream.cluster.svc:50051`|
22
+
| Edge runtime federating cold queries to a core Spice | Spice → Spice | Cluster-internal `https://...`|
23
+
| Local development against a Spice on `localhost`| Spice → Spice |`http://localhost:50051`|
24
+
25
+
Both topologies use the same `spiceai`-prefixed parameters and the same `spice.ai:``from:` URI scheme. See the [connector reference](./index.md#configuration) for the full parameter list and URI formats.
15
26
16
27
## Authentication & Secrets
17
28
18
-
The connector authenticates to the Spice.ai Cloud Platform using an API key supplied via the `spiceai_api_key` parameter. Endpoints default to the Spice.ai production cluster; override via `endpoint` for VPC or regional endpoints.
29
+
The connector authenticates to the upstream Spice runtime using `spiceai_api_key`. The same parameter covers both Cloud and self-hosted upstreams:
|Cloud| Spice.ai Console; written to `.env` by `spice login`| Yes |
34
+
|Self-hosted | Listed in the upstream's `runtime.auth.api-key.keys`| Only if upstream has auth enabled (otherwise anonymous)|
24
35
25
-
API keys must be sourced from a [secret store](../../secret-stores/) in production. Rotate keys from the Spice.ai Console and update the secret store without restarting the runtime if the secret store supports live reloads.
|`spiceai_flight_endpoint`| Legacy alias for `spiceai_endpoint`. |
43
+
|`spiceai_tls_ca_certificate_file`| Path to a CA PEM file for verifying a self-hosted upstream that uses a private CA. Ignored for `http://` endpoints. |
44
+
45
+
Always source production keys from a managed secret store rather than from a checked-in `.env` file. API keys do not expire — rotate manually in the issuing system and update the secret store. Secret stores that support live reload (Kubernetes, Vault) pick up rotations without restarting the runtime.
26
46
27
47
## Resilience Controls
28
48
29
49
### Endpoint Verification
30
50
31
-
On startup the connector performs a DNS + TCP reachability check against the configured `endpoint` before attempting a Flight handshake. This surfaces misconfigured endpoints as actionable startup errors rather than slow-failure query errors.
51
+
On startup the connector performs a DNS + TCP reachability check against the resolved endpoint before attempting a Flight handshake. Misconfigured endpoints surface as actionable startup errors rather than slow-failure query errors.
32
52
33
53
### Flight Transport
34
54
35
-
Data transfer uses [Arrow Flight](https://arrow.apache.org/docs/format/Flight.html) over gRPC with TLS. Transient gRPC errors (`UNAVAILABLE`, `DEADLINE_EXCEEDED`) surface to the caller; retries are handled by the Flight client's default policy.
55
+
Data transfer uses Arrow Flight over gRPC. Transient gRPC errors (`UNAVAILABLE`, `DEADLINE_EXCEEDED`) surface to the caller; retries are handled by the Flight client's default policy.
56
+
57
+
For self-hosted upstreams, prefer `https://` or `grpc+tls://` in production. `http://` is supported for local development and trusted networks but transmits Flight payloads unencrypted. Plain `grpc://` is rejected at startup.
58
+
59
+
### TLS and private CAs
60
+
61
+
By default the connector trusts the system certificate store. For cluster-internal upstreams that present a private-CA-signed certificate, pin the CA explicitly:
The CA file is loaded once at startup; updates require a runtime restart. Mount it via Kubernetes ConfigMap or Secret in containerized deployments.
36
70
37
71
### Append Streams
38
72
39
-
The connector supports long-lived append streams (`supports_changes_stream`) for real-time CDC into accelerated datasets. Stream reconnection is handled automatically; loss of connection to the Cloud Platform results in the dataset being marked `Error` if the lag exceeds the configured acceptable window (see [Data Refresh](../../../features/data-acceleration/data-refresh))..
73
+
The connector supports long-lived append streams for real-time CDC. The upstream — whether Cloud or self-hosted — must expose a dataset with append-stream support. The sidecar subscribes over Flight `DoExchange` and receives each new batch as soon as it's emitted. Stream reconnection is automatic; persistent loss of connection causes the dataset to enter `Error` state if the lag exceeds the acceptable window. See [Data Refresh](../../../features/data-acceleration/data-refresh).
74
+
75
+
Append streams are append-only — deletes and updates from the upstream are **not** propagated. Use `refresh_mode: full` for datasets that mutate.
40
76
41
77
## Capacity & Sizing
42
78
43
79
### Message Sizing
44
80
45
-
Arrow Flight record batches may exceed the default gRPC 4 MiB message limit for wide or dense schemas. Control with:
81
+
Arrow Flight record batches may exceed the default gRPC 4 MiB message limit for wide or dense schemas:
| `max_message_size` | `4MB` | Maximum inbound gRPC message size. Raise for wide result sets or many string columns. |
50
86
51
-
Set via Spice configuration or via environment at runtime startup. Accepted units: `B`, `KB`, `MB`, `GB`.
87
+
Set in spicepod parameters or via environment at runtime startup. Accepted units: `B`, `KB`, `MB`, `GB`. The same limit applies to the upstream's Flight server — raising it on the client without raising it on the server still fails.
52
88
53
89
### Network
54
90
55
-
- The Spice.ai Cloud Platform is a managed service; place Spice runtime in a region geographically close to your Cloud Platform region to minimize round-trip latency.
56
-
- Expect typical query round-trip latency in the tens of milliseconds + result streaming time; for interactive dashboards, accelerate (`acceleration: enabled`) to cache into a local engine.
91
+
#### Cloud topology
92
+
93
+
- Place the Spice runtime in a region geographically close to the Cloud Platform region (`spiceai_region`) to minimize round-trip latency.
94
+
- Expect typical round-trip latency in the tens of milliseconds plus result streaming time. For interactive dashboards, accelerate (`acceleration.enabled: true`) into a local engine.
95
+
96
+
#### Self-hosted / sidecar topology
97
+
98
+
- Run the sidecar **in the same network namespace or cluster** as the upstream when possible — Flight is most efficient over single-digit-millisecond RTT.
99
+
- Size sidecar memory for the local query workspace plus any in-memory acceleration. The sidecar does not need to be sized for the full dataset — only for hot data accelerated locally and the working set of in-flight queries.
100
+
- Use a Kubernetes `Service` (with stable DNS) or a load balancer in front of multi-replica upstreams. Connection pooling is per-endpoint URL.
101
+
102
+
### API key lifetime
103
+
104
+
API keys do not expire. Rotation is manual; coordinate with the secret store used by the runtime.
105
+
106
+
## Sidecar deployment patterns
107
+
108
+
### Per-pod sidecar (Kubernetes)
109
+
110
+
Co-locate a Spice sidecar with each application pod. The sidecar terminates HTTP / OpenAPI / MCP / gRPC for the app and federates queries to a central upstream Spice cluster.
111
+
112
+
```yaml
113
+
# Sidecar spicepod.yaml mounted into each app pod
114
+
version: v1
115
+
kind: Spicepod
116
+
name: app-sidecar
117
+
118
+
runtime:
119
+
http:
120
+
bind_address: 127.0.0.1:8090 # localhost-only, sidecar talks to the app
The application talks to `127.0.0.1:8090`; the sidecar handles federation and caching. The upstream runs as a `Deployment` or `StatefulSet` with persistent storage for the acceleration files.
135
+
136
+
### Edge → core federation
137
+
138
+
Edge Spice runtimes accelerate local datasets and federate the long-tail to a core Spice in the data center. The same connector is used; the difference is that edge runtimes have their own non-federated datasets too:
Flight transport metrics are collected via the shared Flight client instrumentation. The connector does not currently register Spice.ai-specific dataset-level instruments. Monitor the connector via:
65
154
66
155
- Query execution metrics (`query_duration_ms`, `query_processed_rows`, `query_failures_total`) from `runtime.metrics`.
67
156
- Acceleration refresh metrics when the dataset is accelerated (`refresh_last_duration_ms`, `refresh_errors_total`).
68
-
- Upstream Spice.ai Console metrics on the source dataset.
157
+
- For Cloud: upstream Console metrics on the source dataset.
158
+
- For self-hosted: monitor the upstream's own `runtime.metrics` and acceleration metrics.
69
159
70
160
See [Component Metrics](../../../features/observability/component_metrics) for general configuration.
71
161
72
162
## Task History
73
163
74
-
Queries to the Spice.ai Cloud Platform participate in [task history](../../../reference/task_history) via the Flight client spans. Each Flight request is recorded as a child of the enclosing `sql_query` or `accelerated_table_refresh` task.
164
+
Queries to the upstream Spice runtime participate in [task history](../../../reference/task_history) via Flight client spans. Each Flight request is recorded as a child of the enclosing `sql_query` or `accelerated_table_refresh` task. The upstream runtime records its own task history independently — correlate by request timestamps or by propagated trace IDs.
75
165
76
166
## Known Limitations
77
167
78
-
-**Read-only**: The connector is read-only; writes to the Cloud Platform are done via the Spice CLI / Console, not the runtime.
79
-
-**Single endpoint per dataset**: A dataset binds to a single `endpoint`. Multi-region failover must be handled at the load-balancer / DNS layer.
80
-
-**API key auth only**: OIDC / SSO is not currently supported at the data-plane connector.
168
+
- **Read-only.** The connector does not write to the upstream. Cloud writes go through the Spice CLI / Console; self-hosted writes happen on the upstream runtime directly.
169
+
- **Single endpoint per dataset.** A dataset binds to a single endpoint URL. Multi-endpoint failover lives at the load-balancer / DNS layer.
170
+
- **API key auth only.** OIDC / SSO is not supported at the data-plane connector.
171
+
- **Append-only changes stream.** Updates and deletes are not propagated.
172
+
- **Cloud connections cap at 1000 requests per connection.** When the cap is hit the connection is reset; the Flight client retries automatically. The `spiceai-retryable` metadata flag indicates the retry path.
173
+
- **No `grpc://` (clear-text gRPC).** Use `http://` for unencrypted Flight or `https://` / `grpc+tls://` for TLS.
|`Failed to connect to SpiceAI endpoint`| DNS, firewall, or TLS issue against `endpoint`. | Verify DNS resolution and outbound 443 connectivity. Test with `grpcurl -insecure <endpoint>:443 list`. |
87
-
|`UNAUTHENTICATED` on Flight handshake | Invalid / expired / wrong-environment API key. | Regenerate the key in the Spice.ai Console, update the secret store. |
88
-
|`message size exceeded` / `ResourceExhausted`| Row batch exceeds gRPC message limit. | Increase `max_message_size`, or narrow the query projection. |
89
-
| Append stream stalled; acceleration lag climbing | Network partition or upstream dataset paused. | Check Cloud Platform status; verify the source dataset is healthy; restart the runtime to re-establish the stream. |
| `Failed to connect to SpiceAI endpoint` | DNS, firewall, or TLS issue against the resolved endpoint. | Verify DNS resolution and outbound 443/50051 connectivity. Test with `grpcurl -insecure <host>:<port> list`. |
180
+
| `UnsupportedEndpointScheme` | Endpoint uses `grpc://` or another unsupported scheme. | Switch to `http://`, `https://`, or `grpc+tls://`. |
181
+
| `CloudEndpointRegionMismatch` | `spiceai_endpoint` is a Cloud regional URL but `spiceai_region` doesn't match. | Set both to the same region, or remove one and let Spice pick the other. |
182
+
| `UNAUTHENTICATED` on Flight handshake | Invalid / expired / wrong-environment API key. | For Cloud: regenerate in the Console; update the secret store. For self-hosted: confirm the key is in the upstream's `runtime.auth.api-key.keys`. |
183
+
| TLS handshake failure with self-signed upstream cert | System cert store doesn't trust the upstream CA. | Set `spiceai_tls_ca_certificate_file` to the upstream's CA PEM, or have the upstream present a publicly-trusted certificate. |
184
+
| `message size exceeded` / `ResourceExhausted` | Row batch exceeds gRPC message limit. | Increase `max_message_size` on both client and server, or narrow the query projection. |
185
+
| Append stream stalled; acceleration lag climbing | Network partition or upstream dataset paused. | Check upstream status; verify the source dataset is healthy; restart the runtime to re-establish the stream. |
186
+
| Sudden 5xx / `UNAVAILABLE` errors | Transient service-side issue. | Flight client auto-retries; if persistent, check upstream runtime health (or the [Spice.ai status page](https://status.spice.ai)). |
187
+
| `MissingRequiredParameter: api_key or token` | Targeting a Cloud endpoint with no API key configured. | Set `spiceai_api_key` (Cloud requires authentication; self-hosted endpoints accept anonymous if upstream auth is off). |
0 commit comments