Skip to content

Commit 158232a

Browse files
authored
docs: update roadmap and distributions for April 2026 (spiceai#10367)
* docs: update roadmap and distributions for April 2026 - Add Schema Evolution to v2.1 roadmap. - Add OIDC Token Verification to DISTRIBUTIONS.md (Cloud + Enterprise). - Link Spice.ai Enterprise mentions to https://docs.spice.ai/docs/enterprise. * docs(criteria): audit and normalize release criteria - Fix naming: Spice.AI -> Spice.ai, Azure Openai -> Azure OpenAI. - Fix typos: Embdedded -> Embedded, 'connector' -> 'accelerator' in accelerators/stable.md, 'Component ensure' -> 'Components ensure'. - connectors/beta.md: correct schema-inference legend (add native classification). - connectors/stable.md: rename 'Schema Detection' header to 'Schema Inference' for consistency. - connectors: add ADBC, Azure BlobFS (sorted), Debezium, DuckLake, GCS, MongoDB, NFS, ScyllaDB, SMB rows across all tiers; restore Iceberg, Glue, Kafka, Oracle dropouts in beta/rc/stable; alphabetize all tables. - catalogs: add ADBC, Cayenne, DuckLake, MS SQL, MySQL, Oracle, PostgreSQL, Snowflake rows across all tiers; restore Glue in beta/rc/stable; alphabetize. - models: add AWS Bedrock, Databricks, Google (Gemini); restore Nvidia NIM in beta/rc/stable; alphabetize. - embeddings: add AWS Bedrock, Databricks; remove xAI (no embeddings API); alphabetize. - grading.md: fix heading hierarchy (Grade C/B/A as H3) and naming consistency. - PRINCIPLES.md: expand to clarify scope and interaction between component and feature criteria. - definitions.md: add SF5 to TPC-H and TPC-DS parallel-query tables; clarify stop-light percentile boundaries. - features/README.md: add H1 title and reference to PRINCIPLES.md. * docs(criteria): require Deployment Guide for GA/Stable components Adds a Deployment Guide requirement to Stable release criteria for connectors, accelerators, catalogs, models, embeddings, and generic features. References the Databricks Deployment Guide as a canonical example: https://spiceai.org/docs/next/components/data-connectors/databricks/deployment * docs(criteria): add Databricks sql_warehouse mode Tracks the sql_warehouse mode alongside delta_lake and spark_connect across alpha/beta/rc/stable connector criteria tables. New mode defaults to not-started (no DRI). * docs(criteria): rename AWS Bedrock to Amazon Bedrock Use the official 'Amazon Bedrock' branding. Anthropic and Amazon Bedrock are both listed across models and (Bedrock) embeddings tiers. * docs(criteria): standardize accelerator tables format across all criteria documents * docs(criteria): add SpiceBench criteria for accelerators and connectors
1 parent f4e7238 commit 158232a

27 files changed

Lines changed: 417 additions & 228 deletions

docs/DISTRIBUTIONS.md

Lines changed: 23 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ The Spice open source project provides multiple distribution variants to support
44

55
> **Note:** The Spice runtime is **64-bit only**. 32-bit platforms are not supported.
66
7-
> **Note:** Variant distributions (data, allocators, CUDA) are only available in **nightly images** for the open source project. All features and distributions are available in the [Spice Cloud Platform](https://spice.ai/pricing) and [Spice.ai Enterprise](https://spice.ai/pricing).
7+
> **Note:** Variant distributions (data, allocators, CUDA) are only available in **nightly images** for the open source project. All features and distributions are available in the [Spice Cloud Platform](https://spice.ai/pricing) and [Spice.ai Enterprise](https://docs.spice.ai/docs/enterprise).
88
99
## Supported Platforms & Hardware Requirements
1010

@@ -19,15 +19,16 @@ The Spice open source project provides multiple distribution variants to support
1919
2020
## Distribution Availability
2121

22-
| Distribution / Variant | Open Source | Spice Cloud | Enterprise |
23-
| ---------------------- | ---------------- | ----------- | ---------- |
24-
| Default (Data + AI) ||||
25-
| Data-only | Nightly only |||
26-
| NAS (SMB + NFS) | Nightly only |||
27-
| Metal (macOS) ||||
28-
| CUDA (Linux) | Nightly only |||
29-
| Allocator variants | Nightly only |||
30-
| ODBC connector | Local build only |||
22+
| Distribution / Variant | Open Source | Spice Cloud | Enterprise |
23+
| ----------------------- | ---------------- | ----------- | ---------- |
24+
| Default (Data + AI) ||||
25+
| Data-only | Nightly only |||
26+
| NAS (SMB + NFS) | Nightly only |||
27+
| Metal (macOS) ||||
28+
| CUDA (Linux) | Nightly only |||
29+
| Allocator variants | Nightly only |||
30+
| ODBC connector | Local build only |||
31+
| OIDC Token Verification ||||
3132

3233
## Default Distribution
3334

@@ -61,7 +62,7 @@ docker pull spiceai/spiceai:latest
6162

6263
The data distribution excludes AI/ML model support, resulting in a smaller binary size and reduced attack surface. Use this when you only need data federation and acceleration capabilities without AI features.
6364

64-
> **Open Source:** Available in nightly builds only. **[Cloud Platform & Enterprise](https://spice.ai/pricing):** Production-ready data distribution available.
65+
> **Open Source:** Available in nightly builds only. **[Cloud Platform](https://spice.ai/pricing) & [Enterprise](https://docs.spice.ai/docs/enterprise):** Production-ready data distribution available.
6566
6667
**Included Features:**
6768

@@ -108,7 +109,7 @@ make install-metal
108109

109110
For Linux systems with NVIDIA GPUs, CUDA distributions enable GPU-accelerated AI/ML inference. Multiple CUDA compute capability versions are available.
110111

111-
> **Open Source:** Available in nightly builds only. **[Cloud Platform & Enterprise](https://spice.ai/pricing):** Production-ready CUDA distribution available.
112+
> **Open Source:** Available in nightly builds only. **[Cloud Platform](https://spice.ai/pricing) & [Enterprise](https://docs.spice.ai/docs/enterprise):** Production-ready CUDA distribution available.
112113
113114
**Included Features:**
114115

@@ -139,7 +140,7 @@ CUDA_COMPUTE_CAP=89 make install-cuda
139140

140141
The NAS (Network Attached Storage) distribution adds support for SMB and NFS data connectors, enabling federated queries against data stored on network file shares.
141142

142-
> **[Enterprise](https://spice.ai/pricing):** The NAS distribution is available in nightly builds and with Spice.ai Enterprise.
143+
> **[Enterprise](https://docs.spice.ai/docs/enterprise):** The NAS distribution is available in nightly builds and with Spice.ai Enterprise.
143144
144145
**Included Features:**
145146

@@ -157,7 +158,7 @@ make install-nas
157158

158159
Different memory allocators can significantly impact performance depending on workload characteristics.
159160

160-
> **Open Source:** Allocator variants are available in nightly builds only for testing and benchmarking. **[Cloud Platform & Enterprise](https://spice.ai/pricing):** Production-ready allocator variants available.
161+
> **Open Source:** Allocator variants are available in nightly builds only for testing and benchmarking. **[Cloud Platform](https://spice.ai/pricing) & [Enterprise](https://docs.spice.ai/docs/enterprise):** Production-ready allocator variants available.
161162
162163
### snmalloc (Default)
163164

@@ -203,7 +204,7 @@ docker pull ghcr.io/spiceai/spiceai-nightly:latest-sysalloc
203204
| Windows (WSL) || Nightly | Nightly || Nightly |
204205
| Windows (Native) || Enterprise only | Enterprise only || Enterprise only |
205206

206-
> **Note:** Native Windows support for the Spice runtime is available with the [Spice Cloud Platform and Spice.ai Enterprise](https://spice.ai/pricing). Open source users on Windows should use Windows Subsystem for Linux (WSL).
207+
> **Note:** Native Windows support for the Spice runtime is available with the [Spice Cloud Platform](https://spice.ai/pricing) and [Spice.ai Enterprise](https://docs.spice.ai/docs/enterprise). Open source users on Windows should use Windows Subsystem for Linux (WSL).
207208
208209
## Choosing a Distribution
209210

@@ -218,7 +219,7 @@ docker pull ghcr.io/spiceai/spiceai-nightly:latest-sysalloc
218219

219220
## Additional Connectors
220221

221-
Some connectors require additional dependencies and are available with the [Spice Cloud Platform and Spice.ai Enterprise](https://spice.ai/pricing):
222+
Some connectors require additional dependencies and are available with the [Spice Cloud Platform](https://spice.ai/pricing) and [Spice.ai Enterprise](https://docs.spice.ai/docs/enterprise):
222223

223224
- **ODBC** - Connect to any ODBC-compatible data source
224225

@@ -228,6 +229,12 @@ These can be built locally for development and testing:
228229
make install-odbc
229230
```
230231

232+
## Enterprise Authentication
233+
234+
The following authentication features are available with the [Spice Cloud Platform](https://spice.ai/pricing) and [Spice.ai Enterprise](https://docs.spice.ai/docs/enterprise):
235+
236+
- **OIDC Token Verification** - Validate identity tokens from enterprise providers (Okta, Azure AD, Auth0, Google, etc.) for secure access to Spice runtime endpoints.
237+
231238
### Linux arm64 Notes
232239

233240
- **FP16 (FEAT_FP16)** is required because the `gemm` matrix multiplication library (used by the Candle ML framework) contains half-precision ARM inline assembly that requires the `fullfp16` CPU feature. This is supported on AWS Graviton2+, Ampere Altra, Apple M-series (via Linux VM), and most ARMv8.2-A+ processors.

docs/ROADMAP.md

Lines changed: 13 additions & 28 deletions
Original file line numberDiff line numberDiff line change
@@ -15,56 +15,51 @@ To propose features or report issues, please [file an issue](https://github.com/
1515

1616
## Release Timeline
1717

18-
### v2.0 (March 2026)
18+
### v2.0 (April 2026)
1919

2020
**Focus:** Production-Grade HA.
2121

2222
- **Spice Cayenne (GA)**: Production-ready columnar storage format for accelerated datasets.
23-
- **Active-Active HA (GA)**: Production-ready multi-node deployment with zero-downtime failover.
23+
- **Multi-Active HA (GA)**: Production-ready multi-node deployment with zero-downtime failover.
2424
- **Distributed Query (GA)**: Stable multi-node query execution for large-scale workloads.
25-
- **Backup & Point-in-Time Recovery**: Snapshot restore for accelerated datasets.
2625
- **Accelerated Dataset Distribution**: Replicate accelerated datasets across executor nodes.
2726

2827
### v2.1 (H2 2026)
2928

30-
**Focus:** Schema Management & Search.
29+
**Focus:** Schema Management, Search, and Policy.
3130

31+
- **Elasticsearch**: Integration for search and analytics workloads.
3232
- **Distributed Search (Alpha)**: Federated vector and full-text search across multiple nodes.
3333
- **Schema Registry (Initial)**: Versioning and backward compatibility checks.
34+
- **Schema Evolution**: Safe, non-breaking schema changes for accelerated datasets (add/drop/rename columns, type widening) with automatic migration.
3435
- **Hybrid Search Ranking**: Configurable fusion strategies (Reciprocal Rank Fusion (RRF) weights, learned ranking).
36+
- **Policy Engine (Cedar-based) Beta**:
37+
- Role-based security for fine-grained access control.
38+
- Data access policies to enforce compliance and governance.
3539

3640
### v2.2 (H2 2026)
3741

3842
**Focus:** Reactive Actions & Event Processing.
3943

40-
- **Actions (Drasi-based)**: Reactive event-driven actions triggered by data changes.
4144
- **Webhooks & Event Notifications**: Push-based data change alerts for downstream consumers.
45+
- **Actions (Drasi-based)**: Reactive event-driven actions triggered by data changes.
4246

4347
### v2.3 (H2 2026)
4448

45-
**Focus:** Enterprise Security & Compliance.
49+
**Focus:** Enterprise Security, Compliance, & Governance.
4650

4751
- **Data Masking & Anonymization**: Dynamic PII redaction at query time.
48-
- **OIDC Token Verification**: Validate identity tokens from enterprise providers (Okta, Azure AD, etc.).
49-
50-
### v2.4 (2027)
51-
52-
**Focus:** Governance.
53-
54-
- **Policy Engine (Cedar-based)**:
55-
- Role-based security for fine-grained access control.
56-
- Data access policies to enforce compliance and governance.
5752
- **Audit Logging**: Persistent, immutable query and access logs for compliance.
5853
- **Resource Quotas**: Per-user/tenant query limits and throttling.
5954

60-
### v2.5 (2027)
55+
### v2.4 (2027)
6156

6257
**Focus:** Extensibility & Plugin Architecture.
6358

6459
- **Extensible Middleware**: Pluggable extensions for dynamic customization.
6560
- **Custom Tool Providers**: User-defined MCP tool implementations.
6661

67-
### v2.6 (2027)
62+
### v2.5 (2027)
6863

6964
**Focus:** Encryption.
7065

@@ -80,9 +75,6 @@ These are prioritized based on community demand and strategic alignment. Share y
8075
### Data Connectors
8176

8277
- **Delta Lake**: Write support for transactional data lakes.
83-
- **Elasticsearch**: Integration for search and analytics workloads.
84-
- **MotherDuck**: Hybrid cloud DuckDB connector.
85-
- **BigQuery**: Support for Google Cloud's data warehouse.
8678
- **Google Docs**: Experimental connector for collaborative data sources.
8779

8880
### APIs
@@ -94,7 +86,7 @@ These are prioritized based on community demand and strategic alignment. Share y
9486
### AI & Analytics
9587

9688
- **Vision Processing**: Support for image and video.
97-
- **Custom Model Integration**: Framework for user-defined ML models.
89+
- **Custom ML Model Integration**: Framework for user-defined ML models.
9890
- **Model Versioning & A/B Testing**: Canary deployments and version management for models.
9991
- **Hallucination Detection**: Fact-checking LLM responses against source data.
10092

@@ -105,13 +97,6 @@ These are prioritized based on community demand and strategic alignment. Share y
10597
### Data Platform
10698

10799
- **Data Lineage**: Track data provenance and transformations across the pipeline.
108-
- **Schema Registry & Evolution**: Versioning and backward compatibility checks.
109-
110-
### Platform Enhancements
111-
112-
- **Observability**: Comprehensive metrics, logging, and tracing.
113-
- **CLI & SDKs**: Streamlined developer experience with improved tooling.
114-
- **Data Accelerators**: Additional engines for specialized workloads.
115100

116101
---
117102

docs/criteria/PRINCIPLES.md

Lines changed: 34 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -1,23 +1,47 @@
11
# Principles
2+
23
## Overview
3-
- Beta internally QA complete; RC externally QA complete.
4-
- Documentation quality should match code quality standard.
4+
5+
- Beta internally QA complete; RC externally QA complete.
6+
- Documentation quality should match code quality standard.
7+
8+
## Scope
9+
10+
A "component" refers to any of the following:
11+
12+
- Data Connectors — see [connectors/](./connectors/).
13+
- Data Accelerators — see [accelerators/](./accelerators/).
14+
- Catalog Connectors — see [catalogs/](./catalogs/).
15+
- Models (LLM providers) — see [models/](./models/).
16+
- Embedding providers — see [embeddings/](./embeddings/).
17+
18+
Runtime subsystems and cross-cutting capabilities that are not a single component (e.g. search, HA, distributed query) use the generic [features/](./features/) criteria.
19+
20+
The generic feature criteria and the component-specific criteria are complementary:
21+
22+
- **Component criteria apply first.** Connector/accelerator/catalog/model/embedding releases are governed by the files in their respective subdirectories.
23+
- **Feature criteria apply to component releases as well**, wherever the component-specific criteria do not already cover the same requirement (e.g. preview-warning removal, deprecation policy). Where the two conflict, the component criteria win.
524

625
## Per Level
26+
727
### Alpha
28+
829
- Code complete at quality benchmark for most basic, valuable functionality.
930
- Limitations & edge case failures expected.
1031

1132
### Beta
12-
- Functionally complete.
13-
- Satisfies minimal performance bar. Resource efficiency (memory, CPU, and I/O usage) is measured.
14-
- Metrics & tracing in place for observability.
33+
34+
- Functionally complete.
35+
- Satisfies minimal performance bar. Resource efficiency (memory, CPU, and I/O usage) is measured.
36+
- Metrics & tracing in place for observability.
1537

1638
### RC
17-
- Ready for production: security & privacy
18-
- Performance tested at scale
19-
- Feature complete; API stable.
20-
- Component ensure correctness.
39+
40+
- Ready for production: security & privacy.
41+
- Performance tested at scale.
42+
- Feature complete; API stable.
43+
- Components ensure correctness.
2144

2245
### Stable
23-
- Verified in production.
46+
47+
- Verified in production.

docs/criteria/accelerators/alpha.md

Lines changed: 9 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -6,13 +6,15 @@ All criteria must be met for the Accelerator to be considered Alpha. As Alpha si
66

77
## Alpha Quality Accelerators
88

9-
| Accelerator | Alpha Quality | DRI Sign-off |
10-
| ----------- | ------------- | ------------ |
11-
| Arrow || @Sevenannn |
12-
| Cayenne || @lukekim |
13-
| DuckDB || @peasee |
14-
| SQLite || @sgrebnov |
15-
| PostgreSQL || @sgrebnov |
9+
| Accelerator | Alpha Quality | DRI Sign-off |
10+
| ----------------------- | ------------- | ------------ |
11+
| Arrow || @Sevenannn |
12+
| Cayenne || @lukekim |
13+
| DuckDB || @peasee |
14+
| SQLite || @sgrebnov |
15+
| PostgreSQL<sup>\*</sup> || @sgrebnov |
16+
17+
<sup>\*</sup> PostgreSQL is a [Spice.ai Enterprise](https://docs.spice.ai/docs/enterprise)-only accelerator.
1618

1719
## Alpha Release Criteria
1820

docs/criteria/accelerators/beta.md

Lines changed: 10 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -6,13 +6,15 @@ All criteria must be met for the accelerator to be considered Beta, with excepti
66

77
## Beta Quality Accelerators
88

9-
| Accelerator | Beta Quality | DRI Sign-off |
10-
| ----------- | ------------ | ------------ |
11-
| Arrow || @Sevenannn |
12-
| Cayenne || @phillipleblanc |
13-
| DuckDB || @peasee |
14-
| SQLite || @sgrebnov |
15-
| PostgreSQL || @sgrebnov |
9+
| Accelerator | Beta Quality | DRI Sign-off |
10+
| ----------------------- | ------------ | --------------- |
11+
| Arrow || @Sevenannn |
12+
| Cayenne || @phillipleblanc |
13+
| DuckDB || @peasee |
14+
| SQLite || @sgrebnov |
15+
| PostgreSQL<sup>\*</sup> || @sgrebnov |
16+
17+
<sup>\*</sup> PostgreSQL is a [Spice.ai Enterprise](https://docs.spice.ai/docs/enterprise)-only accelerator.
1618

1719
## Beta Release Criteria
1820

@@ -28,7 +30,7 @@ The Beta release criteria expand on and require that all [Alpha release criteria
2830
- [ ] The accelerator supports full federation within a single dataset (e.g. `select * from my_dataset`)
2931
- [ ] The accelerator supports federation push down across multiple datasets within the same accelerator (e.g. `select * from first_dataset, second_dataset`)
3032
- [ ] The accelerator supports resolving on conflict behaviors (e.g. Drop/Upsert)
31-
- [ ] Embdedded accelerators support file-mode storage (e.g. SQLite, DuckDB)
33+
- [ ] Embedded accelerators support file-mode storage (e.g. SQLite, DuckDB)
3234
- [ ] [Core Arrow Data Types](../definitions.md) are supported
3335
- [ ] All known [Major Bugs](../definitions.md) are resolved
3436
- [ ] The accelerator supports reading datasets with the same max column count as the accelerator source.

docs/criteria/accelerators/rc.md

Lines changed: 9 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -6,13 +6,15 @@ All criteria must be met for the accelerator to be considered [RC](../definition
66

77
## RC Quality Accelerators
88

9-
| Accelerator | RC Quality | DRI Sign-off |
10-
| ----------- | ---------- | ------------ |
11-
| Arrow || @sgrebnov |
12-
| Cayenne || @peasee |
13-
| DuckDB || @peasee |
14-
| SQLite || @peasee |
15-
| PostgreSQL || @peasee |
9+
| Accelerator | RC Quality | DRI Sign-off |
10+
| ----------------------- | ---------- | ------------ |
11+
| Arrow || @sgrebnov |
12+
| Cayenne || @peasee |
13+
| DuckDB || @peasee |
14+
| SQLite || @peasee |
15+
| PostgreSQL<sup>\*</sup> || @peasee |
16+
17+
<sup>\*</sup> PostgreSQL is a [Spice.ai Enterprise](https://docs.spice.ai/docs/enterprise)-only accelerator.
1618

1719
## RC Release Criteria
1820

0 commit comments

Comments
 (0)