[pull] trunk from spiceai:trunk#29
Merged
Merged
Conversation
* Tweak enforce pulls with Spice * Use v1.0.5
…sage size (#5407) * Allow `flightsql` and `spiceai` connectors to override default flight max message size * Update
pull Bot
pushed a commit
that referenced
this pull request
May 12, 2026
* v2.0 threat model fix: Address review comments on v2.0 threat model - Remove duplicate threat #14 row in updated-threats table - Clarify async query IDs as UUIDv7-derived (Databricks-style), not raw UUID strings - Fix Threat Dragon flow metadata: mark Cluster Control Plane and Task Dispatch as isEncrypted: true, isPublicNetwork: false to match the mTLS-protected internal port (50052) described in the model Update threat model documentation for v2.0.0 - Refine threat surface description to include AI/model/search endpoints and clarify SQL engine logic. - Enhance details on SMB/NFS internal network access and telemetry exfiltration threats. - Revise DDL and DML enforcement section to specify catalog access mode and backend capability. - Expand on security hardening mitigations and AI/tool/search surface threats. - Update port layout and authentication requirements for various services. - Clarify the implications of anonymous telemetry and task history data retention. fix: address copilot review comments on v2.0 threat model - Fix malformed markdown table row for MCP SSE routes - Correct "writeable" to "writable" in tool description - Replace non-UUID threat IDs with valid UUID v4s for Threat Dragon compatibility - Remove stray .claude/worktrees/fervent-swirles submodule entry * Update threat model * docs(threat-model): address Copilot review feedback on v2.0 threat model - Normalize STRIDE type casing in JSON to "Information disclosure" (lowercase d) to match v1.9.1.json convention and avoid duplicate categories in Threat Dragon - Set isEncrypted: true on Model API Calls, Snapshot Download, and Async Query Results flows since their declared protocols (HTTPS, HTTPS/S3) imply TLS - Bump threatTop from 49 to 74 to match the highest threat number in the model so Threat Dragon does not reuse existing numbers when adding new threats - Expand Spice Runtime process description to mention OIDC/JWT external auth and identity-aware SQL functions (central to Enterprise v2.0) - Refine "No persistent audit logs" threat description to acknowledge that Enterprise v2.0 callers may be authenticated and that task history is the current observability surface but is not persistent - In v2.0.0.md, normalize "Info Disclosure" -> "Information Disclosure" in three STRIDE tables for consistency with the rest of the document - Clarify DDL vs DML in the Overview ("DDL (CREATE/DROP TABLE) and DML (INSERT/UPDATE/DELETE) support") so the parenthetical examples match the category they describe * docs(threat-model): fold post-2026-05-08 trunk changes into v2.0.0 Updates the Enterprise v2.0.0 threat model for security-relevant runtime changes that landed in trunk after the previous threat-model commit but before v2.0.0 final. Changes in the model: - Overview and Spice Runtime process description: note expanded DML backends (Cayenne, Snowflake, Postgres, DynamoDB, Sharepoint, plus DuckLake INSERT on read_write catalogs), public-listener mTLS for HTTP and Flight, on-disk TLS cert hot-reload, and Prometheus `/metrics` rate limiting. - Port Layout: 50051 and 8090 now show `runtime.tls.client_auth_mode` (`none` / `request` / `required`); 9090 notes `/metrics` rate-limit (100/min default) with `/health` exempt; 50052 notes cert/CA hot-reload via on-disk watcher. - DDL and DML Enforcement: split DDL-capable backends (Iceberg, Cayenne) from DML-capable backends, and add a defense-in-depth note about per-backend writer safety (quoted identifiers, escaped literals, schema/finite-float checks, affected-row verification, transactional INSERT/OVERWRITE). - Security Hardening Mitigations: add public-listener mTLS, TLS cert hot-reload, per-backend DML writer safety, and `/metrics` HTTP rate limiting as new implemented mitigations. - runtime-auth section: add a bullet describing mTLS as an alternative caller-identity source independent of `runtime.auth`. - New threat #75: public mTLS `request` mode admits anonymous callers while presented certs are promoted to a principal (Medium / Spoofing). Added to the "New Threats in v2.0" table and the runtime-auth threats table. Added in the JSON on the Spice Runtime process with a fresh UUIDv4 and `threatTop` bumped to 75. * docs(threat-model): correct claims to match trunk implementation After verifying claims against trunk source, three corrections: - Drop Sharepoint from the DML-capable backend list. Sharepoint is an object-store connector with file-level put/delete via `PutOptions`/`PutMode`, not a SQL DML target. Moved to the data-connector list alongside SMB/NFS in the overview. - Rename the metrics rate-limit field from `cluster_metrics_limit` (PR description name) to `metrics_endpoint_limit` (actual field name in `RateLimits` on trunk). - Refine threat #75 and the runtime-auth section to reflect the real channel-vs-identity behavior. The verified peer cert is only promoted to the auth principal when `IdentitySource::Channel` is derived (runtime.auth unset + mTLS request/required); with runtime.auth set, mTLS is channel-only and the cert is recorded as ChannelIdentity for audit while the principal still comes from API key / OIDC. - Replace the non-public `ReloadableClientVerifier` name in the hot-reload mitigation with the actual public surface (`ReloadableServerCerts` plus `ClusterPkiBundle` for cluster mTLS), and add the 250 ms debounce, atomic-rename re-arming behavior, and `runtime_tls_reload_total{result="parse_error"}` failure-mode signal that the implementation actually exposes. * docs(threat-model): re-scope v2.0.0 model to OSS surface This threat model lives in the OSS repository and describes the OSS runtime, not any enterprise distribution. Previous revisions incorrectly labeled it Enterprise and folded in features that do not exist in OSS. Verified against trunk: runtime-auth/src/lib.rs exposes only api_key and mtls (ChannelIdentity, IdentitySource, MtlsPrincipal); spicepod runtime.Auth has only the api_key field; there is no JwtVerifier, OidcConfig, JWKS handling, or current_user_id/current_org_id/ current_role/session_property SQL function in the OSS code. Re-scoping changes: - Title/description: drop "Enterprise" framing; this is the Spice.ai OSS v2.0.0 threat model. - Threat Actors: drop "Authenticated Enterprise Caller" and "Compromised Bearer Token Holder"; clarify "Malicious Client" can hold an API key or a verified mTLS client cert. - Assets: collapse OIDC-claim assets into a single "API keys and TLS client-cert material" item. - Threat Surface / Port Layout: drop OIDC bearer-token mentions; ports 50051 and 8090 now describe auth as "runtime.auth API key and/or verified client cert" only. - Vectors: drop "OIDC/JWT Validation Failure"; reword "Authentication Bypass" to runtime.auth + public mTLS; add an explicit "mTLS Mode Confusion" vector. - Trust Assumptions: drop IdP-configuration mention; rework auth boundary text to API key + public mTLS only. - New Threats: drop #57 (OIDC/JWT validation bypass) and #74 (JWKS retrieval/cache); rewrite #59 (protocol auth drift) to drop OIDC; rewrite #69 to cover API keys and verified-cert principal data rather than OIDC bearer tokens; keep #58 and #75. - Updated Threats from v1.9.1 (#28): replace OIDC claim/token leakage with API-key leakage via captured context. - Mitigations: drop OIDC/JWT validation controls, API-key + OIDC composite auth, and identity-aware SQL functions. - AI/Tool Surface: list /v1/mcp alongside /v1/tools/* as auth-gated routes via require_auth_configured in crates/runtime/src/http/ routes.rs (addresses Copilot review threads on the /v1/mcp gap); /health and /v1/ready remain the only unauthenticated probe routes. - runtime-auth Layer: simplify section to OSS reality — the only runtime.auth provider is api_key (ReadOnly/ReadWrite), with Databricks header passthrough as the only narrow exception, plus public mTLS as an alternative caller-identity source under the IdentitySource derivation rules. - Outbound Telemetry: drop OIDC claim references; replace #69 entry with the API-key + verified-cert principal version. - Inherited mitigations (#23, #29) reworded to say runtime.auth + mTLS instead of mTLS/JWT, so they don't imply Spice itself supports JWT. * docs(threat-model): address Copilot review on OSS re-scope - Add GetTaskHistory to the ClusterService RPC list in both the MD Distributed Query Architecture bullet and the JSON Cluster Control Plane flow description. The proto (`crates/runtime-proto/proto/ spice.proto:25`) defines GetTaskHistory alongside GetAppDefinition, ExpandSecret, GetSchedulers, GetMetrics, ControlStream, and AllocateInitialPartitions. The MD now points readers at the proto file for the canonical surface. - Replace "MCP SSE" with "/v1/mcp (MCP over Streamable HTTP)" in the AI/Tool Endpoint Surface vector. The SSE transport was removed upstream in favor of Streamable HTTP, which is consistent with the route inventory below. --------- Co-authored-by: Phillip LeBlanc <879445+phillipleblanc@users.noreply.github.com> Co-authored-by: claudespice <claude@spice.ai>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
See Commits and Changes for more details.
Created by
pull[bot] (v2.0.0-alpha.1)
Can you help keep this open source service alive? 💖 Please sponsor : )