| name | http-server |
|---|---|
| description | Run or point an agent at a sparq SPARQL 1.1 Protocol HTTP endpoint (sparq-server) — /sparql query+update over GET/POST (plus the query-only HTTP QUERY method, w3c/sparql-protocol#40, for Oxigraph interop), content negotiation (SELECT/ASK JSON/XML/CSV/TSV; CONSTRUCT/DESCRIBE + Graph Store N-Triples/prefix-Turtle/RDF-XML/JSON-LD — JSON-LD via the default-on jsonld feature), Graph Store read AND write (PUT/POST/DELETE/PATCH on graph resources, RDF/XML + default-on JSON-LD bodies accepted, atomic SPARQL-Update + opt-in Solid N3-Patch on PATCH), EXPLAIN, Prometheus /metrics, WebSocket + SSE subscriptions, opt-in grouped facet counts and prefix completion, and generation-pinned snapshot reads (a Sparq-Generation header + ?generation=N pin in the DEFAULT build, bounded to the ring's concurrency-retention window; opt-in time-travel widens it). Use when starting the server, querying/updating a running endpoint, choosing Accept/Content-Type, or embedding the axum router. |
sparq-server is a W3C-conformant HTTP server (axum/tokio) that exposes the sparq
query engine over a sparq_core::Graph — in-memory by default, or durable on disk
with --persist DIR (updates WAL-fsync'd, survive a restart with no rebuild; see the
"Durability" gotcha). It implements the SPARQL 1.1 Protocol (query + update at
/sparql) and the Graph Store HTTP Protocol (read + write), with Accept-driven
content negotiation, hardening guards, Prometheus /metrics, WebSocket + SSE subscriptions,
and opt-in grouped facet counts, prefix completion, time-travel, and GeoSPARQL.
Build with the default-off response-compression feature to negotiate gzip or zstd for outbound
result bodies from Accept-Encoding. Compression is transport-transparent and streaming; SSE
subscription responses and bodies that already have a Content-Encoding are left uncompressed:
cargo run -p sparq-server --features response-compression
curl --compressed -G http://127.0.0.1:3030/sparql \
--data-urlencode 'query=SELECT * WHERE { ?s ?p ?o }'Run the binary (server stack is the default-on server feature):
# serve a Turtle file on the default address 127.0.0.1:3030 (loopback — safe default)
cargo run -p sparq-server -- --format turtle data.ttl
# no data file => empty default graph (still answers queries, just no rows)
cargo run -p sparq-server
# custom bind addr / format (turtle | ntriples | nquads | trig). A NON-loopback bind
# (e.g. 0.0.0.0) is REFUSED unless --allow-remote / SPARQ_ALLOW_REMOTE=1 (no auth — see below).
cargo run -p sparq-server -- --addr 0.0.0.0:8080 --allow-remote --format ntriples data.ntOpt-in HTTP/2 switches the TCP listener from its default HTTP/1.1-only builder to the
hyper-util h1+h2 auto builder. With no certificate flags it accepts HTTP/1.1 and cleartext h2c;
supplying both PEM paths enables TLS 1.3 with ALPN h2,http/1.1:
cargo run -p sparq-server --features http2 -- data.ttl \
--tls-cert ./cert.pem --tls-key ./key.pemThe PEM flags are a pair: supplying only one fails startup. Omitting both preserves plain HTTP.
The HTTP/1 slow-loris header deadline, body idle deadline, peer ConnectInfo, graceful drain, and
WebSocket upgrade remain on the same bespoke serve path; HTTP/2 uses the same router and request
middleware. [GPT-5.6] sq-oprna.6
Opt-in HTTP/3 adds an encrypted QUIC/UDP listener beside the unchanged plain-HTTP TCP
listener. The two listeners dispatch through the same router; the UDP address defaults to
--addr and can be overridden independently:
cargo run -p sparq-server --features http3 -- data.ttl \
--http3 --http3-addr 127.0.0.1:3443 \
--tls-cert ./cert.pem --tls-key ./key.pemBoth PEM paths are required when --http3 is set, and malformed or mismatched material
fails startup. QUIC is mandatorily encrypted. With http3 alone the TCP listener remains plain
HTTP for proxy/backward compatibility; with --features http2,http3 the same PEM pair also
secures TCP and negotiates h2/h1 through ALPN. The pre-1.0 h3 stack is contained behind the
default-off feature. /subscriptions WebSockets remain HTTP/1.1-only; clients must fall back because
HTTP/3 extended CONNECT is not implemented. Once the UDP endpoint has bound, every plain-HTTP TCP
response advertises its live port as Alt-Svc: h3=":<port>"; ma=86400; no header is emitted when
--http3 is absent or startup cannot bind the QUIC listener. [GPT-5.6] sq-oprna.4
Security: optional Bearer-token write gate; loopback by default. With no token configured, every endpoint is unauthenticated (the back-compat default). Set
--auth-token <TOKEN>(envSPARQ_AUTH_TOKEN) to requireAuthorization: Bearer <TOKEN>on every write (a SPARQL Update on/sparql—application/sparql-update, or aquery/updatebody that parses as an update — and the GSPPUT/POST/DELETEmethods); otherwise401withWWW-Authenticate: Bearer(constant-time compared; mirrors QLever's-a <token>). Add--auth-token-read(envSPARQ_AUTH_TOKEN_READ=1) to ALSO gate reads. The subscription transports — the/subscriptionsWebSocket and the/subscriptions/sseSSE stream (both a read surface) — are gated by--auth-token-readtoo (beadsq-cxk5, closing the prior read-auth bypass): the SSE GET takes theAuthorization: Bearerheader; the WS upgrade accepts that header OR (for browsers, which cannot set headers on a WS handshake) aSec-WebSocket-Protocol: bearer.<token>subprotocol. The PrometheusGET /metricsendpoint is gated by--auth-token-readas well (beadsq-9jrx): its gauges leak the live triple count and active-subscription count, so it is a read;/healthstays ungated for liveness probes. With no read gate, both are open (back-compatible). The server binds loopback by default and refuses a non-loopback bind (e.g.0.0.0.0) unless you set--allow-remote(envSPARQ_ALLOW_REMOTE=1) OR the whole surface is authenticated (--auth-tokenAND--auth-token-read) — a write-token alone still leaves reads open. Deliver the token over TLS (terminate it at a proxy). For real per-user authz, front it with a reverse proxy / API gateway (orsparq-solid). SPARQLSERVICEfederation is OFF in the default build (theservicecargo feature is off); aSERVICEclause then errors at execution. Build with--features serviceto enable it — and even then the server is default-DENY-all SERVICE: aSERVICE <iri>reaches nothing unless its host is on the egress allowlist (--service-allow/--service-allow-file/SPARQ_SERVICE_ALLOW; beadsq-4w18). This is an SSRF guard: aSERVICEclause turns attacker-controlled query text into an outbound request from the server host (worst case the169.254.169.254cloud-metadata IP). The allowlist is enforced before any socket is opened, on the resolved IP (DNS-rebinding-safe). See "SERVICE federation (egress allowlist)" below.Security response headers (always on, ASVS V14.4 / ASVS-G1; beads
sq-cmvh,sq-2bhm). Every response — success, streamed, error and auth-gated (401) alike — carries a hardening header set, stamped by amap_responselayer inharden():X-Content-Type-Options: nosniff,Content-Security-Policy: default-src 'none'; frame-ancestors 'none',X-Frame-Options: DENY, andReferrer-Policy: no-referrer. These suit a SPARQL data API (no HTML is rendered): the tightest CSP says the body loads/runs nothing, andframe-ancestors
X-Frame-Options: DENYsay it is never meant to be framed. Each header is only added when absent, so a custom handler can override one. Deliberately omitted:Strict-Transport- Security(the origin serves plain HTTP — HSTS belongs on the fronting TLS proxy);X-XSS-Protection(deprecated, superseded by CSP);Cross-Origin-*/Permissions-Policy(browser-app document policies, meaningless for a data API). CORS is OFF by default (noAccess-Control-*header — a cross-origin browser read is blocked, the safe posture) but is an opt-in first-party origin allowlist (sq-o7o0; see "CORS" below). No blanketCache-Control: no-storeis forced: results are uncached by default (noETag/Cache-Control: publicis ever set), so there is nothing to tighten, and a blanket value would wrongly override/health//metrics— but the sensitive auth-refusal (401fromunauthorized()) does carryCache-Control: no-storeso a shared cache never retains it (sq-2bhm).Error bodies carry a generic class, never internals (ASVS V7 / ASVS-G3; beads
sq-cz89,sq-j9zs, [OPUS-4.8]sq-kfel). Every error is the structured{"error":"<msg>"}envelope where<msg>is a STABLE generic category (malformed-query / auth / not-found / server-error) — never the caller's input, a loaded-RDF fragment, a server filesystem path, a secret, or aDebugof an internal type. The full detail goes to the server log undertarget: "sparq_server"(gated behind--verbose/RUST_LOG), not the response body. All sensitive error paths funnel through onesanitized_errorhelper; regression-guarded bytests/hardening.rs(no_echo_*+FORBIDDEN_INTERNALS) andtests/tpf.rs.
Point a client at it (the endpoint is /sparql):
# GET, query as URL param (URL-encoded); default result media is SPARQL-JSON
curl -G http://127.0.0.1:3030/sparql --data-urlencode 'query=SELECT * WHERE { ?s ?p ?o } LIMIT 5'
# POST direct: body IS the query
curl http://127.0.0.1:3030/sparql -H 'Content-Type: application/sparql-query' \
--data 'ASK { ?s ?p ?o }'
# HTTP QUERY method (sq-b3df9, w3c/sparql-protocol#40) — query-only, interoperates with
# Oxigraph: body IS the query under Content-Type application/sparql-query; graph params
# (default-graph-uri / named-graph-uri) ride the URL query string. Same downstream query
# execution + Accept negotiation as POST; an update= form or application/sparql-update body
# is rejected (400 / 415).
curl -X QUERY http://127.0.0.1:3030/sparql -H 'Content-Type: application/sparql-query' \
-H 'Accept: application/sparql-results+json' --data 'SELECT * WHERE { ?s ?p ?o } LIMIT 5'
# POST url-encoded form, negotiate CSV
curl http://127.0.0.1:3030/sparql -H 'Accept: text/csv' \
--data-urlencode 'query=SELECT * WHERE { ?s ?p ?o }'
# SPARQL Update -> 204 No Content (atomic; failure -> 400, no partial effect)
curl -i http://127.0.0.1:3030/sparql -H 'Content-Type: application/sparql-update' \
--data 'INSERT DATA { <http://ex/a> <http://ex/p> <http://ex/b> }'Embed the router in your own tokio app (library use):
use sparq_core::Graph;
use sparq_server::{router, AppState};
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
let graph = Graph::load_str(include_str!("data.ttl"), "turtle")?;
let app = router(AppState::new(graph)); // axum::Router
let listener = tokio::net::TcpListener::bind("127.0.0.1:3030").await?;
axum::serve(listener, app).await?;
Ok(())
}Library surface re-exported from sparq_server (behind the default server feature):
fn router(state: AppState) -> axum::Router— builds the hardened endpoint router (/sparql,/sparql/graph,/graphs/*path,/subscriptions,/subscriptions/sse,/health,/metrics, plus feature-gated routes such as/facetsand/complete). [OPUS-4.8] sq-bxog:/subscriptions/sseis the SSE transport. [GPT-5.6] sq-lsp7k.5.2:/facetsneeds thefacetsbuild feature and the runtimeServerConfig::facetsflag. [GPT-5.6] sq-lsp7k.9.3:/completesimilarly needs thecompletefeature andServerConfig::complete.AppState::new(graph: Graph) -> AppState— defaultServerConfig.AppState::with_config(graph: Graph, config: ServerConfig) -> AppState.AppState::current(&self) -> PinnedGen— lock-free pin of the current immutable generation snapshot (PinnedGen = Arc<sparq_serve::Generation<Graph>>; call.snapshot() -> &Graph,.number() -> u64,.published_at(),.epochs()).AppState::apply_update(&self, sparql: &str) -> Result<u64, String>— submit a SPARQL Update through the sequenced writer; blocks until the containing generation is published; returns that generation number (read-your-writes token). Call off the async workers (spawn_blocking). With adaptive group-commit (default;adaptive_commit), a serial interactive client commits in engine-time (µs) — the writer drains only the already-queued backlog and commits, instead of paying a fixed group-commit window; concurrent load still batches. FIFO order, per-update atomicity and durability are unchanged (sq-p7kk5).AppState::at(&self, number: u64) -> Option<PinnedGen>— pin a retained generation (time-travelfeature only; the HTTP?generation=Npin does NOT need this — it resolves against the ring's concurrency-retention window directly, so it works in the default build viasq-ci2d6).struct ServerConfig { query_timeout: Option<Duration>, update_where_timeout: Option<Duration>, adaptive_commit: bool, max_body_bytes: usize, max_concurrent: usize, header_read_timeout: Option<Duration>, body_read_timeout: Option<Duration>, max_results: Option<usize>, max_query_rows: Option<usize>, max_query_bytes: Option<usize>, max_decompress_ratio: usize, max_subscriptions: usize, max_subscriptions_per_conn: usize, verbose: bool, redact_logs: bool, allow_remote: bool, auth_token: Option<String>, auth_token_read: bool, service_allow: ServiceAllowlist, /* + time_travel_* / facets / complete under their features, + audit_log under audit-log feature */ }withServerConfig::default()andServerConfig::from_env(). (update_where_timeout= separate, typically-SHORTER writer-side WHERE deadline for SPARQL UPDATE that bounds writer-queue head-of-line blocking from a slow update —None= usequery_timeout,sq-nulp;adaptive_commit= adaptive group-commit (defaulttrue): a serial interactive client commits in engine-time (µs) rather than paying a fixed group-commit window; concurrent load still batches. Pure latency change — FIFO/atomicity/durability unchanged.false= always windowed.--no-adaptive-commit/SPARQ_ADAPTIVE_COMMIT=0—sq-p7kk5;max_query_rows= coarse memory cap;max_query_bytes= byte-accounted memory cap that also prices row WIDTH + computed-literal size —sq-s5is;max_decompress_ratio= zip-bomb guard —sq-ebii;header_read_timeout= slow-loris guard: max time a connection may take to send its complete request-header block, enforced at hyper's HTTP/1 connection layer bysparq_server::serve—Nonedisables; default 15s —sq-2gqr;body_read_timeout= slow-body guard (the complement toheader_read_timeout): an idle deadline between consecutive request-body reads, applied bysparq_server::servevia atower_httpRequestBodyTimeoutLayer/TimeoutBody(the timer resets after each chunk, so an honest large upload is not penalised — only a stall is) — closes the slow-body dribble a complete-header client could otherwise use;Nonedisables; default 30s —sq-lodb.)auth_token(set: gates the write surface with a Bearer token, constant-time compared;None: no write auth) andauth_token_read(gate reads too) are honoured by the libraryrouteritself — embedders get the gate for free (sq-zcby).fn bind_posture(addr: &SocketAddr, allow_remote: bool, auth: AuthPosture) -> BindPosture— the bind gate the binary applies:Loopback(proceed),RemoteAllowed { warning }(proceed + log), orRemoteRefused { message }(refuse).AuthPosture::{None, WriteOnly, ReadAndWrite}(viaAuthPosture::from_config(&config)) folds the token + read-gate into the decision: a non-loopback bind is allowed when--allow-remoteis set OR the surface is fully authenticated (ReadAndWrite); a write-token alone (WriteOnly) still requires--allow-remotebecause reads stay open. This is a bind-time posture gate; per-request auth is theauth_tokenfields above (enforced byrouter/harden-wrapped handlers).fn harden(routes: axum::Router, config: &ServerConfig) -> axum::Router— wrap any router in the production middleware (panic→500, concurrency-limit→429, body-limit→413, JSON error bodies, optional trace).async fn serve(listener: tokio::net::TcpListener, app: axum::Router, header_read_timeout: Option<Duration>, body_read_timeout: Option<Duration>, shutdown: impl Future<Output=()>) -> std::io::Result<()>— the accept + graceful-drain loop the binary runs instead ofaxum::serve(sq-2gqr/sq-lodb). It is a faithful port of axum's own loop (per-connection task, watch-channel drain,with_upgrades()so the/subscriptionsWebSocket still works) with two additions: (1) it installs ahyper_utilTokioTimer and hyper's HTTP/1header_read_timeout(the slow-loris HEADER guard); (2) it optionally wraps the request body in atower_httpRequestBodyTimeoutLayerkeyed onbody_read_timeout(the slow-BODY guard — applied viatower::util::option_layer, soNoneis a true no-opIdentitylayer). Why:axum::servenever installs a timer, so hyper's header-read deadline is inert there and a slow-loris client holds a connection (and aconcurrency_limitslot) open indefinitely; and even with that fixed, a complete-header client can still dribble the BODY to hold the slot —body_read_timeoutcloses that complementary hole. PassNoneon either to opt back out of that guard. With the default-offhttp2feature this same function uses hyper-util's h1+h2 auto builder (h2c on a plain listener) and retains the timer on the builder's HTTP/1 configuration.async fn serve_tls(listener: tokio::net::TcpListener, app: axum::Router, tls_config: Arc<rustls::ServerConfig>, header_read_timeout: Option<Duration>, body_read_timeout: Option<Duration>, shutdown: impl Future<Output=()>) -> std::io::Result<()>(http2feature only) — TLS counterpart toserve; the binary supplies a TLS-1.3 config advertisingh2,http/1.1via ALPN. It shares the auto-builder connection body, middleware, timeout hooks, peerConnectInfo, and graceful drain with the cleartext feature-on path. [GPT-5.6] sq-oprna.6- Re-exports for cache layers/tests:
PinnedGen,GLOBAL_POD: &str("urn:sparq:pod:global"), andsparq_serve::{Epoch, PodEpochs, PodId}. - Response-bytes result cache (opt-in,
sparq-serve'sresult-cachefeature, OFF by default — [OPUS-4.8] sq-jluc). A serving-layer cache from a request identity to the pre-serialized response body, distinct fromsparq-engine's in-engine algebra-keyedresult-cache. Public surface (feature-gated):sparq_serve::{ResultCache, CacheConfig, ScopeKey, ReadFootprint, LeaseOutcome, CacheStats}. Key = canonical-query × visibility-scope × per-pod epoch-vector (ResultCache::lease/get): the scope is the identity of the accessible graph set (ScopeKey::of_graphs(AuthIndex::accessible(session, mode))), never the WebID — bytes cached for one access scope can never be served to another (an access-control correctness invariant, not a privacy guarantee). Invalidation is per-pod epoch bumps (ReadFootprint::Pods) or the global generation (ReadFootprint::Unbounded); single-flight leases dedup a stampede; a byte-budget LRU + admission bound keeps it from caching oversize/streaming bodies. Wiring it into the axum endpoint and the canonical perf-validation are follow-ups (the perf targets need a canonical host). - Serializer/negotiation helpers (always compiled, no
serverfeature): modulesparq_server::negotiate—fn negotiate(accept: Option<&str>) -> Format(lenient, always yields a format),fn negotiate_or_406(accept: Option<&str>) -> Result<Format, NotAcceptable>(Oxigraph-parity strict — the query path uses this; a present-but-unsatisfiableAcceptisErr(NotAcceptable)→ 406),fn negotiate_graph(accept: Option<&str>) -> GraphFormatand its strict siblingfn negotiate_graph_or_406(...) -> Result<GraphFormat, NotAcceptable>; modulesparq_server::exec—fn prepare(&str) -> Result<Prepared, PrepareError>andfn prepare_with_dataset(&str, &DatasetOverride) -> Result<Prepared, PrepareError>(applies the SPARQL-Protocoldefault-graph-uri/named-graph-urioverride, sq-z33x),fn apply_update_dataset(&str, &UsingOverride) -> Result<String, UpdateDatasetError>(the update-sideusing-*override),enum QueryForm { Select, Ask, Construct, Describe }; modulesparq_server::results.
When the consumer is another Rust process (e.g. solid-server-rs) and wants to
drop the HTTP hop entirely, embed the engine in-process via the sparq_serve::embed
facade instead of running sparq-server and talking to it over HTTP. It is the
documented embedding seam: thin wrappers over the engine entry points plus a
re-export of the runtime-agnostic concurrency wrapper. No axum/tokio, no HTTP.
- Data path (over one
&Graph/&mut Graph):embed::query_json(&Graph, sparql) -> Result<String, _>(SPARQL-JSON),embed::query(&Graph, sparql) -> Result<QueryResult, _>,embed::ask(&Graph, sparql) -> Result<bool, _>,embed::update_in_place(&mut Graph, sparql)(+..._atomicfor all-or-nothing on one graph),embed::apply_delta_nquads(&mut Graph, inserts, deletes)(quad-level, per-graph; blank nodes by label), and the probesembed::exists(&Graph) -> bool,embed::named_graph_exists(&Graph, &Term) -> bool,embed::metadata(&Graph) -> Metadata { triples, named_graphs }.query_json_with_budgettakes aQueryBudget. - Concurrency wrapper (re-exported from the crate root):
GenerationRing/Generation/GraphApplier/Writer(+RingConfig,WriterConfig,PodId,TimeTravelConfig) — the SAMEfork → update → publish+ generation-pinning modelsparq-serverwraps behind its endpoint. A readerring.current()pins an immutable snapshot; the writer publishes new generations without blocking readers. - Stability — API tier-1 (proposed-stable): the proposed semver-stable embedding
surface in the API stability & deprecation policy, but
NOT yet frozen — the formal freeze is the maintainer's to ratify on #1248 / #1346
(pre-
1.0minor releases MAY still change it). Pin tosparq_serve::embedrather than reaching intosparq-core/sparq-enginedirectly so the freeze, when ratified, has one well-defined shape.
1. Query forms and result negotiation. Default result media is SPARQL-JSON. Set
Accept to choose (q-value aware, defaults to JSON for SELECT/ASK, N-Triples for
CONSTRUCT/DESCRIBE). An absent / empty / */* Accept gets
that default; a present Accept that names only unsupported media types and no wildcard is
406 Not Acceptable (Oxigraph parity, w3c/sparql-protocol#40) — sparq no longer silently
falls back to JSON in that case (the EXPLAIN Accept: text/x-sparq-explain short-circuits this
and the Graph-Store-Protocol read path keeps its lenient default):
| Query form | Accept | Content-Type returned |
|---|---|---|
| SELECT | application/sparql-results+json (default) / +xml / text/csv / text/tab-separated-values |
matching results media |
| ASK | json (default) / xml | application/sparql-results+json / +xml |
| CONSTRUCT / DESCRIBE | application/n-triples (default) / text/turtle / application/rdf+xml / application/ld+json (the jsonld feature — default-on) |
matching RDF media; N-Triples, prefix-compacting Turtle, RDF/XML, or flattened JSON-LD |
Per the W3C SPARQL Results TSV format, the TSV serialiser abbreviates an
xsd:integer / xsd:decimal / xsd:double / xsd:boolean literal whose lexical form is
a valid Turtle token to its bare token (no quotes, no ^^datatype) — e.g. 30, 2.2,
1.0E6, true; everything else (incl. integer/decimal subtypes like xsd:negativeInteger
and custom datatypes) stays quoted + typed, and xsd:string keeps the implicit-datatype short
form. The bare token is the literal's own lexical form, so a data-sourced literal
round-trips its original spelling ("1.0E6"^^xsd:double → 1.0E6, not canonicalised);
computed numerics arrive already in the engine's canonical form, so they serialise
canonically. CSV writes each value's bare lexical string (datatype/lang dropped — lossy by
spec); JSON/XML carry the full term (value + datatype) unchanged.
SELECT-JSON streams (TTFB). The application/sparql-results+json SELECT path streams its
body: the engine serialises on a blocking worker feeding a bounded channel, so the results
header + early solutions are written to the socket before the whole result is serialised
(first byte before full materialisation, rather than after it). The streamed body is
byte-identical to the single materialised JSON string (same solution order, same escaping) —
query_json_stream_with_budget shares one emit core with the buffered serialiser. Two response
shapes: a single-chunk (small, ≤ one 64 KiB chunk) result is returned buffered with a
Content-Length; a multi-chunk result streams under chunked transfer-encoding with no
Content-Length (the total is unknown until the last row). Note the streaming scope: a
DISTINCT / join / ORDER-BY SELECT must fully evaluate before any solution exists, so its first
byte still lands after the join materialises (the header is flushed the instant that finishes,
before serialisation) — the earliest-first-byte win is largest on scan-shaped and
below-parallel-threshold results; XML/CSV/TSV stay buffered. Error mid-stream (honest
contract): the status is chosen from the first chunk, so a failure detected before any byte
(parse error, or a row/byte cap / deadline the engine confirms before the header) still returns
the correct 400 / 413 / 503. But once the header has been flushed for a genuinely
multi-chunk result, the HTTP status is committed: a later cap/deadline trip can only truncate
the body — a streamed 200 cannot retroactively become a 413/503.
Truncation safety (the invariant). A client MUST NOT be able to mistake a truncated stream for a complete result. Two mechanisms enforce it, the second strictly on top of the first:
- The document is never closed on a truncation. A complete
sparql-results+jsonbody ends with]}}. The server WITHHOLDS the document-closing chunk until the engine confirms it produced the whole result, and DROPS it on any mid-stream abort — so a truncated body is not valid JSON and any conformant parser errors instead of silently accepting a short result. This is the floor guarantee and it needs no client opt-in. - The reason is reported out of band. A client that sends
TE: trailersgets aTrailerresponse header plus, after the last data chunk, eitherX-Sparq-Complete: trueorX-Sparq-Truncated: <reason>where the reason isdeadline|max-rows|max-bytes|cancelled|panic|error— the same classification the pre-first-byte path maps to503/413. A client that did NOT negotiate trailers instead sees the chunked stream abort without its terminating zero-length chunk (a transport error), since hyper would drop a trailers frame for it anyway.
A worker that dies mid-result (an engine panic) is treated as a truncation, not a completion:
the completeness claim is only ever made when the engine itself returned success. A
well-formed, correctly-terminated SHORT 200 is a forbidden outcome — --max-results remains
an honest refusal, never a silent truncation, and hitting it mid-stream drops the closing ]}}
rather than emitting a clean short document.
Single parse per request (HTTP floor). The read path parses each request query with
spargebra exactly ONCE — the server parses to classify the form + apply any protocol dataset
override, then hands the resulting algebra straight to the engine's *_prepared entry points
(query_json_stream_prepared_with_budget for JSON SELECT, ask_prepared_with_budget for ASK),
so the engine does not re-parse the query string. This shaves one full parse off the per-request
floor for ARBITRARY novel queries (no query cache — parsing is still paid, just not twice);
negligible on a trivial ASK{} (sub-µs), and larger on realistic multi-pattern queries.
curl -G http://127.0.0.1:3030/sparql -H 'Accept: application/sparql-results+xml' \
--data-urlencode 'query=SELECT ?s WHERE { ?s ?p ?o }'
# prefix-compacting Turtle:
curl -G http://127.0.0.1:3030/sparql -H 'Accept: text/turtle' \
--data-urlencode 'query=CONSTRUCT { ?s ?p ?o } WHERE { ?s ?p ?o }'
# RDF/XML:
curl -G http://127.0.0.1:3030/sparql -H 'Accept: application/rdf+xml' \
--data-urlencode 'query=CONSTRUCT { ?s ?p ?o } WHERE { ?s ?p ?o }'
# JSON-LD (flattened) — default-on, works on the standard build: [OPUS-4.8] sq-oy1f.1/.4
curl -G http://127.0.0.1:3030/sparql -H 'Accept: application/ld+json' \
--data-urlencode 'query=CONSTRUCT { ?s ?p ?o } WHERE { ?s ?p ?o }'HTTP
QUERYmethod (sq-b3df9, w3c/sparql-protocol#40, epic sq-my8wd) — query-only, Oxigraph-interop./sparqlaccepts the registered HTTPQUERYverb in addition to GET/POST. It is a new INPUT path that feeds the SAME query execution + Accept negotiation as POST, so all the result-media rows above apply unchanged. Mirroring Oxigraph (("/sparql", "POST" | "QUERY"),let is_query = method == "QUERY"):
- Send the query as the raw
application/sparql-querybody (preferred), OR as thequery=value of anapplication/x-www-form-urlencodedbody. No other Content-Type is accepted (else 415); missing Content-Type is 415.- Dataset graphs (
default-graph-uri/named-graph-uri/union-default-graph) ride the URL query string, never the body — even when the query is the raw body.- Query-only: an
update=form field is a 400 and anapplication/sparql-updatebody is a 415 (it is not a valid operation under QUERY). Use POST for an update.- It enforces the SAME auth / egress gates as GET/POST (a QUERY is a read — gated by
--auth-token-read). It does not emitCache-Control/Content-Locationheaders (Oxigraph does not either). [OPUS-4.8]curl -X QUERY 'http://127.0.0.1:3030/sparql?default-graph-uri=http://ex/g1' \ -H 'Content-Type: application/sparql-query' \ -H 'Accept: text/csv' \ --data 'SELECT * WHERE { ?s ?p ?o }'
SPARQL 1.1 Protocol conformance lane (sq-jaj38, epic sq-my8wd) — what is ratcheted. The protocol surface above is now covered by a dedicated W3C SPARQL 1.1 Protocol (HTTP) conformance suite in
sparq-conformance(opt-inhttp-protocolfeature, OFF by default; it reuses the in-process loopback server —sparq_server::serveon an ephemeral127.0.0.1:0port — and drives RAW HTTP at the bound port, NOT the federated SERVICE transport, so it does not touch the engine's egress allowlist). It ratchets a MEASURED PASS floor over: query via GET / POST-urlencoded / POST-direct, theQUERYmethod, update via POST, thedefault-graph-uri/named-graph-urioverrides, SELECT/ASK negotiation (SRJ / SRX / CSV / TSV), a present-but-unsatisfiableAccept→ 406 Not Acceptable (Oxigraph parity, w3c/sparql-protocol#40 — formerly a divergence, now a genuine PASS that raised the floor 20→21), and the 200 / 400 / 405 (withAllow) / 406 / 415 status codes. Honest boundary: two behaviours remain DOCUMENTED DIVERGENCES (reported separately, NOT summed into the floor, so they never inflate the conformance number): an absent /*/*Acceptdefaults to SPARQL-results JSON (a W3C-permitted default representation — only a present-but-unsatisfiableAcceptis a 406), and an ASK withAccept: text/csvfalls back to a JSON boolean (CSV/TSV have no boolean serialisation). Run it withcargo test -p sparq-conformance --features http-protocol --test http_protocol_suite; the row is in the central scoreboard (W3C SPARQL 1.1 Protocol (HTTP)). [OPUS-4.8]
Service Description + Graph Store Protocol conformance lane (sq-1uuxz, epic sq-my8wd) — what is ratcheted. The federation-descriptor + GSP write surfaces (see "Federation discovery" and the Graph-Store-Protocol section below) are covered by a dedicated SD + GSP conformance suite in
sparq-conformance(opt-infederation-descriptorsfeature, OFF by default; it reuses the in-process loopback server —sparq_server::serveon an ephemeral127.0.0.1:0port, stood up with the server'sfederation-descriptorsruntime flag ON — and drives RAW HTTP at the bound port, so it does not touch the engine's egress allowlist). It ratchets a MEASURED PASS floor over (A) the Service Description (GET /sparqlwith no query): thesd:Serviceadvertises exactly the result/input formats (SRJ/SRX/CSV/TSV + Turtle/N-Triples/RDF-XML), the query/update languages, the SPARQL versions (sd:supportedVersion1.0/1.1/1.2) andsd:BasicFederatedQuerythat the server GENUINELY implements — no over-advertising (each advertised result format is cross-checked against a real SELECT request, and JSON-LD — not served in this build — must NOT appear) — and (B) the Graph Store Protocol: a GET/PUT/POST/DELETE round-trip on a named graph (indirect?graph=<iri>+ direct/graphs/<path>) and the default graph (?default), VERIFYING store state after every op (PUT→GET-back-equal; PUT replaces; POST merges; DELETE removes; 200/201/204/400/404/405 (withAllow)/415). Honest boundary: a GSP read of an ABSENT named graph is 200 + empty graph, NOT 404 (GSP-permitted — sparq treats an empty graph as existing); this is a DOCUMENTED DIVERGENCE, reported separately and NOT summed into the floor. Run it withcargo test -p sparq-conformance --features federation-descriptors --test sd_gsp_suite; the row is in the central scoreboard (SPARQL 1.1 Service Description + Graph Store Protocol). [OPUS-4.8]
JSON-LD content negotiation (
jsonldfeature — default-on, [OPUS-4.8] sq-oy1f.4). The server speaksapplication/ld+jsonout of the box (thejsonldfeature is in the default set — a maintainer-directed exception to opt-in-by-default).application/ld+jsonjoins the q-value-aware RDF negotiation in BOTH directions: a CONSTRUCT/DESCRIBE or a Graph-Store-Protocol read withAccept: application/ld+jsonis served as the engine's flattened JSON-LD (a{"@graph": […]}node-object document thattoRdf-round-trips to the same graph), and a GSPPUT/POSTwithContent-Type: application/ld+jsonis parsed into the store via the engine's JSON-LD parser. Toggleable off via--no-default-features --features server: thenapplication/ld+jsonis unrecognised — anAcceptfor it falls back to a supported graph format (never a 406 — the endpoint always has a representation), and a write body in it is a plain415— byte-identical to a JSON-LD-disabled build. What is default-on now: JSON-LD parse + serialise (flattened) + content-negotiation; full conneg-conformance ratcheting is on the sq-oy1f roadmap.
Default-on algebra rewrite (
algebra-rewritefeature — [FABLE-5] sq-7d3dj.30.13). The server's default set also lights sparq-engine's pre-execution algebra rewrite pass (#1735): a result-equivalentFILTER(?v = <iri>)IRI-constant folding +FILTER(!bound)anti-join applied at parse time, so the shipped server executes the same plans the CLI and the canonical benchmarks measure. IRI constants only (a literal equality is never rewritten — thesq-lr2iiavoidance contract); zero new dependencies. Drop it with--no-default-features --features server,jsonldfor an explicitly rewrite-dark build; the sparq-engine LIBRARY default remains OFF for lean library consumers.
Default-on DPccp join-order planner (
dp-plannerfeature — [OPUS-5] sq-7d3dj.30.15). The server's default set also lights sparq-engine's DPccp planner (sq-7d3dj.30.5): a connected BGP with 3 or more patterns that fits the connected-subgraph budget is planned as a cost-optimal BUSHY join tree instead of by greedy GOO. It is DEFAULT-ON once compiled, so every request gets it with no explicit install, and it is result-equivalent — only join ORDER changes, never the answer. This closes the gap where sq-7d3dj.30.5 lit the planner insparq-clialone, which made an HTTP-measured query plan differently from the CLI-measured one the canonical benchmarks use. Zero new dependencies. Drop it with--no-default-features --features server,jsonldfor an explicitly greedy-GOO build; the sparq-engine LIBRARY default remains OFF for lean library consumers.
2. EXPLAIN a query plan (no execution) or analyze (execute + per-operator trace).
text/plain response. Use explain / explain=plan (or Accept: text/x-sparq-explain)
for the dry run, explain=analyze to run + trace (SELECT/ASK only):
curl -G 'http://127.0.0.1:3030/sparql?explain=true' \
--data-urlencode 'query=SELECT * WHERE { ?a <http://ex/knows> ?b . ?b <http://ex/age> ?age }'Structured plan tree (explain-json feature, default-on for the server binary —
sq-ixc3.19). Accept: application/x-sparq-explain+json answers the SAME explain /
explain=analyze surface with the typed plan tree (explain_json::PlanNode) as camelCase
JSON — per operator {"operator", "estimated", "actual", "nanos", "qError", "children"}
(the sq-jbqh4 schema contract the GUI plan explorer renders; qError =
max(est/actual, actual/est)). Like the text CT, the JSON Accept alone requests a
plan-only explain. A --no-default-features --features server,… build answers it 406
(the text explain still works) so clients degrade explicitly, never mis-parse text as JSON:
curl -G -H 'Accept: application/x-sparq-explain+json' \
'http://127.0.0.1:3030/sparql?explain=analyze' \
--data-urlencode 'query=SELECT * WHERE { ?a <http://ex/knows> ?b . ?b <http://ex/age> ?age }'3. Generation-pinned snapshot reads (DEFAULT build — sq-ci2d6). Every /sparql response
carries the Sparq-Generation header (the generation the response was produced against; an
update's 204 carries the generation containing the write — the read-your-writes token).
Capture it and pin a later read with ?generation=N (URL param or url-encoded body — body wins)
to get an immutable snapshot of the store as of that generation. This is available with no
feature enabled: the pin is bounded to the generation ring's concurrency-retention window —
the last K = 4 generations older than current are always kept (sparq_serve::ring::DEFAULT_RETAIN;
RingConfig::retain). That is exactly the window that makes multi-request LIMIT/OFFSET
pagination snapshot-consistent: pin one generation, page it, and the pages union to the
single-shot result at that generation even if writes interleave (with a deterministic order —
add ORDER BY, or rely on the stable order over the immutable snapshot):
# read-your-writes: capture the generation an update lands in (default build — no feature)
G=$(curl -si http://127.0.0.1:3030/sparql -H 'content-type: application/sparql-update' \
--data 'INSERT DATA { <http://ex/a> <http://ex/p> <http://ex/b> }' \
| grep -i sparq-generation | tr -d '\r' | awk '{print $2}')
# page a large read pinned to that generation — page 1, then page 2, still consistent
# even if another client writes between the two requests:
curl -G http://127.0.0.1:3030/sparql --data-urlencode "generation=$G" \
--data-urlencode 'query=SELECT ?child WHERE { <c> <member> ?child } ORDER BY ?child LIMIT 1000 OFFSET 0'
curl -G http://127.0.0.1:3030/sparql --data-urlencode "generation=$G" \
--data-urlencode 'query=SELECT ?child WHERE { <c> <member> ?child } ORDER BY ?child LIMIT 1000 OFFSET 1000'Honest boundary (the retention contract). The default window is bounded by the ring's
concurrency retention (K = 4), not a durable history: a pin more than K generations behind
current has aged out → 410 Gone (the snapshot is gone; re-read the current generation and
restart pagination — the server never silently substitutes a different generation). If a
pagination sweep must survive more than K interleaved writes, build with the opt-in time-travel
feature, which EXTENDS retention (--time-travel-generations N, --time-travel-max-age SECS) so
?generation=N reaches far older snapshots — same header/pin mechanics, wider window:
cargo run -p sparq-server --features time-travel -- data.ttl --time-travel-generations 32Status: aged-out generation → 410 Gone; never-published / unparsable / pinning an
update (an update always applies to current) → 400. All identical in both feature states —
time-travel only changes how far back a pin can reach.
4. WebSocket subscriptions (SEPA-style live SELECT). Connect to
ws://127.0.0.1:3030/subscriptions, send subscribe, get subscribed + an initial
notification (sequence 0 = full result as addedResults), then added/removed bindings
diffs after each committed update that changes the result:
client: {"subscribe": {"query": "SELECT ?s WHERE { ?s <http://ex/age> ?o }", "alias": "ages"}}
server: {"subscribed": {"id": 1, "alias": "ages"}}
server: {"notification": {"id": 1, "sequence": 0, "addedResults": {…full result…},
"removedResults": {"head":{"vars":["s"]},"results":{"bindings":[]}}}}
…POST /sparql update commits…
server: {"notification": {"id": 1, "sequence": 1, "addedResults": {…}, "removedResults": {…}}}
client: {"unsubscribe": {"id": 1}}
server: {"unsubscribed": {"id": 1}}
addedResults/removedResults are each full SPARQL-JSON results objects. Refusals and
failed re-evaluations come back as {"error": {"message": …, "id"?: n}}.
[OPUS-4.8] sq-cxk5: when --auth-token-read is set, the upgrade is gated behind the read
token (401 before upgrade otherwise). A non-browser client sends Authorization: Bearer <TOKEN> on the handshake; a browser (which cannot set WS handshake headers) passes it as
a subprotocol: new WebSocket("ws://host/subscriptions", ["bearer." + token]). The server
takes the substring after the bearer. prefix as the token, validates it (constant-time),
and echoes the subprotocol back per RFC 6455. With no read token configured, the upgrade is
open (back-compatible).
4b. SSE subscriptions (text/event-stream). [OPUS-4.8] sq-bxog: the SAME subscription
engine over Server-Sent Events, for clients that prefer a plain HTTP GET stream to a
WebSocket. GET /subscriptions/sse?query=<SELECT>[&alias=<x>] opens one subscription per
stream (the query is in the query string — SSE is one-way, so there is no subscribe/
unsubscribe frame; close the stream to unsubscribe). The events carry the SAME JSON as
the WS path — only the framing differs (event: / data: / id: lines, blank-line
terminated, : ping keep-alive comments hold idle connections open). The SSE id: mirrors
the per-subscription sequence.
curl -N 'http://127.0.0.1:3030/subscriptions/sse?query=SELECT%20?s%20WHERE%20{%20?s%20%3Chttp://ex/age%3E%20?o%20}&alias=ages'event: subscribed
data: {"subscribed":{"id":1,"alias":"ages"}}
event: notification
id: 0
data: {"notification":{"id":1,"sequence":0,"alias":"ages","addedResults":{…full result…},"removedResults":{…empty…}}}
…POST /sparql update commits…
event: notification
id: 1
data: {"notification":{"id":1,"sequence":1,"alias":"ages","addedResults":{…},"removedResults":{…}}}
[OPUS-4.8] sq-cxk5: like the WS path, when --auth-token-read is set this GET is gated
behind the read token via the Authorization: Bearer <TOKEN> header (it is a plain GET, so
the header is the only auth channel — no WS subprotocol) — 401 before the stream opens
otherwise. A registration refusal (missing/non-SELECT/malformed query → 400; capacity/budget →
503) is returned as a normal {"error":"…"} JSON HTTP response BEFORE the stream opens —
SSE cannot set a status once the stream is flowing. A later re-evaluation failure ends the
stream with a final event: error frame. Both transports share one registry + change
source, so the per-conn / server-wide subscription caps and the sparq_active_subscriptions
gauge count SSE streams and WS subscriptions together.
5. Graph Store read + write + operational endpoints.
# READ (GET/HEAD): serialises the addressed graph in the Accept-negotiated RDF syntax
# (default N-Triples; also text/turtle = prefix-compacting Turtle, application/rdf+xml = RDF/XML,
# and application/ld+json = flattened JSON-LD with `--features jsonld` [OPUS-4.8] sq-oy1f.1)
curl http://127.0.0.1:3030/sparql/graph?default # GSP indirect (default graph)
curl 'http://127.0.0.1:3030/sparql/graph?graph=http://ex/g' # GSP indirect (named graph)
curl http://127.0.0.1:3030/graphs/whatever # GSP direct (request URI is the graph IRI)
curl -H 'Accept: application/rdf+xml' http://127.0.0.1:3030/sparql/graph?default # RDF/XML read [OPUS-4.8] sq-rt6v
curl -H 'Accept: application/ld+json' http://127.0.0.1:3030/sparql/graph?default # JSON-LD read (--features jsonld) [OPUS-4.8] sq-oy1f.1
# WRITE (sq-gxsj): body is RDF, format by Content-Type
# (turtle | n-triples | n-quads | trig | application/rdf+xml [OPUS-4.8] sq-rt6v
# | application/ld+json with `--features jsonld` [OPUS-4.8] sq-oy1f.1)
# PUT = REPLACE graph contents (201 if created, 204 if replaced):
curl -X PUT 'http://127.0.0.1:3030/sparql/graph?graph=http://ex/g' \
-H 'content-type: text/turtle' --data '<http://ex/s> <http://ex/p> <http://ex/o> .'
# POST = MERGE (additive); selector-less POST to /sparql/graph creates a fresh server-named graph:
curl -X POST 'http://127.0.0.1:3030/sparql/graph?graph=http://ex/g' \
-H 'content-type: application/n-triples' --data '<http://ex/s2> <http://ex/p> <http://ex/o2> .'
# DELETE = DROP the graph (204; 404 if a named graph is absent; ?default → CLEAR DEFAULT, always 204):
curl -X DELETE 'http://127.0.0.1:3030/sparql/graph?graph=http://ex/g'
# PATCH = atomic, graph-scoped in-place MODIFY (sq-hj4n; 204 on success). Two body dialects:
# 1) application/sparql-update (ALWAYS-ON) — the body IS a SPARQL Update, applied atomically
# through the same writer, with its WHERE dataset scoped to the addressed graph:
curl -X PATCH 'http://127.0.0.1:3030/sparql/graph?graph=http://ex/g' \
-H 'content-type: application/sparql-update' \
--data 'DELETE DATA { GRAPH <http://ex/g> { <http://ex/s> <http://ex/p> <http://ex/o> } } ;
INSERT DATA { GRAPH <http://ex/g> { <http://ex/s> <http://ex/p> <http://ex/o2> } }'
# 2) text/n3 (Solid N3-Patch, OPT-IN: needs the `n3-patch` build feature AND --n3-patch / SPARQ_N3_PATCH=1;
# else 415). A solid:InsertDeletePatch translated into ONE atomic graph-scoped DELETE/INSERT … WHERE:
curl -X PATCH 'http://127.0.0.1:3030/sparql/graph?default' \
-H 'content-type: text/n3' --data '@prefix solid: <http://www.w3.org/ns/solid/terms#>.
@prefix ex: <http://ex/>.
_:p a solid:InsertDeletePatch;
solid:where { ?s ex:name "Alice" . };
solid:deletes { ?s ex:age 30 . };
solid:inserts { ?s ex:age 31 . }.'
curl http://127.0.0.1:3030/health # -> "ok"
curl http://127.0.0.1:3030/metrics # Prometheus text exposition (gated by --auth-token-read; sq-9jrx)
# Admin WAL compaction / vacuum for ERASURE-COMPLETENESS (sq-x32t). POST-only; gated by the
# WRITE token. Physically purges data removed by DELETE / DROP (incl. orphaned literal VALUES) from the
# on-disk store so a logical erasure is followed by real erasure. 200 ok; 409 if in-memory
# (no --persist, nothing to purge); 503 on a transient durable-write error (retryable):
curl -X POST -H 'Authorization: Bearer <TOKEN>' http://127.0.0.1:3030/admin/compact
# Online consistent-snapshot backup + restore (feature `backup`, default OFF). POST-only, WRITE-gated.
# Backup streams a single self-describing artifact of the LIVE store WHILE SERVING (no stop-the-world):
curl -X POST -H 'Authorization: Bearer <TOKEN>' http://127.0.0.1:3030/admin/backup -o snapshot.spqb
# Restore atomically installs a store rehydrated from that artifact (in-memory; fail-closed):
curl -X POST -H 'Authorization: Bearer <TOKEN>' --data-binary @snapshot.spqb http://127.0.0.1:3030/admin/restore
# On a --persist server, opt into write-through so the restore survives a restart (else 409):
curl -X POST -H 'Authorization: Bearer <TOKEN>' --data-binary @snapshot.spqb 'http://127.0.0.1:3030/admin/restore?persist=true'
# Restore on start (bootstrap a fresh node / PITR base): sparq-server --restore snapshot.spqb
# …into a durable store (survives restart): sparq-server --persist DIR --restore snapshot.spqb --restore-persist
# Incremental change-stream / point-in-time recovery (PITR): a DELTA between a retained
# generation N and the current generation (replayed forward onto the matching base):
curl -X POST -H 'Authorization: Bearer <TOKEN>' 'http://127.0.0.1:3030/admin/backup/delta?from=0' -o d0.spqd
# Recover to the chosen point: restore the base, then replay the delta chain (oldest first):
sparq-server --restore snapshot.spqb --restore-delta d0.spqd --restore-delta d1.spqd/metrics is hand-rolled Prometheus text exposition (no metrics dependency); the
middleware wraps the whole hardening stack, so shed (429), body-limit (413) and panic
(500) responses are counted with the status the client saw:
| Metric | Type | What |
|---|---|---|
sparq_http_requests_total{endpoint,status} |
counter | requests by endpoint + response status |
sparq_query_duration_seconds |
histogram | wall time of /sparql (query + update); buckets 1 ms … 10 s |
sparq_active_subscriptions |
gauge | active WebSocket subscriptions (scrape time) |
sparq_graph_triples |
gauge | triples in the published graph (scrape time) |
sparq_updates_total |
counter | successfully applied SPARQL updates |
POST /admin/compact — WAL compaction / vacuum (erasure-completeness, sq-x32t). A logical
DELETE / DROP GRAPH retracts data from the live view but leaves the superseded bytes in earlier
--persist WAL segments (and the dictionary) until a compaction folds the live state into a fresh
base. This admin op does that on demand: it physically rewrites the on-disk store to only the
current live triples, with a re-interned (purged) dictionary, then atomically swaps the
directory (rollback-safe two-rename + WAL truncate; an interrupted swap is healed on the next
open). So a deleted triple's value — including an orphaned literal value (e.g. personal data) —
is gone from disk, not just hidden. It runs on the single writer thread strictly between
batches (no race with a concurrent write), preserves the live triple set exactly (no
generation published; reads keep flowing throughout). POST-only, gated by the write token
(--auth-token), like an UPDATE. Responses: 200 ok; 409 if the server is in-memory (no
--persist — there is no on-disk history to purge, so a no-op success would mislead); 503 on a
transient durable-write error (retryable, writer stays alive). Offline equivalent: stop the
server and run sparq-cli compact <persist-dir> (see the cli skill). Honest scope: this
scrubs the engine's own on-disk segments + dictionary; it cannot reach bytes already copied off-box
(filesystem snapshots, COW history, external backups) — those are the operator's responsibility
(see compliance/privacy/retention-erasure-runbook.md §7a/§7b).
POST /admin/backup + POST /admin/restore — online consistent snapshot backup/restore
(feature backup, default OFF; sq-o5bi). An ONLINE point-in-time backup of the live serving
store, distinct from the offline sparq-cli save (stop-the-world, index rebuild) and from the
--persist per-graph WAL. /admin/backup pins the CURRENT generation lock-free and serialises
it off the immutable Arc — so it runs while serving: readers never block the writer and the
writer never blocks readers throughout. The response is one self-describing artifact (Stardog
backup-ID model, "Option A"): a small textual header recording the generation number (= the single
writer's seq), the per-pod epoch vectors, the triple count and a body integrity digest, then the
full dataset (default + named graphs) as N-Quads. /admin/restore imports such an artifact and
atomically installs a freshly-built ring+writer rehydrated from it — readers in flight keep
serving from the old store until they release their pin; every read/update after the swap sees the
restored store. Fail-closed: a corrupt / truncated / version-mismatched / non-artifact body is
rejected (400) and the live store is left untouched (the new core is built fully before the
swap). Both routes are POST-only and gated by the write token (the admin gate). Responses:
backup 200 (application/octet-stream); restore 200 on success, 400 on a rejected artifact.
Single-flight (sq-fy8ci): a restore posted while another is still in flight is rejected
409 (a restore is already in progress) instead of being silently serialized behind it by
the writer thread (last-writer-wins) — retry after the first completes. Embedders driving
AppState::restore_from directly can claim the same permit via AppState::try_begin_restore()
(an RAII RestoreGuard; dropping it — even on a panic — releases the permit).
Restore into a live durable store (sq-ft7u). On a --persist server, a plain
/admin/restore (no opt-in) is 409 — an in-memory-only swap on a durable server would be
silently lost on the next restart (a footgun). Add ?persist=true to write the restore
THROUGH to the durable directory, so it survives a restart (the on-disk base becomes the
restored image). The durable swap runs on the single writer thread (sequenced with updates — no
lock, no race with a concurrent durable commit) and is crash-safe: the imported image is written
to a sibling and swapped in with a two-rename directory swap (parent dir fsync'd between renames)
that an interrupted crash heals deterministically (Graph::restore_into_durable reuses the exact WAL
compaction swap + recover_compaction healer). Fail-closed throughout: the artifact is imported
- validated before any on-disk change, so a corrupt artifact is
400with the durable store untouched and no swap-sibling leftovers; a swap I/O error leaves the OLD store intact and the writer alive.?persist=trueon an in-memory server is409(no durable dir). Restore-on-start:--restore <FILE>/SPARQ_RESTOREseeds the in-memory store before binding (bootstrap for horizontal-scaling stage-2 + PITR base); with--persist DIRit is refused unless you add--restore-persist/SPARQ_RESTORE_PERSIST=1, which writes the artifact through toDIRcrash-safely on start. Honest scope: at-rest encryption of the artifact is out of scope — the integrity digest detects accidental corruption, not tampering, and is not a confidentiality or authenticity guarantee; protect the artifact at the storage tier.
POST /admin/backup/delta?from=N + --restore-delta — incremental change-stream / PITR
(same backup feature; sq-bu1a). The Option-A backup above is the BASE half of a
point-in-time-recovery story; this is the incremental companion. /admin/backup/delta streams
a single self-describing delta artifact (distinct magic SPARQ-BACKUP-DELTA) keyed off the
generation/writer-seq: a header recording the from-generation N → the current to-generation
plus the per-pod epoch vector at to, then the quad-set change (inserted + deleted quads, each
as N-Quads) between those two generations. from must still be retained by the ring — without
the time-travel feature only the last few generations (the concurrency window) are retained, so a
from older than that is 410 Gone (widen retention with time-travel); a missing/invalid from
is 400. To recover to a chosen point, restore the base then replay the delta chain forward:
sparq-server --restore base.spqb --restore-delta d0.spqd --restore-delta d1.spqd … (oldest
first; or SPARQ_RESTORE_DELTA). Fail-closed: a corrupt / version-mismatched / out-of-order /
gapped delta aborts the whole recovery and the live store is untouched (import + full replay happen
before any swap). Honest scope: deltas are same-lineage only — the chain must be the writer
history of that base (blank-node labels are stable within a lineage, which is what the quad-set diff
relies on); cross-lineage diffing is unsupported. At-rest encryption is out of scope, same as the
base.
Durable, replayable change-data-capture stream (sparq-serve feature change-stream,
default OFF; sq-b4fns, gh-906). Where the delta above is the PITR backup companion, this is
a continuous CDC stream in the Amazon-Neptune-Streams shape: sparq_serve::change_stream::ChangeLog
records every commit as one ordered, monotonically-sequenced change record —
(seq, generation, timestamp, +inserts / −deletes as N-Quads) — to a segmented, fsync'd
append-only log on disk. A consumer poll(from_seq)s from any offset and replays after a
process restart (ChangeLog::open re-reads the segments, recovers a torn tail, and resumes at the
next seq) — a raw durable cross-service feed (replication / live downstream index / triggers),
distinct from the ephemeral in-process WebSocket/SSE subscriptions (a disconnect misses every
change). Same N-Quads same-lineage quad-set diff + fail-closed FNV-1a digest as the backup family
(no new dependency; no HTTP/async in the library). At-rest encryption + authenticity are out of
scope, same as the backup family. Retention/truncation (sq-n9s4d): ChangeLog::apply_retention
(and the deterministic apply_retention_at) drops whole OLD segments — oldest-first, never the
active segment, never a partial segment — under a RetentionPolicy composing a consumer-ack
watermark (a HARD safety bound: any unacked record keeps its segment), max_age, and
max_total_bytes pressure; the default policy is a no-op and nothing is ever dropped implicitly.
A poll from a trimmed-away offset fails closed (never a silent skip) — consumers resume from
ChangeLog::first_seq.
External-broker sink (sq-l6zks, gh-3216) — the separate, heavier sparq-serve opt-in
change-sink (default OFF, implies change-stream). change_sink::ChangeSink is the pluggable
broker seam; BrokerRelay::open(dir, consumer, sink, config) + pump() is the resumable pump from
that durable log to a sink, carrying a durable delivered-through watermark (persisted as
changesink-<consumer>.offset beside the segments) so a restarted process resumes rather than
replays. Delivery is at-least-once — the watermark is persisted after the sink's flush, so
consumers dedupe on sequenceNumber; the partition key is CONSTANT per stream so a partitioned
broker preserves commit order; a re-base gap record is delivered as an explicit "op": "REBASE"
entry, never as an empty commit. The relay runs off the writer thread (a broker outage stalls
the relay, never commits), and feeding delivered_through_seq() into
RetentionPolicy::acked_through_seq keeps retention from trimming past it (a trimmed watermark
fails the pump closed). One in-tree sink ships — NatsSink (core NATS over plain TCP, std-only,
no TLS). There is deliberately no in-tree Kafka client: Kafka (and TLS/SASL/retries) is
reached by implementing ChangeSink over the host's own client, so no broker client and no async
runtime enter the library-first crate. Payloads are plaintext, unsigned JSON — same boundary as the
backup family — so pointing a relay at a broker is a data-egress decision.
Recording seam (sq-bdaw5): install ChangeLog::into_commit_hook(on_error) via
Writer::spawn_with_commit_hook and every commit-publish is recorded on the writer thread —
one append+fsync per PUBLISHED GENERATION, gapless by construction (the writer is the sole
publisher), with no caller-side lock and no serialisation of submitters; acks happen-after the
record is durable. Restore publishes never fire the hook (not same-lineage — re-baseline
explicitly). A recording failure is dropped + reported to on_error, never failing the
already-published write; the log's discontinuity check then fail-closes later records rather than
silently gapping. Resync (sq-r2cu1): ChangeLog::rebase_to(generation) is that explicit
operator re-base — it appends an honest gap record (ChangeRecord::rebase = true, no changes)
marking the uncaptured span and re-arms recording from generation (strictly forward-only,
fail-closed otherwise) without wiping the log; a consumer replaying past the gap re-bootstraps
(e.g. from a backup at/after that generation) instead of trusting a silently incomplete diff stream.
On a RUNNING recorder, ChangeLog::into_commit_hook_with_control yields a ChangeStreamControl
that runs the resync on the LIVE log (serialized with the hook — no stop/re-open), and
rebase_to_new_lineage is the post-restore variant for a REPLACED lineage whose generation
numbering restarted (gh-2436).
POST /admin/change-stream/rebase — operator resync of the running server's tracked log
(sparq-server feature change-stream; gh-2436). WRITE/admin-gated, POST-only, same
double-opt-in 404 as /streams. Empty body = re-baseline to the writer's CURRENT generation
(the dropped-record resync); JSON fields generation (explicit target) and newLineage: true
(AFTER /admin/restore — the restored ring restarts at generation 0, so the same-lineage
forward-only check would reject it forever). 200 returns the gap record (seq/generation/
rebase + nextSequenceNumber); a non-forward same-lineage target is a 409; an unknown body
field is a fail-closed 400. GET /streams renders the gap as ONE explicit "op": "REBASE"
marker entry (no data) — never silently flattened away.
GET /streams — CDC poll endpoint (sparq-server feature change-stream, default OFF;
sq-2999l, gh-906). The HTTP poll surface over that durable log, in the Amazon-Neptune-Streams
GetRecords shape. Configure a log directory (--change-stream DIR / SPARQ_CHANGE_STREAM,
ServerConfig::change_stream_dir) and the server (1) RECORDS every published group-commit
generation as one ordered change record on the writer thread (possibly containing several
concurrent SPARQL Updates; [GPT-5.6] sq-kqofk), and (2) serves GET /streams over it (the route
is 404 unless the directory is set — the same double-opt-in as /tpf). Parameters: iteratorType
(TRIM_HORIZON = replay everything RETAINED, the default; AT_SEQUENCE_NUMBER + at=N;
AFTER_SEQUENCE_NUMBER + after=N, the resume case; LATEST = tail only) and limit (max commits per page, default 100,
clamped to 10000). The JSON response flattens each commit's quad-level changes to one stream record
per (op, quad) — { eventId: { commitNum: <seq>, opNum: <1-based> }, op: ADD|REMOVE, generation, commitTimestampNanos, data: { stmt: "<n-quads line>" } } — plus a nextSequenceNumber continuation
token (pass as at=/after=), lastSequenceNumber, totalRecords (commit count) and
hasMoreRecords. A poll is a READ (gated by the read auth). A sequence-anchored iteratorType with
no anchor is a fail-closed 400 (never a silent replay-all). Retention-aware (sq-iz7ag):
TRIM_HORIZON resolves to the log's TRIM HORIZON (ChangeLog::first_seq), not seq 0 — polling
below the horizon fails closed in the durable log, so pinning the whole-stream replay at 0 would
500 on any trimmed log. An at/after anchor whose RESOLVED start (after=N ⇒ N+1) is below
the horizon is a 410 Gone naming the seq to resume from — the records are permanently gone and
that consumer must re-bootstrap — never a silent restart at the horizon (which would skip records
it has not seen). The horizon is the read handle's view, taken when the server opened the log
directory: sparq-server never applies retention itself, so a host that trims through its own handle
while the server runs still gets the fail-closed 500 until a restart. With the feature off the route + the
recording hook are #[cfg]-stripped (byte-identical); with it on, recording rides the sequenced
writer's group commit and does not serialise submitters. Relaying that log onward to an external
broker is the separate sparq-serve change-sink opt-in (sq-l6zks, above), not part of this
endpoint.
GET /queries + DELETE /queries/{id} — running-query registry (sparq-server feature
query-registry, default OFF; sq-qsm5z, [SONNET-4.6]). An opt-in in-memory registry of
currently executing SPARQL queries, providing GraphDB query-monitoring and kill parity.
GET /queries— READ-gated (fail-closed). Returns{"queries":[…]}where each entry carriesid(opaque hex),kind("query"or"update"),start(Unix epoch ms),fingerprint(FNV-1a hex of the trimmed query text — never raw text, per the #241 / sq-m9prn audit-log posture), andelapsed_ms. Sorted byid(registration order).DELETE /queries/{id}— WRITE-gated (fail-closed). Cooperatively cancels the named query by flipping thesq-kq9iaArc<AtomicBool>cancel flag wired into itsQueryBudget. The engine observes it at the next poll site and aborts. Returns204 No Contenton success;404if the id is not found (already finished or bad id).- RAII lifetime: each executing SELECT/ASK/CONSTRUCT/DESCRIBE/EXPLAIN (plan + analyze) registers on start and deregisters on completion, error, or panic — the entry is always cleaned up. (EXPLAIN ANALYZE wiring added in sq-t1isr, [SONNET-4.6].)
- SPARQL UPDATEs are registered too (
kind: "update"; sq-m9prn, [SONNET-4.6]). An UPDATE registers when the sequenced writer thread STARTS applying it — not when it is queued — so a row names exactly what is consuming the writer, which is the operation whose cancellation also unblocks every write queued behind it. The flag reaches theDELETE/INSERT … WHEREevaluation (the engine installs the UPDATE'sQueryBudgetthread-locally for the whole update). A cancelled UPDATE is rejected, not partially applied: the writer forks per batch and seals only on success, so the store is left at its pre-update state and the writer keeps serving. Non-WHERE operations (INSERT DATA,CLEAR/DROP, …) do not consult the budget, so they are listed but complete rather than cancel. - Fingerprint-only — and what that does NOT buy you (honest boundary). The
GET /querieslisting never exposes raw query or update text; only the fingerprint is visible, satisfying the audit-log redaction discipline. But the construction is FNV-1a: an unkeyed, 64-bit, non-cryptographic checksum. It is a stable correlation tag, not a confidentiality boundary and not a one-way function in any cryptographic sense: anyone who can read the listing can hash candidate texts offline and confirm a match, so a guessable — or low-entropy, or template-generated — body is recoverable by search, and 64 bits resists neither exhaustive guessing of a small candidate set nor collision search. This matters most forkind: "update", whoseINSERT DATAbody can embed secrets or personal data: the fingerprint stops the payload being transmitted, it does not stop it being guessed. Treat the fingerprint as sensitive metadata, keep/queriesbehind its READ gate (it is fail-closed), and do not rely on it to protect update content from a party who already has read access. (Keying the fingerprint — e.g. an HMAC under a per-process secret — would close the offline-guessing gap and is tracked as follow-up work, not shipped here.) - Zero cost when feature is OFF: no
AppStatefields, no routes; byte-identical to before.
# (--features query-registry required at build time)
# List running queries:
curl -H 'Authorization: Bearer <TOKEN>' http://127.0.0.1:3030/queries
# Cancel a query by id:
curl -X DELETE -H 'Authorization: Bearer <TOKEN>' http://127.0.0.1:3030/queries/0000000000000001GSP writes translate into a server-minted SPARQL Update (DROP/CLEAR + INSERT DATA) and submit through the SAME sequenced group-commit writer the
application/sparql-update operation uses — so they share its atomicity, snapshot
consistency, blocking-on-commit semantics, the Sparq-Generation header (default build,
sq-ci2d6), AND its auth gate (a GSP write is as powerful as an UPDATE, so PUT/POST/
DELETE/PATCH are gated by --auth-token exactly like an UPDATE; GET/HEAD are reads). A
malformed body → 400; an unsupported body Content-Type → 415.
PATCH — atomic, graph-scoped in-place modify (sq-hj4n, gh-916). A PATCH on a graph
resource applies an atomic modify to the addressed graph, with two body dialects:
application/sparql-update(ALWAYS-ON, no feature): the body IS a SPARQL Update, applied atomically through the shared writer. It is scoped to the addressed graph by defaulting the update's WHERE dataset to that graph (the SPARQL 1.1 Protocol §2.2using-graph-urimechanism). Honest scope: this scopes what the WHERE reads; an operation that explicitly names a different graph (INSERT DATA { GRAPH <other> { … } }, or an in-bodyWITH/USING) still targets where it says — and supplying the override alongside an in-bodyUSING/WITHon a named-graph PATCH is the §2.2 conflict400. For?defaultthe scoping is a no-op (the default graph is already the WHERE default). Success →204.text/n3(OPT-IN — then3-patchbuild feature AND--n3-patch/SPARQ_N3_PATCH=1): a Solid-style N3-Patch (solid:InsertDeletePatchwithsolid:where/solid:deletes/solid:insertsformulas), translated into ONE atomic graph-scoped SPARQL Update (every block wrapped inGRAPH <g> { … }for a named graph). With asolid:whereit is a pattern-basedDELETE/INSERT … WHERE; without one it is groundDELETE DATA/INSERT DATA— either way one writer submission. Validation: exactly one patch resource, each formula property at most once, at least one ofdeletes/inserts, and no blank node indeletes/where(onlyinsertsmay mint one) → otherwise400. Double opt-in (build feature + runtime flag) mirrorstpf/shacl: with the feature off the N3-Patch parser + dispatch are#[cfg]-stripped (atext/n3body is then a plain415, byte-identical to before, and no new dependency / nosparq-core/sparq-engine/ wasm impact); with the feature on but the flag off atext/n3body is also415. The N3 parsing ridesoxttl'sN3Parser— already a server dependency — so even feature-on adds NO new crate. Success →204.PATCHis a WRITE, gated by--auth-tokenlike an UPDATE.
5b. Container (ghcr.io). Published on every vX.Y.Z release tag as a distroless OCI image
index at ghcr.io/sparq-org/sparq-server, with linux/amd64 and linux/arm64 runtime images.
[GPT-5.6] sq-fvzi6: each release publishes :X.Y.Z (pin this for reproducible deployments),
:X.Y (tracks the newest patch in that minor line), and :latest (tracks the newest release);
an omitted tag selects :latest. The image sets SPARQ_ALLOW_REMOTE=1 so the 0.0.0.0 bind it
needs (loopback is unreachable through Docker's port map) boots out of the box — running the
container is the operator's explicit choice to publish a surface. This default has NO auth.
Because every SPARQ_* var is read from the environment, secure it with -e (no flag wiring):
docker run --rm -p 3030:3030 ghcr.io/sparq-org/sparq-server # empty graph, no auth
docker run --rm -p 3030:3030 -v "$PWD/data:/data:ro" \
ghcr.io/sparq-org/sparq-server --format turtle /data/dataset.ttl # serve a dataset
docker run --rm -p 3030:3030 \
-e SPARQ_AUTH_TOKEN="$TOK" -e SPARQ_AUTH_TOKEN_READ=1 \
ghcr.io/sparq-org/sparq-server # fully Bearer-gatedDeliver the token over TLS (terminate at a proxy). See crates/sparq-server/README.md →
"Running the container image".
The image self-declares a HEALTHCHECK (CIS Docker §4.6, sq-toze.36). Since distroless has
no shell/curl/wget, the check is the server binary probing itself: sparq-server --health-probe opens a TCP connection to the loopback /health and exits 0 (healthy)/non-zero
(unhealthy) — exec-form in the Dockerfile. Override the probed address with --health-probe-addr HOST:PORT or SPARQ_HEALTH_PROBE_ADDR. docker ps then shows (healthy)/(unhealthy); k8s
usually runs its own /health probe and ignores the image healthcheck.
5c. Federation discovery — VoID + Service Description (OPT-IN, sq-d3d8). A server can
advertise itself as a discoverable federation node by serving two read-only RDF descriptors:
-
GET /.well-known/void— a W3C VoID dataset description (void:triples/void:entities/void:classes/void:properties+ onevoid:classPartitionper class andvoid:propertyPartitionper predicate), plus the characteristic-set source statistics (sq-mr32, federation A3/Z2): sparq already mines per-entity-type predicate co-occurrence + multiplicity (Neumann & Moerkotte characteristic sets), so the served VoID also emits them under a documented sparq extension vocabscs:(<http://sparq.dev/ns/cs#>) —scs:characteristicSetlinking the dataset to onescs:CharacteristicSetnode per retained set (scs:subjects=count(C), plus one per-predicatescs:predicateStatreusingvoid:property/void:triplesand addingscs:avgMultiplicity), with the EXACT distinct-set count onscs:distinctCharacteristicSets. The CS stats are a strict superset of the standard VoID (a VoID-only client ignores thescs:triples; a CostFed/Odyssey-class source-selector gets sharp star/multi-join cardinalities). Generated bysparq-introspect'sIntrospection::to_void_with_cs(Introspection::to_voidremains the CS-free base). -
GET /sparqlwith noqueryparameter — a SPARQL 1.1 Service Description generated from the server's actual capabilities (sq-qfcb), never a hard-coded fiction:sd:Service+sd:endpoint(the requestHost's/sparql);sd:supportedLanguage—SPARQL11Queryalways, plus the SPARQL-1.2-SD version-agnosticSPARQLQuery(sq-2msb);SPARQL11Update+SPARQLUpdateonly when an anonymous client can run one (suppressed when a--auth-tokenwrite gate is configured, because then an unauthenticated SD reader cannot use Update);sd:supportedVersion(sq-2msb, gh-917) — the SPARQL language versions this build conformance-verifies, assparql:version-*IRIs (http://www.w3.org/ns/sparql#). SPARQL 1.2 SD moves version negotiation offsd:Languageontosd:supportedVersion, so a 1.2-aware federation client can discover triple-term /dir-lang support without probing. sparq advertisesversion-1.0,version-1.1and the fullversion-1.2(not theversion-1.2-basicprofile) because the engine passes the complete W3C SPARQL 1.0/1.1/1.2 suites at 100% (conformance-report.md). HONESTY GATE: there is nosparql12/rdf12cargo feature — SPARQL 1.2 is compiled into the base engine — so this is keyed off the DOCUMENTED conformance state (descriptors::CONFORMANCE_VERIFIED_VERSIONS), never acfg!or an aspiration; were any 1.2 group to regress to a partial pass, the honest edit is to drop toversion-1.2-basic(or omit 1.2) in that one constant;sd:resultFormat— the four SPARQL-results serialisations (JSON/XML/CSV/TSV) plus the RDF graph serialisations the CONSTRUCT/DESCRIBE/GSP-read path emits (Turtle/N-Triples/RDF-XML), andsd:inputFormat— the RDF serialisations the GSP write path parses (Turtle/N-Triples/ RDF-XML). These mirror exactly whatcrate::negotiateproduces/accepts; an integration test drives a real SELECT per advertised result format and asserts the returnedContent-Typematches, so the advertisement cannot over-promise;sd:feature sd:BasicFederatedQuery— only when theservicecargo feature is compiled in (theSERVICEclause is then evaluable); omitted otherwise;sd:feature <http://sparq.dev/ns/prov#lineage>(sq-yyy3) — the sparq W3C-PROV-O data-lineage extension, advertised only when this node genuinely serves PROV-O lineage for the data it derives (Capabilities::provenance).sparq-serverexposes no lineage endpoint today, so it keeps the flagfalseand the feature is omitted — advertising it would over-promise. The descriptor + thesparq-fedclientdiscovery parser (Capability::provenance_lineage) support the round-trip so a lineage-serving node can flip it honestly without a vocabulary change;sd:extensionFunction— one per function the engine has actually registered: thegeof:GeoSPARQL set with thegeofeature (read back throughFunctionRegistry::iris(), so it can never drift from what runs), none without it;- the default dataset — its default graph linked to the VoID document via
dcterms:source, plus ansd:namedGraphenumeration of every IRI-named graph in the served dataset (sq-optl): each is ansd:NamedGraphwith itssd:name(theFROM NAMED-referenceable IRI) and ansd:graphsd:Graphcarrying that graph'svoid:triplescount. The names come off the same pinned snapshot the VoID descriptor reads, sorted for determinism, and IRI-only — a blank-node-named graph is skipped because it is notFROM NAMED-referenceable, so advertising it would be a fiction. A default-only dataset emits nosd:namedGraph.
Note
sd:UnionDefaultGraphis deliberately not advertised — the engine's default graph holds only default-graph triples (named graphs are not folded in), so claiming it would be dishonest.
Both are double opt-in and OFF by default: compiled only with the
federation-descriptors cargo feature and served only when
--federation-descriptors / SPARQ_FEDERATION_DESCRIPTORS=1 is also set. Without the
feature there is zero cost (no sparq-introspect dependency, no routes); with the feature
but not the flag, /.well-known/void is 404 and a no-query GET /sparql is the historical
400 missing 'query'. Both content-negotiate the RDF syntax (Accept: text/turtle default,
application/n-triples, application/rdf+xml); reads are gated by --auth-token-read like
any GET. The dataset/endpoint IRIs self-describe the Host the client used.
cargo run -p sparq-server --features federation-descriptors -- data.ttl --federation-descriptors
curl http://127.0.0.1:3030/.well-known/void # VoID + scs: char-set stats (Turtle by default)
curl -H 'Accept: application/n-triples' http://127.0.0.1:3030/sparql # Service Description (no query)5c-bis. Grouped facet counts (OPT-IN, [GPT-5.6] sq-lsp7k.5.2; feature facets).
POST /facets evaluates one sparq_introspect::FacetRequest against a pinned store snapshot and
returns its FacetResponse JSON: candidate-subject count plus ranked type, predicate, and requested
object-value distributions. class is an optional rdf:type class IRI; every constraints pair is
[predicate IRI, object N-Triples term] and the pairs are AND-combined; facet_predicates: null
requests values for every candidate predicate; top_k bounds every retained distribution.
cargo run -p sparq-server --features facets -- data.ttl --facets
curl -X POST -H 'Content-Type: application/json' http://127.0.0.1:3030/facets -d '{
"class":"http://ex/Person",
"constraints":[["http://ex/status","<http://ex/active>"]],
"facet_predicates":["http://ex/tag"],
"top_k":10
}'The response is 200 application/json; malformed JSON is a sanitized 400. The scan runs on the
blocking pool and holds one immutable generation pin for the whole evaluation. The endpoint is a
READ, so --auth-token-read gates it. It is double opt-in and OFF by default: the facets cargo
feature compiles the route + sparq-introspect dependency, and --facets / SPARQ_FACETS=1
serves it. Feature on but flag off gives 404; feature off compiles no route or dependency.
5c-ter. IRI/label prefix completion (OPT-IN, [GPT-5.6] sq-lsp7k.9.3; feature
complete). GET /complete?q=<prefix>&limit=<k> returns case-insensitive prefix matches from
complete IRI strings, IRI local names, and literal rdfs:label / skos:prefLabel values. q is
required (missing → 400); limit defaults to 20 and is capped at 100. The response is a JSON
array in deterministic rank/key/id/source order:
[
{"iri":"http://ex/alice","key":"alice","kind":"localName","score":0.0},
{"iri":"http://ex/alice","key":"alice","kind":"label","score":0.0}
]kind is iri, localName, or label. One entity may appear more than once when multiple keys
or sources match. v1 passes no external rank scores, so score is 0.0; PageRank-backed ranking
is a separate composition step. The server pins one immutable generation, lazily builds its
CompletionIndex on the blocking pool, caches it in AppState, and rebuilds when a later update
publishes a different generation. Thus a successful update is visible to the next completion
request without serving stale candidates.
cargo run -p sparq-server --features complete -- data.ttl --complete
curl -G http://127.0.0.1:3030/complete --data-urlencode 'q=ali' --data-urlencode 'limit=10'The endpoint is a READ, so --auth-token-read gates it. It is double opt-in and OFF by default:
the complete cargo feature compiles the route and the pure-index sparq-text dependency
(default-features = false), while --complete / SPARQ_COMPLETE=1 serves it. Feature on but
flag off gives 404; feature off compiles no route, cache, or text dependency.
5d. Triple Pattern Fragments / LDF source endpoint (OPT-IN, sq-bzh1). A server can expose
itself as a low-cost Linked Data Fragments /
Triple Pattern Fragments
source that a TPF client (Comunica / the LDF client) drives a join against — far cheaper per
request than a full SPARQL endpoint:
GET /tpf?subject=&predicate=&object=— a paged RDF fragment of the triples matching one triple pattern. Each ofsubject/predicate/objectis an N-Triples term (<iri>,"lit","lit"@en,"lit"^^<dt>); an absent / empty parameter is a variable (unbound).page=N(0-based) selects the page; the page size is bounded (default 100).- The fragment carries Hydra controls:
hydra:totalItems/void:triples(the matched-triple count, reusing the engine's cheap cardinality estimate — NOT a full scan),hydra:itemsPerPage, the fullPartialCollectionViewpaging vocabulary —hydra:next/hydra:previous(present only when a next / previous page exists) plushydra:first/hydra:last(emitted on EVERY page so a client can jump to either end of the view from anywhere;firstis always page 0,lastis the page holding the final match — derived from the same estimate astotalItems/next) — and ahydra:search/hydra:template/hydra:mappingcontrol describing the{subject,predicate,object}URI template so a generic client can request any other pattern.
5d-bis. brTPF — bind-restricted Triple Pattern Fragments (OPT-IN, sq-dxhb; feature brtpf,
implies tpf). brTPF (Hartig & Buil-Aranda, ODBASE 2016)
extends the SAME /tpf endpoint so a client can attach a set of solution mappings and the
server returns only the page of pattern matches COMPATIBLE WITH AT LEAST ONE supplied binding —
pushing a bind-join's semi-join down to the source, so far less data crosses the wire than
re-fetching the whole pattern once per binding.
- The binding set rides the
valuesquery parameter (GET) or — preferred for a large set — the request body of aPOST /tpf. The wire format is one mapping per line; within a line, whitespace-separatedposition=termpairs wherepositionissubject/predicate/object(or shorts/p/o) andtermis the SAME N-Triples-term grammar as the pattern parameters (e.g.s=<http://ex/alice>). A blank line / empty payload is the no-restriction (plain-TPF) case; a malformed payload is a sanitized400(the offending input is NOT echoed). - The fragment is the deduplicated union of each mapping's specialised-pattern matches.
hydra:totalItemsand the paging window reflect the bindings-RESTRICTED result, not the unrestricted pattern, and thehydra:searchcontrol advertises an extrahydra:mappingfor thevaluesvariable so a client discovers the dataset accepts a restriction. - A
tpf-only build is byte-identical to before: thevaluesparsing + thePOSTroute are#[cfg]-stripped, so a strayvaluesparameter is just an ignored unknown parameter (plain TPF). Still governed by the same--tpfruntime flag and read-auth (POST /tpfis a READ — it returns a fragment, it never writes). - DoS caps on the binding set (
sq-r74h). The brTPF fragment runs ONE index scan per attached mapping (tpf::evaluate_brtpf), so the per-request cost is super-linear in the mapping count, not the payload bytes — and--max-body-bytesbounds the count only transitively (a 1 MiB body ofs=<a>-sized mappings is ~150k scans) and does NOT cover thevaluesquery-string carrier of aGET /tpfat all (it is a body limit). Two dedicated, ON-by-default caps close that:--brtpf-max-bindings(default1024) bounds the mapping count, and--brtpf-max-values-bytes(default1 MiB) bounds the rawvaluespayload bytes — enforced BEFORE any parse/index work. A breach is a413(the same refusal class as--max-body-bytes, distinct from the malformed-payload400); the message names the cap, never the caller's input (no echo).0disables either cap. The pure parser istpf::parse_bindings_capped(payload, tpf::BindingLimits { max_mappings, max_payload_bytes }), returningtpf::BindingError::{Malformed → 400, TooLarge → 413}.
cargo run -p sparq-server --features brtpf -- data.ttl --tpf
# restrict `?s ex:knows ?o` to a single subject via the `values` parameter
curl 'http://127.0.0.1:3030/tpf?predicate=%3Chttp%3A%2F%2Fex%2Fknows%3E&values=s%3D%3Chttp%3A%2F%2Fex%2Fcarol%3E'
# a larger binding set in a POST body (one `position=term` mapping per line)
curl -X POST -H 'Accept: application/n-triples' \
--data $'s=<http://ex/alice>\ns=<http://ex/carol>' \
'http://127.0.0.1:3030/tpf?predicate=%3Chttp%3A%2F%2Fex%2Fknows%3E'Double opt-in, OFF by default and READ-only (no write path): compiled only with the tpf
cargo feature and served only when --tpf / SPARQ_TPF=1 is also set (mirrors
federation-descriptors). Without the feature, zero cost (no route); with the feature but not
the flag, /tpf is 404. Content-negotiates Accept: text/turtle (default) /
application/n-triples / application/rdf+xml; reads are gated by --auth-token-read like any
GET. The fragment/dataset/template IRIs self-describe the Host the client used.
cargo run -p sparq-server --features tpf -- data.ttl --tpf
# all triples with predicate ex:knows, page 0 (Turtle by default)
curl 'http://127.0.0.1:3030/tpf?predicate=%3Chttp%3A%2F%2Fex%2Fknows%3E'
# a fully-bound pattern, as N-Triples
curl -H 'Accept: application/n-triples' \
'http://127.0.0.1:3030/tpf?subject=%3Chttp%3A%2F%2Fex%2Falice%3E&predicate=%3Chttp%3A%2F%2Fex%2Fknows%3E'5e. SHACL validation endpoint (OPT-IN, sq-r868, from-pss gh-162 follow-up; feature
shacl). Validate the server's currently-loaded data graph against a SHACL shapes
graph the client POSTs — the server-side / large-graph path from gh-162 (the store is already in
memory, so there is no per-request data parse, and the 100k-node case where the JS
rdf-validate-shacl OOMs is handled natively by sparq-shacl).
POST /shacl/validate— the request body is the SHACL shapes graph (RDF:text/turtle/application/n-triples/application/n-quads/application/trig/application/rdf+xml, classified byContent-Typelike a GSP write body, and gzip-decoded under the same zip-bomb cap). The data graph is the server's pinned store snapshot.- Response, content-negotiated from
Accept: the default is the JSON projection PSS / the wasmshaclbinding consume —{ "conforms": bool, "results": [{ "focusNode", "path", "value", "sourceShape", "sourceConstraintComponent", "severity", "message" }] };Accept: text/turtleyields the W3C SHACL report-vocabulary graph (sparq_shacl::ValidationReport::to_turtle). - Always
200regardless of conformance — the verdict is in the body (conforms), not the HTTP status. A malformed shapes body is a400, an unsupportedContent-Typea415, a non-POSTmethod a405. - Covers SHACL Core + SHACL-SPARQL (
sh:sparql, §5.2) + custom SPARQL constraint components (§6) — whateversparq-shacl::validatesupports (it is the SAME engine the wasm binding and thesparq-shaclcrate expose; see theshacl-validationskill). SHACL-AFsh:ruleinference is not part of validation and is not run by this endpoint.
cargo run -p sparq-server --features shacl -- data.ttl --shacl
# validate the loaded store against POSTed shapes → JSON report (default)
curl -X POST -H 'Content-Type: text/turtle' --data-binary @shapes.ttl \
http://127.0.0.1:3030/shacl/validate
# the W3C report vocabulary as Turtle
curl -X POST -H 'Content-Type: text/turtle' -H 'Accept: text/turtle' --data-binary @shapes.ttl \
http://127.0.0.1:3030/shacl/validateDouble opt-in, OFF by default and READ-only (validation never mutates the store):
compiled only with the shacl cargo feature and served only when --shacl / SPARQ_SHACL=1
is also set (mirrors tpf / federation-descriptors). Without the feature, zero cost (no route,
no SHACL code — sparq-core/the wasm bundle are untouched); with the feature but not the flag,
/shacl/validate is 404. Reads are gated by --auth-token-read like any GET.
SHACL transaction guard (sq-lsp7k.2.4, same shacl feature; [GPT-5.6]). Build with
--features shacl, load a shapes graph with --shacl-shapes FILE (env
SPARQ_SHACL_SHAPES), and enable --shacl-guard (env SPARQ_SHACL_GUARD=1) to validate the
post-state of every SPARQL Update and Graph Store write. A conforming candidate commits normally;
a non-conforming candidate is rejected with 422 Unprocessable Entity and the same JSON
validation-report projection as /shacl/validate, and the published/durable store is unchanged.
Validation runs on the writer's private candidate before persistence and publish, so GSP
PUT/POST/DELETE/PATCH, template UPDATEs, and /sparql updates share the guard. The guard is
runtime-OFF by default and independent of the read-only --shacl endpoint flag. Enabling it
without a loaded shapes graph is a startup error. Library embedders set
ServerConfig { shacl_guard: true, shacl_shapes: Some(ShaclShapes::new(shapes)), ..Default::default() }.
cargo run -p sparq-server --features shacl -- data.ttl \
--shacl-guard --shacl-shapes shapes.ttl5f. Terse transpiler endpoint (OPT-IN, sq-vczh2, epic sq-2m6zm; feature terse). Transpile
a terse query (the K:<name> keyword layer over canonical SPARQL) into the canonical,
conformant SPARQL it expands to, returning the verifiable expansion — NOT an answer. This is
the LLM-ergonomic surface from research/llm-ergonomic-sparql-surface.md §4: the network contract
is the auditable expansion the agent can inspect, never an opaque oracle.
POST /terse/transpile— the request body is the terse query text (read verbatim as UTF-8, gzip-decoded under the same zip-bomb cap). The server runs the LEANsparq_terse::terse_to_sparql(theK:<name>keyword layer + the silent-rewrite canary) and returns JSON:{ "canonical_sparql", "keywords": [{ "keyword", "iri", "legendVersion" }], "resolutions": [], "warnings": [], "legendVersion" }.- The whole contract is
canonical_sparql— standard SPARQL the agent then runs through the normal/sparqlpath. The endpoint never executes the query and never reads the store. - Loud-fail, never a silent guess: an unknown
K:<name>keyword, aPREFIX K:collision, or non-conformant input (the canary) is a400carrying the transpiler's own message; a non-POSTmethod is a405. resolutionsis always[]in this server build:V("phrase")concept resolution needs a graph-bound resolver + an embedder (the crate'svectorsfeature), a future extension, so aV(...)construct is a400here rather than guessed. Caveat (sq-26fdp): the lexical linker can loud-fail a phrase that IS a verbatimskos:prefLabelwhen it shares a token with a sibling — a known soundness-conservative behaviour, fix tracked separately.
cargo run -p sparq-server --features terse -- data.ttl --terse
# expand the K: keyword layer to canonical SPARQL (the agent then runs canonical_sparql at /sparql)
curl -X POST --data-binary 'SELECT ?s WHERE { ?s K:derivedFrom ?o }' \
http://127.0.0.1:3030/terse/transpile
# → {"canonical_sparql":"SELECT ?s WHERE { ?s <http://www.w3.org/ns/prov#wasDerivedFrom> ?o }",
# "keywords":[{"keyword":"derivedFrom","iri":"http://www.w3.org/ns/prov#wasDerivedFrom",
# "legendVersion":"pkg-keywords/v1"}],"resolutions":[],"warnings":[],"legendVersion":"pkg-keywords/v1"}Double opt-in, OFF by default: compiled only with the terse cargo feature and served only
when --terse / SPARQ_TERSE=1 is also set (mirrors shacl / tpf). Without the feature, zero
cost (no route, no terse code — sparq-core/sparq-engine/the wasm bundle untouched); with the
feature but not the flag, /terse/transpile is 404. Transpiling is query-shaped, so it is gated
by --auth-token-read like a GET. The CLI exposes the same transpiler as sparq-cli terse (see the
cli skill).
5g. Named parameterized SPARQL templates (OPT-IN, sq-lsp7k.10; feature templates).
[FABLE-5] Server-stored, IRI-identified query/UPDATE templates with typed, fail-closed
parameter binding — the GraphDB "SPARQL templates" (smart updates) / Stardog "stored
queries" parity surface, and what an app backend or LLM agent should call instead of
composing free-form UPDATE strings.
GET /templates— list every stored definition (read-gated).GET /templates/{name}— one definition; the{name}segment is a wildcard capture, so IRI names work verbatim.PUT /templates/{name}— store/replace (write-gated); the JSON body is{ "text" | "sparql", "parameters": { name → auto|iri|string|boolean|integer|decimal|double|<datatype-IRI> }, "description"? }. Fail-closed registration: unparseable text, a declared parameter that is not a free bindable placeholder (e.g. a literal type in a predicate slot), or a result/aggregate/BIND output variable is a400— a stored template is always invocable.DELETE /templates/{name}— remove (write-gated).POST /templates/{name}— invoke with a JSON argument object. Binding is the #901paramsalgebra rewrite (sparq_engine::templates): values become typed terms inside the parsed AST, never string concatenation, so a hostile value cannot inject syntax. Fail-closed invocation: an unknown/missing/mistyped argument is a400. SELECT/ASK → SPARQL-JSON; CONSTRUCT/DESCRIBE → N-Triples; an UPDATE template is write-gated and runs through the SAME sequenced-writer path (run_update) as a/sparqlupdate — budgets, atomicity, durability and the gated-update posture are identical.--templates-file PATH(envSPARQ_TEMPLATES_FILE) makes the store durable: loaded fail-closed at startup (a corrupt file is a startup ERROR, never a silently-empty store), rewritten atomically (write-temp + rename) on every successfulPUT/DELETE.
cargo run -p sparq-server --features templates -- data.ttl --templates
curl -X PUT -H 'Content-Type: application/json' http://127.0.0.1:3030/templates/friends \
-d '{"text":"SELECT ?f WHERE { ?who <http://ex/knows> ?f }","parameters":{"who":"iri"}}'
curl -X POST -H 'Content-Type: application/json' http://127.0.0.1:3030/templates/friends \
-d '{"who":"http://ex/alice"}' # → SPARQL-JSON, typed + injection-safeDouble opt-in, OFF by default: compiled only with the templates cargo feature and
served only when --templates / SPARQ_TEMPLATES=1 is also set (mirrors shacl/tpf/
terse); without the flag every /templates path is 404. The MCP surface exposes the same
template layer as template_invoke (see the agent-tools skill).
Solid WAC/ACP authorization endpoints (solid-authz feature, default OFF; sq-snopa.6, issue #992 FR-4)
A thin, fail-closed HTTP shell over the sparq-solid library
authoriser — the deliberately-opt-in sparq-server → sparq-solid workspace dependency. All three
endpoints are POST and take the pod dataset (N-Quads, incl. the .acl/.acr control graphs)
plus an already-resolved session in a JSON body — or, with "source":"server" (sq-snopa.8, see
below), the server's own loaded store instead of a body dataset.
The server does NOT authenticate — mapping a
WebID + a request path is the caller's job (sparq-solid is a library authoriser with no HTTP
surface, research/sparq-solid-scope.md §4); this is exactly that missing shell. view ("wac" /
"acp") selects the model, else it is inferred (.acr present → ACP, else WAC). See also the
access-control skill for the library decide / wac_allow / query_as surface underneath.
POST /authz/decide— body{ "dataset", "session": { "agent"?, "client"?, "issuer"?, "now"? }, "resource", "mode": "read"|"write"|"append"|"control", "view"? }. Returns{ "allow", "grantedModes", "governingAcl", "scope", "status", "aclLink" }. An allow is200; a deny maps the FR-6 status — a definitive one (resolvedwithout the mode /noAcl) is403, a retryable one (unloaded/transient) is503. When a governing ACL was discovered the response also carriesLink: <acl-iri>; rel="acl"(RFC 8288, FR-5, sq-snopa.7) — theaclLinkbody field holds the same value. Fail-closed: no Link header when no governing ACL exists.POST /authz/wac-allow— body{ "dataset", "session", "resource", "view"? }→{ "wacAllow": "user=\"…\",public=\"…\"" }, the RFC permission advertisement. Also emitsLink: <acl-iri>; rel="acl"when a governing ACL was discovered (FR-5, sq-snopa.7); omitted when none.POST /authz/query— body{ "dataset", "session", "mode"?, "query", "view"? }runs an access-controlled SPARQL query as the session and returns SPARQL-results JSON; a grant-less session sees ZERO rows (empty view), never the whole store.- ODRL lane (
odrl-authzcargo feature, default OFF; sq-lrtc3.1 + sq-3mu76): when the request dataset carries ODRL policy rules (odrl:permission/odrl:prohibition), the handler parses them from the UNION of the dataset's graphs and runs thesparq-solidbridge (PodStore::materialize_odrl_policy, BOTH sides, deny-overrides) for (party = session agent, action =odrl:read, target = each rule's target graph) BEFORE the query — an ODRL prohibition beats a static WAC grant through the unchanged∪ allow ∖ ∪ denyenforcement. The lane runs on all three endpoints (sq-3mu76): the advisory/authz/decideand/authz/wac-allownever report an allow/authz/querywould refuse to honour, and their advertisements are read-scoped while the lane evidences onlyodrl:read—grantedModesis masked to the requested mode,wacAllowcarries at mostuser="read"withpublic=""(anonymous is refused wherever the lane fires). Fail-closed 4xx refusals (never a silent allow): malformed policy, unimplementableodrl:conflictstrategy, a rule without a concrete target graph IRI (pattern targets = sq-lrtc3.3), a non-readmode(query-action contract = sq-lrtc3.2), an anonymous session, or atrustblock combined with an ODRL-carrying dataset on/decide(the trust dispatch would bypass the lane). Constraints the stateless lane cannot evidence (odrl:purpose,odrl:count— stateful budgets remain unimplemented) never grant. The lane reads the request BODY dataset, so it does not apply to"source":"server"; an ODRL-carrying server store is REFUSED there rather than un-enforced (see"source"below)./decidestays governing-ACL-scoped: with no discoverable.acl/.acrit returns itsnoAcldeny even where a bridged grant lets/authz/querysee rows (a deny-side-only divergence). Feature OFF ⇒ byte-identicalsolid-authzbehaviour (the standard build never compilessparq-policy).
- ODRL lane (
FAIL-CLOSED (the soundness invariant): every error path DENIES — an unparseable dataset is a
400 (never an empty-dataset allow), a materialisation failure a 503, an unknown mode a 403
deny, never a grant. Double opt-in, OFF by default: compiled only with the solid-authz cargo
feature and served only when --solid-authz / SPARQ_SOLID_AUTHZ=1 is set (mirrors shacl /
terse); with the feature but not the flag, /authz/* is 404. Read-gated by --auth-token-read.
Which pod: "source" (sq-snopa.8). Every endpoint takes an optional "source" field:
"body"(default, and the only mode before sq-snopa.8) — stateless: the pod dataset arrives in"dataset"and dies with the response. Unchanged behaviour."server"— stateful: authorise over the server's own loaded store. The body must NOT carry a"dataset"(a body naming both pods is ambiguous →400). A pod resource IS a named graph of the loaded store, so"resource"must be that graph's absolute IRI; relative paths are not resolved. The pinned current generation is forked, materialised once, and cached inAppStatekeyed by the generation number — so the N3 materialisation is paid once per generation, not once per request, and an.acl/.acrwrite re-materialises automatically (every commit publishes a new generation, which makes the cached view stale by construction; there is no window in which a revoked grant is still served).
The stateful lane is fail-closed on the same terms, and additionally refuses (never silently
un-enforces) two combinations it cannot evaluate faithfully: under odrl-authz, a server store
carrying ODRL policy rules (400 — dropping a prohibition would be fail-OPEN); under
solid-authz-trust, a request carrying a "trust" block (400 — that extension decides over its
own body-derived store).
curl -s -X POST http://127.0.0.1:3030/authz/decide -H 'Content-Type: application/json' -d '{
"source": "server", "session": { "agent": "https://alice.ex/card#me" },
"resource": "https://pod.ex/notes/n1", "mode": "read", "view": "wac" }'cargo run -p sparq-server --features solid-authz -- data.ttl --solid-authz
curl -si -X POST http://127.0.0.1:3030/authz/decide -H 'Content-Type: application/json' -d '{
"dataset": "<https://pod.ex/n1#it> <https://ex.dev/ns#t> \"hi\" <https://pod.ex/n1> .\n<https://pod.ex/.acl#o> <http://www.w3.org/ns/auth/acl#agent> <https://alice.ex/card#me> <https://pod.ex/.acl> .\n<https://pod.ex/.acl#o> <http://www.w3.org/ns/auth/acl#default> <https://pod.ex/> <https://pod.ex/.acl> .\n<https://pod.ex/.acl#o> <http://www.w3.org/ns/auth/acl#mode> <http://www.w3.org/ns/auth/acl#Read> <https://pod.ex/.acl> .",
"session": { "agent": "https://alice.ex/card#me" }, "resource": "https://pod.ex/n1", "mode": "read", "view": "wac" }'
# HTTP/1.1 200 OK
# link: <https://pod.ex/.acl>; rel="acl"
# → {"allow":true,"grantedModes":["read"],"governingAcl":"https://pod.ex/.acl","scope":"http://www.w3.org/ns/auth/acl#default","status":"resolved","aclLink":"<https://pod.ex/.acl>; rel=\"acl\""}6. Hardening — flags / env / library. Each flag overrides its SPARQ_* env var; the
env overrides the default.
| Flag | Env | Default | Effect |
|---|---|---|---|
--query-timeout SECS |
SPARQ_QUERY_TIMEOUT |
30 (0=off) |
per-request timeout → 503 |
--update-where-timeout SECS |
SPARQ_UPDATE_WHERE_TIMEOUT |
unset (0/unset = use --query-timeout) |
separate, typically-SHORTER writer-side WHERE deadline for SPARQL UPDATE — bounds writer-queue head-of-line blocking from a slow update (the single sequenced writer is released within this window instead of holding it for the full read timeout); the update WHERE budget is min(query_timeout, update_where_timeout) → slow update 503 (sq-nulp) |
--no-adaptive-commit |
SPARQ_ADAPTIVE_COMMIT (0 disables) |
adaptive on | adaptive group-commit: a serial interactive client commits in engine-time (µs) instead of paying a fixed group-commit window; concurrent load still batches. Pure latency change — FIFO/atomicity/durability unchanged. --no-adaptive-commit restores the always-windowed writer (sq-p7kk5) |
--max-body-bytes N |
SPARQ_MAX_BODY_BYTES |
1048576 |
body cap → 413 |
--max-concurrent N |
SPARQ_MAX_CONCURRENT |
32 |
in-flight cap, load-shed → 429 |
--header-read-timeout SECS |
SPARQ_HEADER_READ_TIMEOUT |
15 (0=off) |
slow-loris guard: max time a connection may take to send its complete request-header block — enforced at hyper's HTTP/1 connection layer by sparq_server::serve (NOT axum::serve, which never installs a timer so its header deadline is inert), so it fires BEFORE a handler and frees the concurrency slot a dribbling client would otherwise hold forever; connection closed when exceeded (sq-2gqr) |
--body-read-timeout SECS |
SPARQ_BODY_READ_TIMEOUT |
30 (0=off) |
slow-body guard (complement to --header-read-timeout): idle deadline between consecutive request-body reads — applied by sparq_server::serve via a tower_http RequestBodyTimeoutLayer. Closes the slow-body dribble a complete-header client could otherwise use (declare a large Content-Length, send a few bytes, stall) to hold the slot under --max-body-bytes and before --query-timeout starts. The timer RESETS after each received chunk, so an honest large upload is not penalised — only an idle stall is; body read fails → request aborted (sq-lodb) |
--max-results N |
SPARQ_MAX_RESULTS |
unlimited (0=off) |
result/solution cap (SELECT + CONSTRUCT/DESCRIBE WHERE-solutions + EXPLAIN ANALYZE; not ASK/GSP-read/UPDATE) → honest 413 (not truncation) |
--max-query-rows N |
SPARQ_MAX_QUERY_ROWS |
unlimited (0=off) |
memory cap (coarse): working-set ROW ceiling on every form → honest 413 (sq-ebii) |
--max-query-bytes N |
SPARQ_MAX_QUERY_BYTES |
unlimited (0=off) |
byte-accounted memory cap: prices working-set row WIDTH (rows × vars × id-size) + computed-literal bytes on every form → honest 413 (sq-s5is) |
--max-decompress-ratio N |
SPARQ_MAX_DECOMPRESS_RATIO |
20 (0=refuse gzip) |
zip-bomb guard: cap on decompressed:compressed for a Content-Encoding: gzip body → 413 (sq-ebii) |
--max-subscriptions N |
SPARQ_MAX_SUBSCRIPTIONS |
256 |
server-wide subs |
--max-subscriptions-per-conn N |
SPARQ_MAX_SUBSCRIPTIONS_PER_CONN |
16 |
per-socket subs |
--verbose |
— | off | TraceLayer request logging (respects RUST_LOG); request URIs redacted by default — see "Request-log redaction" |
--log-full-requests |
SPARQ_LOG_FULL_REQUESTS |
off (redaction ON) | (sq-toze.34) OPT OUT of request-log redaction: log the raw request URI (incl. the full ?query= SPARQL text) verbatim. Inert without --verbose — see "Request-log redaction" |
--auth-token TOKEN |
SPARQ_AUTH_TOKEN |
off (no auth) | require Authorization: Bearer TOKEN on every WRITE (SPARQL Update + GSP PUT/POST/DELETE) → 401 + WWW-Authenticate: Bearer otherwise; constant-time compared (QLever's -a) |
--auth-token-read |
SPARQ_AUTH_TOKEN_READ |
off | ALSO gate reads with the same token (only meaningful with a token set) |
--allow-remote |
SPARQ_ALLOW_REMOTE |
off | opt in to a non-loopback bind; without it a non-loopback --addr is refused unless the surface is fully authenticated (--auth-token AND --auth-token-read), with it it warns and proceeds |
--service-allow HOST[:PORT]|*.SUFFIX[:PORT] (repeatable) |
SPARQ_SERVICE_ALLOW (comma/ws-sep) |
empty = deny ALL SERVICE | (feature service) allowlist a SERVICE egress host (exact or *.suffix wildcard); a :PORT makes it port-scoped — that host on THAT port only (else every port) (sq-a7jw4); CLI + file + env are all merged (combined additively) |
--service-allow-file PATH |
— | — | (feature service) load allowlist entries, one per line (# comments + blanks ignored) |
--cors-allow-origin ORIGIN (repeatable) |
SPARQ_CORS_ALLOW_ORIGIN (comma/ws-sep) |
empty = no CORS headers | allowlist a first-party browser origin (scheme://host[:port]); a listed Origin is reflected into Access-Control-Allow-Origin (never *, never credentials) + Vary: Origin, preflight OPTIONS answered; CLI + file + env merged additively — see "CORS" |
--cors-allow-origin-file PATH |
— | — | load CORS origins, one per line (# comments + blanks ignored) |
--time-travel-generations N |
SPARQ_TIME_TRAVEL_GENERATIONS |
16 |
(feature) retained generations |
--time-travel-max-age SECS |
SPARQ_TIME_TRAVEL_MAX_AGE |
off | (feature) age-out window |
--federation-descriptors |
SPARQ_FEDERATION_DESCRIPTORS |
off | (feature federation-descriptors) serve a VoID at /.well-known/void + a SPARQL Service Description on GET /sparql with no query — see "Federation discovery" |
--facets |
SPARQ_FACETS |
off | (feature facets) serve POST /facets grouped type/predicate/value counts over a pinned snapshot; read-gated — see "Grouped facet counts" ([GPT-5.6] sq-lsp7k.5.2) |
--complete |
SPARQ_COMPLETE |
off | (feature complete) serve GET /complete?q=<prefix>&limit=<k> IRI/local-name/label prefix completion from a generation-cached index; read-gated — see "IRI/label prefix completion" ([GPT-5.6] sq-lsp7k.9.3) |
--tpf |
SPARQ_TPF |
off | (feature tpf) serve a Triple Pattern Fragments / LDF source endpoint at GET /tpf?subject=&predicate=&object= (paged, full Hydra paging incl. first/last, read-only); same flag also serves brTPF bind-restricted fragments (values param / POST body) when built with the brtpf feature — see "Triple Pattern Fragments" |
--shacl |
SPARQ_SHACL |
off | (feature shacl) serve the SHACL validate endpoint POST /shacl/validate — POST a shapes graph, the server validates its loaded data graph against it; JSON report (default) or W3C report Turtle (Accept: text/turtle); read-only — see "SHACL validation endpoint" |
--shacl-guard |
SPARQ_SHACL_GUARD |
off | (feature shacl) reject non-conforming UPDATE/GSP post-states with 422 + JSON validation report; store unchanged |
--shacl-shapes FILE |
SPARQ_SHACL_SHAPES |
unset | (feature shacl) load the guard shapes graph once at startup; required when the guard is on |
--solid-authz |
SPARQ_SOLID_AUTHZ |
off | (feature solid-authz) serve the Solid WAC/ACP authorization endpoints POST /authz/decide+/wac-allow+/query — a fail-closed HTTP shell over sparq-solid; POST the pod dataset (or "source":"server" for the server's own loaded store, sq-snopa.8) + an already-resolved session, get the decision / WAC-Allow value / access-controlled query result; read-only — see "Solid WAC/ACP authorization endpoints" |
--solid-authz-trust |
SPARQ_SOLID_AUTHZ_TRUST |
off | (feature solid-authz-trust, implies solid-authz) opt-in stateless trust-graph extension to POST /authz/decide — a request may carry an additional "trust" JSON block containing credentials, a trust policy, and signed certification edges; the server runs the cert-graph closure (derive_effective_rules) and the sparq_trust::admit gate over them, injects any admitted facts into the pod dataset, then runs the unchanged WAC/ACP decision; double-opt-in: the feature must be compiled AND this flag set AND the request must carry a "trust" block — see "Stateless trust-graph decision extension (sq-pfae.17)"; honest scope: anchored-not-proven clear-path only (no ZK/unlinkability claim; sq-qhy4 external audit PENDING) |
--brtpf-max-bindings N |
SPARQ_BRTPF_MAX_BINDINGS |
1024 (0=off) |
(feature brtpf) DoS cap on the brTPF binding-set mapping COUNT — one index scan per mapping, so cost is super-linear in the count, not the bytes → 413 (sq-r74h) |
--brtpf-max-values-bytes N |
SPARQ_BRTPF_MAX_VALUES_BYTES |
1048576 (0=off) |
(feature brtpf) DoS cap on the raw brTPF values payload BYTES — bounds the GET query-string carrier that --max-body-bytes never sees → 413 (sq-r74h) |
--audit-log |
SPARQ_AUDIT_LOG |
off | (feature audit-log) per-query access audit log — see "Access audit log" |
--access-audit <file|stderr> |
SPARQ_ACCESS_AUDIT |
off | (feature access-audit) richer structured access-audit sink (typed JSON-Lines: actor / action / resource / decision+basis / ts / fingerprint) — see "Structured access-audit sink" |
--audit-webid-header <name> |
SPARQ_AUDIT_WEBID_HEADER |
off | (feature access-audit) trust a fronting auth layer's forwarded WebID header so the audit actor is the real authenticated subject (webid:<iri>) — set ONLY behind a trusted front (spoofable otherwise); see "Structured access-audit sink" |
In a library: AppState::with_config(graph, ServerConfig { max_concurrent: 64, ..Default::default() })
then router(state), or harden(my_router, &config).
The threat model (a public, unauthenticated endpoint behind a gateway) calls for these distinct limits. Be precise about what each bounds — only the body-size and ratio caps are byte-hard; the timeout and both memory caps are cooperative (approximate in time), and the memory caps are coarse working-set ceilings (row count / estimated bytes), not an RSS quota:
- Query timeout (
--query-timeout,SPARQ_QUERY_TIMEOUT, default30s,0=off). The engine's cooperativeQueryBudget.deadlinestops the worker at its next coarse check (operator entry / per outer loop iteration); a wall-clock hard cap oftimeout + 2sgrace guarantees the HTTP503even if the engine is mid-stretch. Applies to all forms now — SELECT / ASK / CONSTRUCT / DESCRIBE / GSP-read and SPARQL Update (application/sparql-update+ GSPPUT/POST/DELETE): the update path runs under the same cooperative budget on the writer thread and the same wall-clock await cap on the HTTP side. Bounds: wall-clock per request, approximately (next-check granularity). Caveat: updates are sequenced on a single writer, so a long update blocks the queue behind it until it finishes — this cap bounds the client's own wait, while the writer runs the WHERE to its cooperative stop. To bound that head-of-line blocking of the queue, set a separate, shorter WHERE deadline — see (1b).- 1b. UPDATE writer-side WHERE deadline / head-of-line bound
(
--update-where-timeout,SPARQ_UPDATE_WHERE_TIMEOUT, default unset = use--query-timeout;sq-nulp). A SEPARATE, typically-shorter cooperative deadline applied ONLY to the WHERE phase of a SPARQL UPDATE on the writer thread: the update's budget deadline becomesmin(query_timeout, update_where_timeout), so a slow update releases the single sequenced writer within this window instead of holding it for the full (usually longer) read timeout — bounding how long one slow update can head-of-line block every queued update behind it. Cooperative, like the query timeout (next-check granularity); a tunable backstop, not hard preemption. Unset ⇒ the update WHERE budget is exactly--query-timeout(unchanged). The offending update gets a503.
- 1b. UPDATE writer-side WHERE deadline / head-of-line bound
(
- Memory cap (
--max-query-rows,SPARQ_MAX_QUERY_ROWS, default off,0=off). A coarse OOM circuit-breaker: an upper bound on the row count of any materialised intermediate or final result the engine builds, on every form (including CONSTRUCT/DESCRIBE and an UPDATE'sDELETE/INSERT … WHERE), via the engine'sQueryBudget.max_rowsworking-set bound. A join blow-up aborts with an honest413instead of OOMing; the speculative cross-product allocation is also capped up-front. Bounds: *cardinality (rows), not bytes. Peak heap ≈rows × per-row term cost, so a query with few but very wide rows (many vars / huge literals) can still exceed the implied memory; dictionary growth, sort/group scratch and a CONSTRUCT template are outside it. It is also approximate in time (coarse checks). Treat it as a blunt anti-OOM breaker, not an RSS quota. Distinct from--max-results(the result/solution cap, folded into the budget on SELECT / CONSTRUCT/DESCRIBE / EXPLAIN ANALYZE — but not ASK / GSP-read / UPDATE); on a path where both apply, the effective cap is the tighter of the two. For the row cap's width/literal blind spots, see the byte-accounted companion (2b); writer-queue head-of-line blocking from a slow UPDATE is deferred (sq-nulp). 2b. Byte-accounted memory cap (--max-query-bytes,SPARQ_MAX_QUERY_BYTES, default off,0=off;sq-s5is). The byte-accounted twin of (2): instead of counting ROWS it costs the estimated working-set BYTES —rows × width × size_of::<Id>()for each materialised intermediate (so it prices the WIDTH the row cap is blind to) PLUS the bytes of query-COMPUTED terms (BIND / aggregate / CONSTRUCT scratch interned into the per-query local vocab — the non-row allocations the row cap misses). Enforced viaQueryBudget.max_byteson every form (incl. an UPDATE's WHERE), at the same coarse cooperative sites, honest413on overflow. Bounds: the QUERY working set, estimated as a portable lower bound on real heap (ignores allocator overhead,SmallVecinline-vs-spill, and the pre-existing dictionary/index memory) — so it is strictly tighter and more width/literal-aware than the row cap, but still a coarse circuit-breaker, not an exact RSS quota. Composes with (2) and--max-results: whichever ceiling trips first aborts. - Decompression-ratio cap (
--max-decompress-ratio,SPARQ_MAX_DECOMPRESS_RATIO, default20×,0=refuse gzip). When a GSP write body arrivesContent-Encoding: gzipthe server inflates it with a hard ceiling ofmin(ratio × compressed_len, max_body_bytes), checked during inflate (boundedRead::take), and refuses with413the moment the decompressed output would cross it — so a tiny but pathologically compressible body cannot inflate into an OOM.0refuses gzip bodies outright (fail-closed). Bounds: the bodies the server itself inflates (GSPPUT/POST). An unknownContent-Encodingis a415. Caveat: it does not cover a compressed payload the engine fetches behind a SPARQLLOAD <url>/SERVICE— those use their own ingest;SERVICEegress is bounded separately by limit 4. The compressed bytes pass the--max-body-bytesgate first, and the decompressed ceiling ismin(ratio × compressed_len, max_body_bytes)— so the decompressed output is itself capped atmax_body_bytes, nevermax_body_bytes × ratio. - SERVICE-SSRF egress allowlist (
--service-allow/--service-allow-file/SPARQ_SERVICE_ALLOW, default deny ALL, featureservice) — shipped insq-4w18, see "SERVICE federation (egress allowlist)" below. ASERVICE <iri>turns attacker-controlled query text into an outbound request from the server host (textbook SSRF; worst case the169.254.169.254cloud-metadata IP), so it reaches nothing unless its host is allowlisted, enforced on the resolved IP before any socket opens (DNS-rebinding-safe), uniformly across queries / ASK / CONSTRUCT/DESCRIBE / subscriptions / federatedINSERT … WHERE. An entry may be port-scoped (host:port) to permit a host on exactly one port — strictly narrower than the bare host (sq-a7jw4).
Library callers set these on ServerConfig (query_timeout, max_query_rows,
max_query_bytes, max_decompress_ratio, service_allow). Embedders driving the engine
directly thread a sparq_engine::QueryBudget { deadline, max_rows, max_bytes, ..QueryBudget::unlimited() } into
*_with_budget query entry points and update_in_place_with_budget, and wrap calls in
sparq_engine::with_service_egress_policy(strict, [host], || …).
sparq-server is a SPARQL data API, so by default it emits no CORS headers — a
cross-origin browser fetch cannot read its responses (the safe posture). Do nothing and you
keep that. For a first-party browser app on another origin, allowlist its exact origin(s):
cargo run -p sparq-server -- --cors-allow-origin https://app.example.org data.ttl
SPARQ_CORS_ALLOW_ORIGIN='https://app.example.org, http://localhost:5173' \
cargo run -p sparq-server -- data.ttl
# or one origin per line: --cors-allow-origin-file ./cors-origins.txtEach entry is an RFC 6454 origin scheme://host[:port]. CLI + --cors-allow-origin-file + env
merge additively (same precedence as the SERVICE allowlist). When non-empty, a middleware in
harden() reflects an allowlisted request Origin into Access-Control-Allow-Origin (+
Vary: Origin) and answers the OPTIONS preflight for it; an un-listed origin gets no
CORS header (browser blocks it). Deliberately conservative: never Access-Control-Allow- Origin: *, never Access-Control-Allow-Credentials, a * entry is rejected at startup. It
is a browser-read gate only — it does not relax the Bearer-token auth gate, the bind
posture, the body limit, the SERVICE allowlist, or the row caps (an allowlisted browser still
needs the token to write). Library callers set ServerConfig.cors_allow: CorsAllowlist
(re-exported at the crate root).
A per-query access audit log for compliance regimes that need a per-subject / per-query
trail (CDMC CD-2, ISO 27001 A.8.15, EU CRA logging) — distinct from the aggregate-only
/metrics. Doubly opt-in: compile with the audit-log cargo feature, then turn it on at
runtime with --audit-log (env SPARQ_AUDIT_LOG=1). Off (either), the module and every call
site are #[cfg]-stripped or short-circuited before any record is built — a request pays
essentially zero.
For each query / update / Graph-Store request the server emits one structured tracing
event under the dedicated target: "sparq_server::audit" (tracing::info!). Route it to your
sink with the standard RUST_LOG machinery, independently of the --verbose request log:
RUST_LOG=sparq_server::audit=info. (--audit-log installs a subscriber on its own if
--verbose did not.) Fields:
| field | meaning |
|---|---|
requester |
anonymous, or token:<fnv1a-hash> of the presented Bearer token — never the raw token |
op |
query / update / graph_read / graph_write (operation class, keyed on whether it mutates) |
fingerprint |
FNV-1a hash (hex) of the trimmed query/update text — not the query text |
decision |
allowed or denied |
reason |
denial reason (auth: missing or invalid Bearer token); empty when allowed |
status |
the HTTP status the client saw (200 / 401 / 413 / …) |
rows |
result-row count when known (else absent — status is the authoritative outcome) |
duration_us |
handler wall-clock in microseconds |
No-PII / no-info-leak posture (reuses the #241 lesson): this is a server-side log under
the operator's control — it is NEVER written to the HTTP response. It deliberately does not
log the full query text (raw SPARQL can disclose loaded-data fragments or caller PII — the #241
contract) nor the Bearer secret; only a stable, non-reversible fingerprint of each, enough to
correlate repeated identical queries / a recurring caller across requests and restarts. Library
callers set ServerConfig { audit_log: true, .. } (field present only with the feature). See
crates/sparq-server/src/audit.rs.
--verbose installs tower_http::trace, whose default span records the request URI — and
for GET /sparql?query=… the full SPARQL query text is in that URI, where it can carry PII (a
patient IRI, an email in a FILTER, a literal in INSERT DATA). Logging it verbatim leaks
sensitive content into operator logs.
Redaction is ON by default (always compiled — no feature gate): with --verbose the request
log keeps the URI path verbatim but replaces the query string with ?<redacted len=N fp=…>
— a length signal + a stable, non-reversible FNV-1a fingerprint (the same construction the audit
log uses). Logs stay correlation-useful (same query → same fp) without exposing content.
--log-full-requests (env SPARQ_LOG_FULL_REQUESTS=1) opts OUT and logs the URI verbatim,
as the bare TraceLayer did — the deliberate debug escape hatch. Library: ServerConfig { verbose: true, redact_logs: true /* default */, .. }.
Default rationale: enabling --verbose for debugging should not silently write
potentially-sensitive query text to disk / a SIEM; content-logging is the deliberate choice.
Honest boundary — log-CONTENT redaction, not anonymity: the log still records method,
path/endpoint, status, a size signal and timing (it would not be a request log otherwise), so
an adversary still learns that a request of roughly-this-size hit this endpoint at this
time and (via fp) that the same query recurred. That metadata is not erased. It is also NOT
the ZK/MPC privacy story — purely operator-log hygiene, complementary to error-body sanitisation
(sq-kfel/#241) and the audit fingerprint. See crates/sparq-server/src/redact.rs.
Structured access-audit sink — opt-in pluggable JSON-Lines trail (sq-gos8, epic sq-toze, ASVS V7 / ISO 27001 A.8.15 / CDMC CD-2)
A richer, structured sibling of the audit-log trail above, for compliance audit trails that
need a TYPED, self-describing access record per ENFORCED decision rather than a flat tracing
line. Opt-in: compile with the access-audit cargo feature, then configure a sink with
--access-audit <file|stderr> (env SPARQ_ACCESS_AUDIT; the literal stderr writes to stderr,
any other value is a file path). Off (no feature, or no sink configured), the module + every call
site are #[cfg]-stripped / short-circuited (Option check) — a request pays essentially zero.
It hooks the real enforcement seam (the same auth_gate that actually allows/denies the
request), so the recorded decision is the one the server enforced — never a claimed-but-
disconnected one. Each event is emitted through a pluggable AuditSink trait (the default
WriterSink writes one JSON object per line; heavy/external sinks — a SIEM client, an OTel
exporter — stay OUT of core, an embedder implements the trait and installs an
Arc<dyn AuditSink>). Record fields:
| field | meaning |
|---|---|
ts |
RFC-3339 UTC timestamp (YYYY-MM-DDTHH:MM:SS.mmmZ) |
actor |
anonymous, token:<fnv1a> (Bearer fingerprint, never the raw token), or webid:<iri> (an authenticated WebID/agent IRI — recorded verbatim) |
action |
query / update / graph_read / graph_write |
resource + resource_kind |
the dataset (/sparql) or the named-graph IRI the request addressed (named_graph) |
decision |
allow / deny (the ACTUALLY-enforced outcome) |
policy_basis |
the enforcement reason (bearer-auth: allowed / bearer-auth: missing or invalid token) |
fingerprint |
FNV-1a hash (hex) of the trimmed query/update — not the query text (- for a GSP body) |
status |
the HTTP status the client saw |
duration_us |
handler wall-clock, microseconds |
Privacy boundary — stated honestly: an audit trail exists to record WHO accessed WHAT, so —
by design, and unlike the request log — this sink records identities and resource IRIs
(the actor + the named-graph IRI are first-class fields; that is the operator's deliberate opt-in
choice). What it does NOT record is query CONTENT: the query/update text is logged only as
its non-reversible fingerprint, never raw, because a query body can carry PII (a patient IRI in a
FILTER, an email literal) — the #241 / sq-toze.34 redaction posture. It does not double-log
the content the redaction work just protected. One line: identities + resources are logged;
content stays fingerprinted. Library callers set ServerConfig { access_audit: Some(SinkTarget::File(path)), .. } (field present only with the feature). See
crates/sparq-server/src/access_audit.rs.
Actor enrichment from a trusted front (sq-ljfz). By default the actor is derived from the
local Bearer gate (token:<fnv1a> / anonymous) — sparq-server's own auth is a single shared
secret, not per-user. To record the real authenticated WebID instead, run the server behind a
fronting authorization layer — sparq-solid, or any Solid/WAC reverse-proxy / identity gateway
(the "reverse proxy / gateway or sparq-solid" the bind warnings name) — that authenticates the
user (resolves their WebID from a WAC/ACP session) and forwards it in a request header, then name
that header with --audit-webid-header <name> (env SPARQ_AUDIT_WEBID_HEADER, library field
ServerConfig { audit_webid_header: Some(name), .. }). When set, a present non-empty value of that
header becomes actor = webid:<iri> (the audit attributes access to who, not to which shared
secret); empty/absent falls back to the Bearer gate, byte-identical to before. Security: a
forwarded header is client-spoofable, so this is honoured ONLY when an operator explicitly names a
trusted header — set it only when the server runs behind that trusted front (never exposed
directly to untrusted clients), and ensure the front sets/overwrites the header so a direct client
cannot inject an arbitrary WebID. With no trusted header configured (the default) any such header
is ignored.
Composing the two audit sinks (sq-bif.14). audit-log (a process-global tracing sink) and
access-audit (a per-AppState JSON-Lines file sink) can be compiled in and turned on at runtime
TOGETHER. They hook the SAME request handler, so a single request emits BOTH records. The
load-bearing invariant — verified by tests/audit_composition.rs (gated on
--features audit-log,access-audit) — is that the two independently-derived records AGREE on the
request's operation, the ACTUALLY-enforced decision, the HTTP status, and the NON-reversible query
fingerprint (both sinks share the same FNV-1a-over-trimmed-text fingerprint), and that the
redaction boundary holds in BOTH (neither writes the raw query text or the Bearer token, and the
fingerprint never leaks to the HTTP response body) for an allowed AND a denied request. This is a
redaction-posture invariant, not an unqualified security guarantee — the audit trail records
identities and resource IRIs by design (see the privacy-boundary note above).
- Auth — optional Bearer write gate; loopback-by-default; non-loopback bind refused
unless safe. With no
--auth-tokenthe server is unauthenticated (read+write open to anyone who can reach the port) — the back-compat default. Set--auth-token <TOKEN>(envSPARQ_AUTH_TOKEN) to requireAuthorization: Bearer <TOKEN>on every WRITE (SPARQL Update- GSP
PUT/POST/DELETE);401+WWW-Authenticate: Bearerotherwise (constant-time compared; QLever's-a <token>; missing-vs-wrong are indistinguishable). The classification keys on whether the request mutates, not the route — an Update smuggled through the query path is gated too. Add--auth-token-read(envSPARQ_AUTH_TOKEN_READ=1) to ALSO gate reads — INCLUDING the subscription transports (/subscriptionsWS +/subscriptions/sseSSE, both a read surface), closing the prior read-auth bypass (beadsq-cxk5): the SSE GET takes theAuthorization: Bearerheader; the WS upgrade accepts that header OR (for browsers) aSec-WebSocket-Protocol: bearer.<token>subprotocol. The binary binds127.0.0.1:3030by default and refuses a non-loopback--addr(incl.0.0.0.0/::) unless--allow-remote/SPARQ_ALLOW_REMOTE=1OR the whole surface is authenticated (--auth-tokenAND--auth-token-read); even then it warns. A write-token alone still leaves reads open, so it does NOT by itself make a remote bind safe. Deliver the token over TLS (terminate at a proxy). For per-user authz front it with a reverse proxy / gateway (orsparq-solid). The token is authentication, NOT a resource cap: it is orthogonal to the DoS caps. No rate limit;--max-results/--max-query-rowsare unlimited by default — set the four hardening caps (timeout, memory, decompression-ratio, SERVICE allowlist; see "Server hardening — the four DoS/SSRF limits") plus a gateway rate limiter before exposing it (beadssq-zcby,sq-o4qf,sq-ebii,sq-4w18).
- GSP
- SERVICE federation (egress allowlist).
SERVICEis OFF in the default build (build with--features serviceto enable it). Even enabled, the server is default-DENY-all SERVICE: aSERVICE <iri>clause reaches nothing unless its host is allowlisted — via--service-allow HOST/*.SUFFIX(repeatable),--service-allow-file PATH(one entry per line), orSPARQ_SERVICE_ALLOW(comma/whitespace-separated). All three are combined additively (the CLI only ever widens the env baseline). Rationale: aSERVICEclause turns attacker-controlled query text into an outbound request from the server host (textbook SSRF; worst case the169.254.169.254cloud-metadata IP), so the network-exposed surface must opt in to every reachable host. Matching is case-insensitive against the SERVICE IRI authority; a*.example.orgentry matches the apexexample.organd any subdomain. An entry may be host-level (192.0.2.10— every port) or port-scoped (192.0.2.10:8080,[::1]:8053,sparql.example.org:8443,*.example.org:443— that host on THAT port ONLY, rejecting every other port) (sq-a7jw4): a port-scoped entry is strictly NARROWER — the way to permit exactly one ephemeral loopback endpoint for in-process SERVICE federation without re-opening the whole host. (This TIGHTENS earlier behaviour: ahost:portentry used to have its port stripped and widen to every port; it now means EXACTLY that port. A port-LESS entry is unchanged — still every port.) Unlike the engine's standalone default (which lets public IPs through and only blocks private ones), the server is strict: even a public host must be on the allowlist. The allowlist applies uniformly to queries, ASK, CONSTRUCT/DESCRIBE, subscriptions and federatedINSERT … WHEREupdates, and is enforced before any socket is opened (DNS-rebinding-safe, on the resolved IP). The startup log prints the effective allowlist. A non-SILENTSERVICEthat is refused is an honest403 Forbidden(sq-iu0c) — a policy refusal, distinct from the500an unclassified execution error gives, so clients and alerting can tell a blocked egress target apart from a real server fault (the refused host is sanitized out of the body and goes only to the server log).SERVICE SILENTstill swallows the refusal to the empty relation (no403). Beadssq-4w18(this wiring),sq-2v6f(the engine SSRF filter),sq-iu0c(the403classification). Embedders that drive the engine directly usesparq_engine::with_service_egress_policy(strict, [host], || …)/with_service_egress_allow([host], || …).- Per-request / per-query egress override (
sq-9xoh, featureservice). A multi-tenant or gateway deployment can make the reachable SERVICE host set depend on the request instead of the single static allowlist: setServerConfig::service_allow_overrideto aServiceAllowOverride::new(|headers| …)hook (aFn(&HeaderMap) -> Option<ServiceAllowlist>,Send + Sync). It is invoked once per read request (SELECT / ASK / CONSTRUCT / DESCRIBE / EXPLAIN ANALYZE); returningSome(allowlist)substitutes that allowlist for that one request (installed in the same STRICT/allowlist-only mode — an empty returned allowlist therefore denies all SERVICE for the request), andNonefalls back to the staticservice_allow.None(the default) keeps the historical single-allowlist behaviour exactly. The hook can only narrow or substitute the host set — it can never relax the fail-closed posture. It is not applied to the SPARQL-Update writer path: updates are sequenced and group-committed on one shared writer thread with no per-request header context (batch-mates may carry different tokens), so federatedINSERT … WHEREupdates continue to use the operator's staticservice_allow.
- Per-request / per-query egress override (
- Feature flags.
server(default-on) pulls axum/tokio/tower — the binary needs it (required-features = ["server"]).?generation=Npinning + theSparq-Generationheader are in the default build (bounded to the ring's concurrency-retention window —sq-ci2d6);time-travel(default off) only EXTENDS the retention window (the--time-travel-generations/--time-travel-max-ageflags,AppState::at, and the wider?generation=Nreach).geo(default off) installs sparq-geo'sgeof:GeoSPARQL functions on query/update/subscription paths; without it an unknowngeof:IRI is a500.federation-descriptors(default off,sq-d3d8) pulls the lightsparq-introspectcrate and serves the OPT-IN VoID + Service-Description discovery endpoints (still gated at runtime by--federation-descriptors; see "Federation discovery").facets(default off, [GPT-5.6]sq-lsp7k.5.2) uses the same lightsparq-introspectdependency forPOST /facets, still gated at runtime by--facets.complete(default off, [GPT-5.6]sq-lsp7k.9.3) pullssparq-textwith its default features disabled and servesGET /complete, still gated at runtime by--complete. Run feature tests:cargo test -p sparq-server --features time-travel/--features geo/--features federation-descriptors/--features facets/--features complete. - Named graphs are real (since conformance round 3). The engine stores the FULL dataset
— default graph + named graphs — so
GRAPH <g> { … }/GRAPH ?g { … }evaluate, and a GSP graph resource (?graph=<iri>or the direct request URI) addresses a genuine named graph (no longer a default-graph alias).FROM/FROM NAMEDre-scope the active dataset, and the protocol's dataset-override params are applied (sq-z33x), not just accepted:default-graph-uri/named-graph-urisynthesize the query's active dataset (replacing any in-queryFROM/FROM NAMEDper §2.1.4), andusing-graph-uri/using-named-graph-urire-scope an update'sWHERE(per §2.2; combining with an in-updateUSING/USING NAMED/WITHis a400, as is a non-IRI graph value). Time-travel pinning is/sparqlqueries only (GSP read/write and subscriptions always operate on current). - Update operations. Engine handles
INSERT DATA,DELETE DATA,CLEAR/DROP/CREATE(DEFAULT / named / ALL),LOAD, andDELETE/INSERT … WHERE— over the default graph AND named graphs. A failing operation is refused with400, atomically (no partial effect published).apply_updateblocks (group-commit + O(graph) fork) — never call it on the async runtime directly; the HTTP handler (and the GSP write verbs) already usespawn_blocking. - Multi-operation update bodies are accepted and applied ATOMICALLY (one request, one
commit). A SPARQL 1.1 Update request is a sequence of
;-separated operations, and the endpoint takes the WHOLE body as ONE update (it is never split on;): e.g. a Solid PSSputDocumentbody —DROP SILENT GRAPH <r> ; INSERT DATA { GRAPH <r> … ; GRAPH <parent> ldp:contains <r> }— is one request → a single204, with the resource graph AND the parent containment triple either BOTH applied or (on any operation failing) NEITHER. The sequenced writer applies the body to a private fork and publishes only on full success, so a partial body is never visible (in-memory all-or-nothing). On--persistthe whole body's resolved delta is committed as ONE fsync'd transaction-journal frame BEFORE the204, so a crash mid-body can never leave the parentldp:containsdesynced from the child graph it points at (the journal redoes the whole frame, or none of it, onGraph::open). (sq-ycle / gh-48; seecrates/sparq-server/tests/persist.rs::pss_combined_multiop_body_accepted_and_atomicand::invalid_second_op_leaves_no_partial_write.) - GSP write created-vs-replaced status is advisory. PUT/POST sample graph existence from
the current generation to choose
201vs204/200; the write itself is atomic on the sequenced writer regardless. An existing-but-empty named graph reads as absent (the engine has no separate "empty graph exists" bit outside an in-flight update), so it may report201on a write — this never affects correctness of the data, only the status code. - Durability — opt-in via
--persist DIR(default in-memory). With NO--persistthe server is in-memory and updates are lost on restart (the back-compat default). Pass--persist <DIR>(envSPARQ_PERSIST_DIR) to make the on-disk index atDIRthe durable, rebuildable source of truth (QLever's--persist-updates): every committed UPDATE — default graph AND named graphs — is write-ahead-logged + fsync'd before the group-commit ack (the204), so a process restart on the sameDIRrestores all prior updates with no rebuild (Graph::openreplays the WAL). On startup an existing store atDIRis opened (its WAL replayed; anyDATA_FILEseed ignored — the persisted store wins); an emptyDIRis seeded fromDATA_FILE. Library callers setServerConfig::persist_dirand build withAppState::try_with_config(returns the durable-open error). Deferred hardening (beaded): byte-accounted durability metrics, graceful degradation on a transient disk error (today a durability failure refuses the write rather than losing it), and WAL-durableCLEAR/DROP GRAPH <g>of an existing named graph. (sq-7cxr / gh-44.) The fail-closed contract under a durable-write failure is regression-guarded over HTTP bytests/persist.rsvia thetest-seams-only seal seam (AppState::with_config_inject_durable_failure): a refused write returns503and is NEVER published, the writer thread survives, reads keep serving the last published snapshot, and a recovered write is durable across a restart — covered for a transient burst, a permanent jam, AND (sq-bif.14) an INTERLEAVED recover→fail→recover sequence whose final durable set is exactly the successful writes (no refused write resurrects). - Online backup/restore — opt-in
backupfeature (default OFF). Build with--features backupto mount the WRITE-gated, POST-only/admin/backup(stream a self-describing snapshot artifact of the live store WITHOUT stopping the world) and/admin/restore(atomically install a store rehydrated from one, fail-closed on a corrupt/mismatched artifact). On an in-memory server the restore is RAM-only; on a--persistserver it is409UNLESS you opt in with?persist=true, which writes the restore THROUGH to the durable dir (crash-safe, fail-closed) so it survives a restart (sq-ft7u).--restore <FILE>/SPARQ_RESTOREruns the same restore on start (bootstrap a fresh replica / PITR base); with--persistit is refused unless--restore-persist/SPARQ_RESTORE_PERSIST=1is set (then it writes through to the durable dir on start). DISTINCT from the offlinesparq-cli saveand from the--persistWAL. At-rest encryption of the artifact is out of scope — the body digest detects accidental corruption, not tampering. Without the feature the routes + the--restoresetting are compiled out AND the serving core stays the plainring/writerpair — theArcSwap<ServingCore>that backs the atomic online restore isbackup-gated, so the DEFAULT read path is byte-identical to before #941 (no extra atomic load whenbackupis OFF; sq-0g6g resolved in the lean direction). (sq-o5bi.) - Incremental change-stream / PITR — same
backupfeature (sq-bu1a). The base backup above is the BASE half of point-in-time recovery;/admin/backup/delta?from=Nstreams an incremental delta artifact (distinctSPARQ-BACKUP-DELTAkind) — the quad-set change between a retained generation N and the current one, keyed off the generation/writer-seq range + the epoch vector atto. Recover to a chosen point by restoring the base then replaying the chain forward:--restore base --restore-delta d0 --restore-delta d1 …(oldest first; orSPARQ_RESTORE_DELTA). Fail-closed on a corrupt / out-of-order / gapped chain (import + full replay before any swap);fromnot retained is410(widen retention withtime-travel). Same-lineage only — the chain must be the writer history of that base (the diff relies on lineage-stable blank-node labels). At-rest encryption out of scope, same as the base. - Time-travel memory cost is real. Each retained generation is a full logical
Graphsnapshot, but not an independent copy of the store: today'sGraph::forkshares the base's immutable storage structurally (the permutation indexes, planner stats, the dictionary's frozen base and the numeric/temporal caches areArc-shared) and copies only the pending delta. So the extra memory a generation costs grows with how far it has diverged from the base it shares — and a compaction folds the overlay into a fresh base, after which generations retained from before it hold the older base resident on their own. Size--time-travel-generationsconservatively — retain only as many generations as your deployment's memory budget allows. Seeresearch/concurrent-serving.mdfor architecture details. - Error bodies. Every error is structured JSON
{"error": "..."}withContent-Type: application/json(the405keeps itsAllowheader). POST query requiresContent-Type: application/sparql-queryorapplication/x-www-form-urlencoded(else415); a GET withoutquery=is400. An unmatched route is a404with the categorised body{"error":"not found"}([OPUS-4.8] sq-pj6u — previously the bare{"error":""}); the message is server-constructed and never echoes the requested path. - Transient vs permanent status contract (for retry classifiers — sq-r5bv / gh-50). A retry
classifier should treat only
429and503as transient (a retry of the identical request may succeed):429is a concurrency shed (the request never ran),503is a query/UPDATE timeout, a durable-write refusal (write NOT applied), or a subscription-capacity refusal. Everything else is permanent for the identical request —400/401/404/405/410/413/415— and500is a defect (caught panic / unclassified internal error), not back-pressure. The trap a5xx-only classifier hits against sparq: a413result/row cap is a PERMANENT honest refusal (narrow the query / addLIMIT), not a transient signal and not a truncation. Classify on the status code, not the body text (bodies are sanitised generic classes — see the next bullet). There is noRetry-Afterheader today. Full contract + rationale: thesparq_server::status_contractcrate doc, asserted bytests/status_contract.rs. - The versioned HTTP wire contract ([FABLE-5] sq-fdurb / gh-1416, PSS ask). The endpoints,
params, media types, negotiation rules, status codes and error-body shape an HTTP-only
consumer may rely on are enumerated as the v1 wire contract in
docs/http-wire-contract.md— frozen-vs-unstable partition, plus the wire-semver policy (breaking vs additive). Pinned end-to-end by the served-surface snapshot suitetests/wire_contract.rs(one direct test per documented endpoint behaviour / error class), so an accidental wire break fails CI. Status: PROPOSED — the freeze ratification is the maintainer's call, like the embedding freeze indocs/api-stability.md. - Error bodies are sanitized — no information leak (sq-cz89 / sq-j9zs). On the
no-auth-by-default path an error body carries only a stable, generic CLASS message
(e.g.
malformed query,malformed RDF body,malformed gzip body,query execution error,update failed: invalid SPARQL update). It deliberately does NOT echo the caller's submitted query/UPDATE/RDF text, a fragment of the loaded RDF (parsers likeoxttl/spargebraquote the offending token — that would confirm loaded triples), or any server-side filesystem path (e.g. a--persistmirror's path inside a transient durable-write503). The full detailed cause is preserved for the operator via the server-sidetracinglog (targetsparq_server), surfaced only through the opt-in--verbose/RUST_LOGsubscriber — never the HTTP response. Status semantics (the400/413/415/503/500classification) are unchanged; only the prose detail is withheld. Regression tests assert a sentinel token never appears in any error body. - mimalloc is the binary's global allocator (matters under concurrent load).
serve— the underlyingsparq-servegeneration-ring + sequenced group-commit writer (the concurrency primitivesrouter/AppStatewire up).engine—sparq-enginequery/ask/construct/describe +QueryBudgetthe server drives.cli— thesparqcommand-line surface over the same engine.geo— thegeof:GeoSPARQL extension functions enabled by--features geo.core—sparq_core::Graph(Graph::load_str) the dataset is loaded into.