Skip to content

DESIGN.md

Latest commit

 

History

History
190 lines (141 loc) · 44 KB

DESIGN.md

File metadata and controls

190 lines (141 loc) · 44 KB
Raw

server/ — Navigation Guide

This layer accepts inbound traffic on every supported protocol (HTTP/1.1, HTTP/2, HTTPS, WebSockets, MQTT, NATS) and routes it through to the Resource layer.

Read this when: you're touching request/response, protocol handling, middleware ordering, or WebSocket upgrade behavior.

Navigation convention. This guide references code by symbol name (function/const). Use your editor's go-to-symbol or grep -n '<name>' server/<file> to jump. Line numbers drift; symbols don't.

Three HTTP stacks coexist — know which one

Stack File Used for
Native http.ts Direct socket handling for application-level HTTP/1.1, HTTPS, HTTP/2, and WebSockets. Highest performance. This is the path most user requests take (REST, GraphQL, custom resource endpoints).
Operations API operationsServer.ts Fastify-based JSON operations API ({operation: 'create_table', ...}). Internal/admin surface — not on the hot path for application data.
Custom Functions (legacy) fastifyRoutes.ts Legacy custom functions only. Wraps Fastify with autoload. Don't add new code here.

A request entering http.ts does not go through Fastify. The two handleApplication(scope) functions (one in each Fastify file) load independently from component config.

File overview

Core dispatch

File Purpose
Server.ts Defines the Server interface — the contract that protocol plugins use to register listeners. Has socket(), http(), ws(), upgrade(), contentTypes, getUser(), operation(), replication, etc.
http.ts Native HTTP/WS server. Registration entry points (onRequest, onUpgrade, onWebSocket), per-port middleware chains, UDS support, PROXY protocol. See section map below.
middlewareChain.ts Topological sort respecting before/after constraints on listener registrations (topoSort). Falls back to registration order on cycle. Also urlPath/host sub-route dispatch: a mount is a prefix with a segment boundary, the mount prefix is stripped from request.pathname before the sub-chain runs, and the root mount '/' normalizes to no path constraint (joins the default chain, nothing stripped — #1766).
REST.ts Resource-routed REST handler: URL → Resource.getResource() → method dispatch + content negotiation.
graphqlQuerying.ts GraphQL query/mutation/subscription execution against Resources.
mqtt.ts MQTT broker (connect/sub/pub mapped onto Resource interface).
DurableSubscriptionsSession.ts Persistent subscription state (resume across reconnects).

Operations & Fastify

File Purpose
operationsServer.ts Boots Fastify for operations API. buildServer() constructs the server; handler() parses {operation: ...} and dispatches.
fastifyRoutes.ts Legacy custom functions. Discovers routes from each component's routes/ folder.

Helpers

File Purpose
serverHelpers/Request.ts Wraps IncomingMessage with Harper-specific fields (user, response, headers).
serverHelpers/Headers.ts Header mutation/merge utilities.
serverHelpers/contentTypes.ts (de)serialization registry; serialize, serializeMessage, getDeserializer.
serverHelpers/serverUtilities.ts OperationDefinition and shared helpers.
serverHelpers/OperationFunctionObject.ts Wraps an operation handler with metadata.
serverHelpers/JSONStream.ts Streaming JSON output for large responses.
nodeName.ts Resolves this node's name (config → hostname).
static.ts Static file serving for component-bundled assets.
throttle.ts Per-IP / per-user request throttling.
storageReclamation.ts Disk-pressure signals to downstream consumers; getStorageSpaceStats() is the shared quota-aware (falls back to statfs) source of available/free/size storage numbers — used by Table.getStorageStats() (#1976). NOT used for blob storage path weighting (resources/blob.ts): quota-status.json is a single instance-wide figure, so it can't distinguish between multiple STORAGE_BLOBPATHS disks — that still needs raw per-path statfs.
serverRegistry.ts Trivial registry export.
status/ Server status reporting (cluster status, per-port info).

Threads

File Purpose
threads/socketRouter.ts Routes accepted sockets to worker threads based on port.
threads/manageThreads.js Thread pool lifecycle.
threads/threadServer.js Worker entry point — receives sockets via IPC.
threads/itc.js Inter-thread comms primitives.
transactionLogCooling.ts Main-thread timer that cools transaction-log mmaps.

Workers receive workerData.noServerStart = true — never start the server inside a worker.

threadServer.listenOnDomainSocket() skips a listener only when its path exceeds the platform's sockaddr_un.sun_path byte limit (some Node versions reject it; others silently truncate it). Every actual listen() error rejects startup, and the temporary bind-error listener is removed once the socket is listening.

Where periodic maintenance runs (main thread vs last worker)

Single-instance background tasks pick their thread by what state they touch:

  • Last worker (getWorkerIndex() === getWorkerCount() - 1) — for tasks that operate on worker-resident JS state: audit cleanup (resources/auditStore.ts) and disk reclamation (storageReclamation.ts) walk per-store objects that only exist in a worker.
  • Main thread (isMainThread) — for tasks that drive a process-global native singleton and need no JS state. transactionLogCooling.ts is the example: rocksdb-js's transaction-log registry is one C++ static shared across all worker threads, so any thread cools every log. The main thread is chosen because it is the only thread that lives for the whole process — a worker-driven timer would stall whenever that worker is recycled.

http.ts — symbol map

Every entry is a top-level function or named const. Jump via go-to-symbol or grep -n 'function <name>' server/http.ts.

Symbol What it does
registerUdsCleanupPaths, recordUdsBindSuccess, cleanupUdsFiles, markUdsBindFailed, writeUdsMetadata, cleanupSocketsDirectory UDS socket / metadata file lifecycle. Ownership-aware: recordUdsBindSuccess captures the inode a worker's own bind confirmed; cleanupUdsFiles/markUdsBindFailed only unlink a path when the inode on disk still matches, so an overlapping restart's outgoing worker can never delete the replacement that already rebound the same path (see restartWorkers() in manageThreads.js). cleanupSocketsDirectory is the separate crash-path sweep, run once from socketRouter.ts's startHTTPThreads on main-thread startup, before any worker can bind.
handleApplication(scope) Component entry point — captures httpOptions for the scope.
getHttpOptions() Returns the current scope's HttpOptions.
deliverSocket() IPC-delivered socket handoff from socketRouter.
proxyRequest() Cross-port request routing.
registerServer() Records a server for a port in the SERVERS map.
getPorts() Resolves listener options → list of {port, secure}.
httpServer() Main listener registration entry point.
getHTTPServer(port, secure, options) The largest function in the file. Creates/retrieves the underlying Node HTTP/HTTPS server. Wires request, upgrade, error handlers, TLS context, and the per-port middleware chain.
makeCallbackChain() Builds the per-port handler chain via middlewareChain.topoSort.
unhandled() Terminal 404 handler.
onRequest() Thin alias of httpServer({requestOnly: true}).
onUpgrade() / upgradeListeners (const) Register HTTP upgrade listener; underlying list.
onWebSocket() / websocketListeners (const) Register WebSocket listener; auto-adds default upgrade handler the first time it runs for a port. Underlying list of registrations.
enableProxyProtocol() PROXY v1/v2 stripping on UDS mirrors (Node 24+-compatible workaround). Decoding lives in serverHelpers/proxyProtocol.ts; v2 TLVs forward the client source address plus the connection's TLS facts a fronting proxy (symphony) observed — ALPN, SNI authority, TLS version/cipher, JA3/JA4 fingerprints, and the mTLS client cert chain. These are surfaced on request.connectionInfo (see below); the verified cert chain is additionally exposed with TLSSocket semantics (authorized, getPeerCertificate()) so HTTP/MQTT mTLS auth works unchanged, and the SSL TLV lets request.protocol report https on the plaintext UDS mirror. A peer that stalls mid-header is destroyed after prehandoffTimeout (default 10s), matching withProxyProtocol's guard on the raw-socket path.
defaultNotFound() Default 404 response.
logRequest() Per-request access log line.
getRequestId() Generates the per-request correlation ID.

Middleware ordering (before / after)

Components register listeners with optional before: 'name' / after: 'name' options. middlewareChain.topoSort resolves order; cycles fall back to registration order with a warning. Three lists hold the registrations:

  • httpResponders — request handlers
  • upgradeListeners (in http.ts)
  • websocketListeners (in http.ts)

The default WebSocket upgrade handler is registered automatically inside onWebSocket() the first time it runs for a given port.

Application mounts (host / urlPath in the root config)

An operator mounts an application by putting host/urlPath on its entry in the root config; components/scopeMount.ts models it and the loader threads it into every Scope for that application (both load paths — the root-config package recursion and the components-root directory scan).

The mount is applied at exactly one place: Scope.routeFor(), used by the scope.server proxy. Do not push it anywhere else. In particular, do not compose it into the plugin config the entry pipeline reads: entry.urlPath is what graphqlSchema and jsResource derive resource paths from, and the router strips the mount before REST resolves them. Composing it there registers a table at /v1/Thing while REST looks up Thing, and every mounted REST route 404s. A single-app static test will not catch it — static de-prefixes its own map keys, so it stays self-consistent either way.

Consequences worth knowing:

  • Everything inside an application addresses itself mount-relative. Only two things need the absolute path: code that emits a URL back to the client (use Scope.externalBasePath() — static's redirect Location), and code that bypasses the routed chain (legacy fastify registers on the bare server, so its route prefix must be the full external path). static.ts's mount-root redirect gates on the external base path, not the plugin-local one — a root-level static plugin (baseURLPath === '/') still needs the redirect when the application itself carries a mount, since the client-visible mount root is then externalBaseURLPath, not /.
  • A plugin registering per-mount state must key it on Scope.routeFor()'s resolved route, not on the parts it composes from — distinct (mount, pluginUrlPath) pairs can flatten to the same string (/a+bc and /ab+c). REST.ts's startedMounts does this; it replaced a process-global started flag that silently 404'd the second mounted application's REST API. handleApplication also closes over resources/httpOptions per call rather than a module-level var, and skips deploy pre-flight validation scopes (scope.isTransientValidation) entirely — registering handlers from a throwaway validation scope would splice a validation run into the live request path and permanently mark that mount started, silently skipping the real scope's later registration.
  • A mount is routing, not isolation: exported resources stay instance-wide, and a host mount cannot constrain legacy fastify routes — fastifyRoutes.ts refuses to load (throws) rather than warn when a host mount is configured, since the fallback really is reachable on every host.
  • An invalid mount (unparseable host/urlPath) fails the application closed: componentLoader.tryRootConfigMount skips loading it entirely rather than falling back to unmounted access — loading unconstrained would silently drop the isolation the operator asked for, which is worse than not loading at all.
  • Two applications mounted at different routes can register same-named middleware (e.g. both enable rest) without colliding: middlewareChain.resolveRoutedChains resolves before/after name references against a registry scoped to that route's own group, falling back to a global registry that only holds genuinely unmounted entries (e.g. authentication) — never another mounted route's entries.
  • host matching reads request.host (Harper's Request.host getter), not the raw Host header — HTTP/2 clients send :authority, never Host, so reading the header directly silently 404s every host-mounted app under h2 while h1 keeps working. hostnameFromHeader also strips a trailing dot (api.example.com., the absolute-FQDN form some resolvers emit) since it names the same origin.
  • scopeMount.normalizeMountHost validates against the same grammar as the deploy_component operation's host field (bare DNS hostname or IPv6 literal) and throws otherwise, so a hand-typed root-config host with a port/scheme/path fails the application closed too, instead of loading it unreachably. nestScopeMount logs a warning when a child's host is discarded by the parent-authority rule, so that isn't silent.

Operations authorization boundary

Operations request bodies are untrusted data. serverHandlers.js → handlePostRequest() rejects prototype-mutating property names and strips the legacy bypass_auth property before dispatch. serverUtilities.ts → chooseOperation() never reads authorization control from the body: trusted internal callers pass bypass state as a separate argument and expose it to operation handlers only through operationAuthorizationState.ts's async context. When an operation registered by a component must run on a worker, registeredOperations.ts carries that state in the same-process ITC envelope, separately from the structured-cloned body. Never attach trusted dispatch state to an operation payload.

Resource ↔ HTTP boundary

REST.ts → http(request, nextHandler) is the chief integration point: it takes a Request, asks the Resources registry for a match, builds a RequestTarget, and dispatches into the Resource class's static method. Cache headers are translated to request.expiresAt / onlyIfCached / noCache flags within the same function.

Response Cache-Control / Vary policy (#1518, #1565)

Three tiers, applied in two places:

  1. App/resource explicit — a Cache-Control set by the resource (or @table(cacheControl: "...") for anonymous reads, emitted in REST.ts → http()) always wins. The declaration is required: anonymous readability alone never emits shared-cache headers, because a request-attribute-gated allowRead (IP, headers) would make inferred public unsound.
  2. Identity floor — security/auth.ts → applyResponseHeaders stamps Cache-Control: private, no-cache + Vary: Authorization (+ Cookie when sessions are on) on any response where a principal was resolved or credentials were rejected (401), unless the app opted into shared caching with public/s-maxage (the RFC 9111 opt-in).
  3. CORS partitioning — when CORS is enabled, every response gets Vary: Origin (the ACAO header is reflected per-origin, and its absence on no-Origin requests is origin-dependent too).

The @table(cacheControl:) value is persisted on the primary-key attribute (like expiration), so all threads and future boots see it; resources/databases.ts → table() treats null as "schema explicitly has none" (clears on reload) and undefined as "caller is not schema-defining" (no clobber from add_attribute/cluster schema events).

"Where is X" cheat sheet

Question Where
Where do I register a new HTTP handler? http.ts → httpServer() (or onRequest() for the request-only form)
Where do I register a WebSocket handler? http.ts → onWebSocket()
How does before/after middleware ordering work? middlewareChain.ts → topoSort
Where does PROXY protocol get parsed? serverHelpers/proxyProtocol.ts (applied by http.ts → enableProxyProtocol / createH2CProxyFront)
How does an app read forwarded TLS facts (JA3/JA4, ALPN, SNI, mTLS cert)? request.connectionInfo (ConnectionInfo from serverHelpers/proxyProtocol.ts); only set from a trusted PROXY v2 header on the UDS mirror, never from a request header
Where is the REST request → Resource dispatch? REST.ts → http()
Where is the operations API request handled? operationsServer.ts → handler
How are content types (de)serialized? serverHelpers/contentTypes.ts
Where do durable subscriptions live? DurableSubscriptionsSession.ts
How are sockets dispatched to worker threads? threads/socketRouter.ts
Where is the Operations API wired into Fastify? operationsServer.ts → buildServer

Conventions

  • Don't add new code to fastifyRoutes.ts — it's the legacy custom-functions path.
  • New protocol plugins implement the Server interface (in Server.ts) and register via onRequest/onUpgrade/onWebSocket.
  • Always pass name when registering a listener with before/after — anonymous entries can't be ordered against.
  • Tests live in ../unitTests/server/.