diff --git a/AGENTS.md b/AGENTS.md index f981c08893..2f44ae3546 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -59,19 +59,20 @@ The plugin/application loader. Applications export a `handleApplication(scope)` Files within a component are discovered via micromatch glob patterns and automatically mapped to URL paths. **Server** (`server/`) -Two HTTP stacks coexist: +Multiple HTTP entry points coexist: -- **Native layer** (`server/http.ts`) — direct socket handling for HTTP/1.1, HTTPS, HTTP/2, and WebSockets in one path; highest performance -- **Fastify layer** (`server/fastifyRoutes.ts`) — used for legacy custom functions; wraps Fastify with autoload +- **Native layer** (`server/http.ts`) — direct socket handling for application-level HTTP/1.1, HTTPS, HTTP/2, and WebSockets in one path; highest performance. Most user traffic goes through here. +- **Operations API** (`server/operationsServer.ts`) — Fastify-based JSON operations API (`{operation: 'create_table', ...}`); internal/admin surface. +- **Custom Functions (legacy)** (`server/fastifyRoutes.ts`) — legacy Fastify autoload for user-defined routes. Don't add new code here. -All inbound protocols (REST, GraphQL, MQTT, NATS, WebSockets) eventually resolve to the same **Resource interface**. +All inbound protocols (REST, GraphQL, MQTT, NATS, WebSockets) eventually resolve to the same **Resource interface**. See `server/DESIGN.md` for the file-by-file map and the `http.ts` section index. **Resources** (`resources/`) The universal abstraction. Everything that can be queried or mutated — database tables, caches, message topics, custom endpoints — extends `Resource` (`resources/Resource.ts`). Static methods (`Resource.get`, `Resource.put`, `Resource.post`, `Resource.delete`, `Resource.patch`, `Resource.subscribe`) are the entry points and are automatically wrapped with `transactional()` for transaction management. Override instance methods (`get`, `put`, etc.) for custom behavior. -`Table.ts` is the database table implementation (~177KB) — the most complex file in the codebase. +`Table.ts` is the database table implementation (4744 lines, one giant `makeTable()` factory) — the most complex file in the codebase. **Use `resources/DESIGN.md` as a section index instead of reading top-to-bottom.** **Data Layer** (`dataLayer/`) Legacy translation modules plus SQL translation (`sqlTranslator/`) via AlaSQL; these should be avoided. The storage engine is selectable via `HARPER_STORAGE_ENGINE=lmdb`. @@ -80,7 +81,62 @@ Legacy translation modules plus SQL translation (`sqlTranslator/`) via AlaSQL; t YAML-based. `configUtils.js` parses config; `RootConfigWatcher.ts` enables hot reload. Environment variables override YAML values. **Utility** (`utility/`) -Logging, error types, helpers, async utilities. +Logging, error types, helpers, async utilities. Most-used: `utility/hdbTerms.ts` (global constants), `utility/logging/harper_logger.js`, `utility/errors/hdbError.js`. + +--- + +## Repository map + +Use this to land in the right folder before grepping. Every top-level folder is listed; deeper docs are noted where they exist. + +### Source — covered above + +- **`components/`** — plugin/app loader. Entry: `Scope.ts`, `OptionsWatcher.ts`. Tests: `unitTests/components/`. +- **`server/`** — HTTP/WS/MQTT/etc. Entry: `operationsServer.ts` (boot), `http.ts` (native HTTP). **See [server/DESIGN.md](server/DESIGN.md).** Tests: `unitTests/server/`. +- **`resources/`** — universal Resource abstraction; tables. Entry: `Resource.ts`, `Table.ts`. **See [resources/DESIGN.md](resources/DESIGN.md).** Tests: `unitTests/resources/`. +- **`dataLayer/`** — legacy translation modules (`insert.js`, `search.js`, `update.js`). **Avoid for new code.** Tests: `unitTests/dataLayer/`. +- **`config/`** — YAML config + hot reload. Entry: `configUtils.js`, `RootConfigWatcher.ts`. Tests: `unitTests/config/`. +- **`utility/`** — logging, errors, helpers. Tests: `unitTests/utility/`. + +### Other source folders + +- **`bin/`** — CLI entry points. `harper.js` is the executable; `run.js` initializes and runs the server; `cliOperations.js` translates CLI args → API operations. Tests: `unitTests/bin/`. **Don't look here for** business logic. +- **`security/`** — auth, authz, certificate handling, context. Entry: `jsLoader.ts` exposes `getContext()`, `getResponse()`, `getUser()`; `user.ts` for User/Role; `certificateVerification/` for TLS validation; `data_objects/` for permission/role models. Tests: `unitTests/security/`. +- **`sqlTranslator/`** — SQL → internal operations via AlaSQL AST. Entry: `sqlTranslator/index.js` exports `evaluateSQL`, `processAST`, `convertSQLToAST`, `checkASTPermissions`. **Legacy — avoid for new code.** Tests: `unitTests/sqlTranslator/`. +- **`validation/`** — input shape validation (Joi + `validate.js`). Entry: `validationWrapper.js`. **Not authorization** — that's in `security/`. Tests: `unitTests/validation/`. +- **`upgrade/`** — version-upgrade orchestration. Entry: `directivesManager.js` exports `processDirectives()`. Per-version logic in `directives/`. Tests: `integrationTests/upgrade/`. +- **`launchServiceScripts/`** — thin launchers that delegate to `server/operationsServer.ts`. `checkNodeVersion.js` is the pre-flight Node version check. +- **`json/`** — system schema definitions. `systemSchema.json` defines built-in tables (`hdb_user`, `hdb_role`, `hdb_permission`). Loaded at startup; no code. + +### Non-source + +- **`bin/`** — covered above (it's source). +- **`benchmarks/`** — HNSW vector-search benchmark only (`hnsw-search.js`). Stand-alone; not part of CI. +- **`build-tools/`** — shell scripts for the build pipeline (`build.sh`, `build-studio.sh`, `download-prebuilds.js`). No tests. +- **`dev/`** — single dev utility (`sync-commits.js`) for cross-repo commit syncing. Not runtime. +- **`integrationTests/`** — end-to-end tests against a built distribution. Run with `npm run test:integration` / `npm run test:integration:all`. Subdirs mirror source. See `integrationTests/README.md`. +- **`unitTests/`** — Mocha unit tests; subdir per source layer. Run with `npm run test:unit:`. +- **`static/`** — assets only: `defaultConfig.yaml`, `ascii_logo.txt`. + +### Top-level docs to consult + +- **[DESIGN.md](DESIGN.md)** — running list of non-obvious internals (RecordObject prototype, getFromSource timing, blob orphan cleanup). Read this before debugging anything record-store-related. +- **[dependencies.md](dependencies.md)** — rationale for every npm dependency. Required reading before adding a new package. +- **[storage-format.md](storage-format.md)** — on-disk layout (RocksDB/LMDB). +- **[CONTRIBUTING.md](CONTRIBUTING.md)** — contribution workflow. + +--- + +## Detailed navigation + +For megafiles and complex subsystems, jump to the section index instead of reading top-to-bottom: + +| If you are touching… | Read first | +| ------------------------------------------------------ | ------------------------------------------ | +| Anything in `resources/` (especially `Table.ts`) | [resources/DESIGN.md](resources/DESIGN.md) | +| HTTP/WS/MQTT, middleware ordering, content types | [server/DESIGN.md](server/DESIGN.md) | +| Record-store internals (commit timing, blobs, encoder) | [DESIGN.md](DESIGN.md) | +| Adding a dependency | [dependencies.md](dependencies.md) | --- diff --git a/resources/DESIGN.md b/resources/DESIGN.md new file mode 100644 index 0000000000..62f376c921 --- /dev/null +++ b/resources/DESIGN.md @@ -0,0 +1,98 @@ +# resources/ — Navigation Guide + +The Resource layer is Harper's universal abstraction: all queryable/mutable things (tables, caches, message topics, custom endpoints) extend `Resource`. Inbound protocols (REST, GraphQL, MQTT, NATS, WebSockets) all converge on this interface. + +**Read this when:** you're touching the read/write path, authorization, subscriptions, or table CRUD semantics. + +See also: `../DESIGN.md` for cross-cutting non-obvious internals (RecordObject prototype, `getFromSource` timing, blob orphan cleanup). + +> **Navigation convention.** This guide references code by **symbol name** (e.g. `_writeUpdate`) and by **section marker** (e.g. `// #section: write-path-internals`). Jump in your editor via go-to-symbol, or `grep` for the section marker. Line numbers drift; symbols and section markers don't. + +--- + +## File overview + +| File | Purpose | +| ----------------------- | ------------------------------------------------------------------------------------------------------------------------------------ | +| `Resource.ts` | Base class; `transactional()` wrapper; method routing | +| `Table.ts` | Table-as-Resource implementation. Factory `makeTable()` returns a `TableResource` subclass per table. **See section markers below.** | +| `Resources.ts` | Registry mapping URL paths → Resource classes | +| `RequestTarget.ts` | Parses path/query into a structured target | +| `ResourceInterface.ts` | Type definitions (`Context`, `Record`, etc.) | +| `RecordEncoder.ts` | msgpack encoding + `entryMap` (record → storage entry) | +| `IterableEventQueue.ts` | Async iterable used for subscriptions and streaming responses | +| `transaction.ts` | Per-request transaction object stored in `contextStorage` | +| `auditStore.ts` | Append-only audit log records | +| `nodeIdMapping.ts` | Maps node IDs ↔ timestamps for replication ordering | +| `openApi.ts` | Generates OpenAPI/JSON Schema from `@export` schemas | +| `analytics/` | Telemetry recording (separate from monitoring) | + +--- + +## `Resource.ts` — base class + +Static methods are protocol entry points (each wrapped in `transactional()`); instance methods are the per-resource behavior hooks subclasses override. + +| Member | Notes | +| ---------------------------------- | -------------------------------------------------------------------------------------------------- | +| `class Resource` | Generic over `Record extends object` | +| `constructor(identifier, source)` | | +| Static CRUD entry points | `get`, `put`, `patch`, `delete`, `post`, `update`, `create`, `invalidate` | +| Static pub/sub entry points | `connect`, `subscribe`, `publish` | +| Static query entry points | `search`, `query` | +| Static path helpers | `parsePath` (URL → `RequestTarget`), `getResource` (path → class) | +| Other statics | `getNewId`, `copy`, `move` | +| Authorization hooks | `allowRead` / `allowUpdate` / `allowCreate` / `allowDelete` — default impls; override per resource | +| Instance helpers | `getId`, `getContext`, `getCurrentUser` | +| `transactional()` wrapper | **Do not remove from static methods** — owns transaction context lifetime | +| `missingMethod` / `allowedMethods` | 405 response helpers | +| `transformForSelect` | Select-clause expansion | + +--- + +## `Table.ts` — section map + +One giant `makeTable()` factory that returns a `TableResource extends Resource` class. The file is divided into the sections below; each is anchored by a `// #section: ` marker — grep for the marker (or use VS Code's go-to-symbol within the section) to land directly. + +| Section marker | Contents | +| -------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `#section: setup-and-factory` | `makeTable(options)` entry, attribute parsing & primary-key detection, replication wiring, `class Updatable` (RecordObject prototype: `getUpdatedTime`, `getExpiresAt`, `addTo`, `subtractFrom`). Ends where `class TableResource` opens. | +| `#section: static-config` | Static configuration properties: `name`, `primaryStore`, `auditStore`, `primaryKey`, `indices`, `audit`, `databasePath`, `attributes`, `replicate`, `sealed`, `splitSegments`, `getResidencyById`, `dbisDB`, `schemaDefined`, `expirationMS`. | +| `#section: resource-registry` | `sourcedFrom()` (cache/source hierarchy — the largest static), `isCaching`, `shouldRevalidateEvents`, `getResource()`, `_updateResource`, `ensureLoaded()`. | +| `#section: lifecycle-admin` | `getNewId()` (UUID / autoincrement / prefix / time-based strategies), `setTTLExpiration`, residency (`getResidencyRecord`, `setResidency`, `setResidencyById`, `getResidency`), `enableAuditing`, `coerceId`, `dropTable`. | +| `#section: read-path` | `get()` overloads & impl. | +| `#section: authz-hooks` | `allowRead`, `allowUpdate`, `allowCreate`, `allowDelete`. | +| `#section: write-path-public` | `update()`, `save()`, `addTo()`, `subtractFrom()`, `getMetadata`, `getRecord`, `getChanges`, `_setChanges`, `setRecord`, `invalidate()`, `operation()`, `put()`, `create()`, `patch()`. | +| `#section: write-path-internals` | **`_writeUpdate()` — the central write routine** (versioning, conflict resolution, audit, residency, replication metadata, blob orphan tracking). The `write.skipped` flag mentioned in `../DESIGN.md` is set in this method's early-return paths. Also `_writeInvalidate`, `_writeRelocate`, `_recordRelocate`, `evict()`, `lock()`, `delete()`, `_writeDelete`. | +| `#section: search-query` | `search()` (the query engine — index selection, filter evaluation), `transformToOrderedSelect` (select-clause ordering), `transformEntryForSelect` (record → response shape). | +| `#section: pub-sub` | `subscribe()` (subscription request handling, replay, cursor management), `subscribeOnThisThread`, `doesExist()`, `publish()`, `_writePublish()`. | +| `#section: validation` | `validate(record, patch?)` — schema enforcement, computed attributes, attribute coercion. | +| `#section: stats-admin` | `getUpdatedTime`, `addAttributes`, `removeAttributes`, `getSize`, `getAuditSize`, `getStorageStats`, `getRecordCount`, `updatedAttributes` (schema diff machinery). | +| `#section: computed-history` | `setComputedAttribute`, `deleteHistory`, `getHistory` (generator), `getHistoryOfRecord`, `clear`, `cleanup`, `_readTxnForContext`. | +| _(after the class)_ | `getFromSource()` — cache miss → source load (see `../DESIGN.md` for the resolve-before-commit timing trap); local helpers (`coerceType`, `isDescendantId`, etc.). | + +--- + +## "Where is X" cheat sheet + +| Question | Where | +| --------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------- | +| How is a CRUD request authorized? | `Table.ts → #section: authz-hooks`; defaults in `Resource.ts` (`allowRead` etc.) | +| Where does versioning / conflict resolution happen? | `Table.ts → _writeUpdate` (`#section: write-path-internals`) | +| How does `search()` choose an index? | `Table.ts → search` (`#section: search-query`) | +| How are subscriptions replayed? | `Table.ts → subscribe` (`#section: pub-sub`) | +| How is the response body shaped (select clause)? | `Table.ts → transformEntryForSelect` (`#section: search-query`) | +| Where is record-level TTL evaluated? | `Table.ts → setTTLExpiration` (`#section: lifecycle-admin`); `Updatable.getExpiresAt` (`#section: setup-and-factory`) | +| How are residencies enforced (replication)? | `Table.ts → #section: lifecycle-admin` (residency block: `getResidencyRecord`, `setResidency`, `setResidencyById`, `getResidency`) | +| How is the RecordObject prototype applied? | `RecordEncoder.ts` (see `../DESIGN.md`) | +| Where is the per-request transaction stored? | `transaction.ts` + `contextStorage` (AsyncLocalStorage) | + +--- + +## Conventions + +- **Never** remove `transactional()` from a static method on `Resource` — it owns transaction context lifetime. +- New `Resource` subclasses should override **instance** methods (`get`, `put`, ...) for behavior; static methods are the protocol entry points and stay generic. +- When adding a new early-return path inside a commit handler in `_writeUpdate`, follow the blob-cleanup protocol documented in `../DESIGN.md` ("Blob orphan cleanup"). +- If you add a new top-level section to `Table.ts`, drop a `// #section: ` marker at its start and add a row to the section map above. +- Tests for this layer live in `../unitTests/resources/`. diff --git a/resources/Table.ts b/resources/Table.ts index a4f3713003..7bb8d1624e 100644 --- a/resources/Table.ts +++ b/resources/Table.ts @@ -133,6 +133,7 @@ type ResidencyDefinition = number | string[] | void; * Instances of the returned class are Resource instances, intended to provide a consistent view or transaction of the table * @param options */ +// #section: setup-and-factory export function makeTable(options) { const { primaryKey, @@ -225,6 +226,7 @@ export function makeTable(options) { #savingOperation?: any; // operation for the record is currently being saved declare getProperty: (name: string) => any; + // #section: static-config static name = tableName; // for display/debugging purposes static primaryStore = primaryStore; static auditStore = auditStore; @@ -260,6 +262,7 @@ export function makeTable(options) { * @param options * @returns */ + // #section: resource-registry static sourcedFrom(source, options) { // define a source for retrieving invalidated entries for caching purposes if (options) { @@ -637,6 +640,7 @@ export function makeTable(options) { }); } } + // #section: lifecycle-admin static getNewId(): any { const type = primaryKeyAttribute?.type; // the default Resource behavior is to return a GUID, but for a table we can return incrementing numeric keys if the type is (or can be) numeric @@ -936,6 +940,7 @@ export function makeTable(options) { new SchemaEventMsg(process.pid, OPERATIONS_ENUM.DROP_TABLE, databaseName, tableName) ); } + // #section: read-path /** * This retrieves the data of this resource. By default, with no argument, just return `this`. */ @@ -1050,6 +1055,7 @@ export function makeTable(options) { } return undefined; } + // #section: authz-hooks /** * Determine if the user is allowed to get/read data from the current resource */ @@ -1160,6 +1166,7 @@ export function makeTable(options) { return !!tablePermission?.delete && checkContextPermissions(context); } + // #section: write-path-public /** * Start updating a record. The returned resource will record changes which are written * once the corresponding transaction is committed. These changes can (eventually) include CRDT type operations. @@ -1583,6 +1590,7 @@ export function makeTable(options) { }); } } + // #section: write-path-internals // perform the actual write operation; this may come from a user request to write (put, post, etc.), or // a notification that a write has already occurred in the canonical data source, we need to update our // local copy @@ -2019,6 +2027,7 @@ export function makeTable(options) { return true; } + // #section: search-query search(target: RequestTarget): AsyncIterable> { const context = this.getContext(); const txn = txnForContext(context); @@ -2637,6 +2646,7 @@ export function makeTable(options) { return transform; } + // #section: pub-sub async subscribe(request: SubscriptionRequest): Promise> { if (!auditStore) throw new Error('Can not subscribe to a table without an audit log'); if (!audit) { @@ -3026,6 +3036,7 @@ export function makeTable(options) { write.beforeIntermediate = preCommitBlobsForRecordBefore(write, message, undefined, true); transaction.addWrite(write); } + // #section: validation validate(record: any, patch?: boolean) { let validationErrors; const validateValue = (value, attribute: Attribute, name) => { @@ -3195,6 +3206,7 @@ export function makeTable(options) { throw new ClientError(validationErrors.join('. ')); } } + // #section: stats-admin getUpdatedTime() { return this.#version; } @@ -3499,6 +3511,7 @@ export function makeTable(options) { } } } + // #section: computed-history static setComputedAttribute(attribute_name, resolver) { const attribute = findAttribute(attributes, attribute_name); if (!attribute) { diff --git a/server/DESIGN.md b/server/DESIGN.md new file mode 100644 index 0000000000..1b18ab6de9 --- /dev/null +++ b/server/DESIGN.md @@ -0,0 +1,139 @@ +# server/ — Navigation Guide + +This layer accepts inbound traffic on every supported protocol (HTTP/1.1, HTTP/2, HTTPS, WebSockets, MQTT, NATS) and routes it through to the Resource layer. + +**Read this when:** you're touching request/response, protocol handling, middleware ordering, or WebSocket upgrade behavior. + +> **Navigation convention.** This guide references code by **symbol name** (function/const). Use your editor's go-to-symbol or `grep -n '' server/` to jump. Line numbers drift; symbols don't. + +--- + +## Three HTTP stacks coexist — know which one + +| Stack | File | Used for | +| ----------------------------- | --------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| **Native** | `http.ts` | Direct socket handling for application-level HTTP/1.1, HTTPS, HTTP/2, and WebSockets. Highest performance. This is the path most user requests take (REST, GraphQL, custom resource endpoints). | +| **Operations API** | `operationsServer.ts` | Fastify-based JSON operations API (`{operation: 'create_table', ...}`). Internal/admin surface — not on the hot path for application data. | +| **Custom Functions (legacy)** | `fastifyRoutes.ts` | Legacy custom functions only. Wraps Fastify with autoload. Don't add new code here. | + +A request entering `http.ts` does **not** go through Fastify. The two `handleApplication(scope)` functions (one in each Fastify file) load independently from component config. + +--- + +## File overview + +### Core dispatch + +| File | Purpose | +| -------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `Server.ts` | Defines the `Server` interface — the contract that protocol plugins use to register listeners. Has `socket()`, `http()`, `ws()`, `upgrade()`, `contentTypes`, `getUser()`, `operation()`, `replication`, etc. | +| `http.ts` | Native HTTP/WS server. Registration entry points (`onRequest`, `onUpgrade`, `onWebSocket`), per-port middleware chains, UDS support, PROXY protocol. **See section map below.** | +| `middlewareChain.ts` | Topological sort respecting `before`/`after` constraints on listener registrations (`topoSort`). Falls back to registration order on cycle. | +| `REST.ts` | Resource-routed REST handler: URL → `Resource.getResource()` → method dispatch + content negotiation. | +| `graphqlQuerying.ts` | GraphQL query/mutation/subscription execution against Resources. | +| `mqtt.ts` | MQTT broker (connect/sub/pub mapped onto Resource interface). | +| `DurableSubscriptionsSession.ts` | Persistent subscription state (resume across reconnects). | + +### Operations & Fastify + +| File | Purpose | +| --------------------- | ------------------------------------------------------------------------------------------------------------------------------ | +| `operationsServer.ts` | Boots Fastify for operations API. `buildServer()` constructs the server; `handler()` parses `{operation: ...}` and dispatches. | +| `fastifyRoutes.ts` | Legacy custom functions. Discovers routes from each component's `routes/` folder. | + +### Helpers + +| File | Purpose | +| ------------------------------------------ | ------------------------------------------------------------------------------- | +| `serverHelpers/Request.ts` | Wraps `IncomingMessage` with Harper-specific fields (user, response, headers). | +| `serverHelpers/Headers.ts` | Header mutation/merge utilities. | +| `serverHelpers/contentTypes.ts` | (de)serialization registry; `serialize`, `serializeMessage`, `getDeserializer`. | +| `serverHelpers/serverUtilities.ts` | `OperationDefinition` and shared helpers. | +| `serverHelpers/OperationFunctionObject.ts` | Wraps an operation handler with metadata. | +| `serverHelpers/JSONStream.ts` | Streaming JSON output for large responses. | +| `nodeName.ts` | Resolves this node's name (config → hostname). | +| `static.ts` | Static file serving for component-bundled assets. | +| `throttle.ts` | Per-IP / per-user request throttling. | +| `storageReclamation.ts` | Disk-pressure signals to downstream consumers. | +| `serverRegistry.ts` | Trivial registry export. | +| `status/` | Server status reporting (cluster status, per-port info). | + +### Threads + +| File | Purpose | +| -------------------------- | -------------------------------------------------------- | +| `threads/socketRouter.ts` | Routes accepted sockets to worker threads based on port. | +| `threads/manageThreads.js` | Thread pool lifecycle. | +| `threads/threadServer.js` | Worker entry point — receives sockets via IPC. | +| `threads/itc.js` | Inter-thread comms primitives. | + +> Workers receive `workerData.noServerStart = true` — never start the server inside a worker. + +--- + +## `http.ts` — symbol map + +Every entry is a top-level function or named const. Jump via go-to-symbol or `grep -n 'function ' server/http.ts`. + +| Symbol | What it does | +| ------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| `registerUdsCleanupPaths`, `cleanupUdsFiles`, `writeUdsMetadata`, `cleanupSocketsDirectory` | UDS socket / metadata file lifecycle. | +| `handleApplication(scope)` | Component entry point — captures `httpOptions` for the scope. | +| `getHttpOptions()` | Returns the current scope's `HttpOptions`. | +| `deliverSocket()` | IPC-delivered socket handoff from `socketRouter`. | +| `proxyRequest()` | Cross-port request routing. | +| `registerServer()` | Records a server for a port in the `SERVERS` map. | +| `getPorts()` | Resolves listener options → list of `{port, secure}`. | +| `httpServer()` | Main listener registration entry point. | +| `getHTTPServer(port, secure, options)` | **The largest function in the file.** Creates/retrieves the underlying Node HTTP/HTTPS server. Wires `request`, `upgrade`, error handlers, TLS context, and the per-port middleware chain. | +| `makeCallbackChain()` | Builds the per-port handler chain via `middlewareChain.topoSort`. | +| `unhandled()` | Terminal 404 handler. | +| `onRequest()` | Thin alias of `httpServer({requestOnly: true})`. | +| `onUpgrade()` / `upgradeListeners` (const) | Register HTTP upgrade listener; underlying list. | +| `onWebSocket()` / `websocketListeners` (const) | Register WebSocket listener; auto-adds default upgrade handler the first time it runs for a port. Underlying list of registrations. | +| `enableProxyProtocol()` | PROXY v1 parsing (Node 24+-compatible workaround). | +| `defaultNotFound()` | Default 404 response. | +| `logRequest()` | Per-request access log line. | +| `getRequestId()` | Generates the per-request correlation ID. | + +### Middleware ordering (`before` / `after`) + +Components register listeners with optional `before: 'name'` / `after: 'name'` options. `middlewareChain.topoSort` resolves order; cycles fall back to registration order with a warning. Three lists hold the registrations: + +- `httpResponders` — request handlers +- `upgradeListeners` (in `http.ts`) +- `websocketListeners` (in `http.ts`) + +The default WebSocket upgrade handler is registered automatically inside `onWebSocket()` the first time it runs for a given port. + +--- + +## Resource ↔ HTTP boundary + +`REST.ts → http(request, nextHandler)` is the chief integration point: it takes a `Request`, asks the `Resources` registry for a match, builds a `RequestTarget`, and dispatches into the Resource class's static method. Cache headers are translated to `request.expiresAt` / `onlyIfCached` / `noCache` flags within the same function. + +--- + +## "Where is X" cheat sheet + +| Question | Where | +| --------------------------------------------------- | --------------------------------------------------------------------- | +| Where do I register a new HTTP handler? | `http.ts → httpServer()` (or `onRequest()` for the request-only form) | +| Where do I register a WebSocket handler? | `http.ts → onWebSocket()` | +| How does `before`/`after` middleware ordering work? | `middlewareChain.ts → topoSort` | +| Where does PROXY protocol get parsed? | `http.ts → enableProxyProtocol` | +| Where is the REST request → Resource dispatch? | `REST.ts → http()` | +| Where is the operations API request handled? | `operationsServer.ts → handler` | +| How are content types (de)serialized? | `serverHelpers/contentTypes.ts` | +| Where do durable subscriptions live? | `DurableSubscriptionsSession.ts` | +| How are sockets dispatched to worker threads? | `threads/socketRouter.ts` | +| Where is the Operations API wired into Fastify? | `operationsServer.ts → buildServer` | + +--- + +## Conventions + +- Don't add new code to `fastifyRoutes.ts` — it's the legacy custom-functions path. +- New protocol plugins implement the `Server` interface (in `Server.ts`) and register via `onRequest`/`onUpgrade`/`onWebSocket`. +- Always pass `name` when registering a listener with `before`/`after` — anonymous entries can't be ordered against. +- Tests live in `../unitTests/server/`.