Skip to content

API.md

Latest commit

 

History

History
2409 lines (2101 loc) · 183 KB

API.md

File metadata and controls

2409 lines (2101 loc) · 183 KB
Raw

Newspack Nodes REST API

The runtime ships a small REST surface for worker lifecycle, session auth, command dispatch, an MCP door over its own commands, an SSE stream, and an internal cache-health probe. Application plugins register their own endpoints (dashboards, additional streams) on top, and mount service Command_Interpreter_Nodes into the dispatch endpoint's graph through the newspack_nodes/request_graph_ready hook.

Everything lives under one namespace, newspack-nodes/v1, registered by Bootstrap::register_rest_routes() on rest_api_init. The order is load-bearing: /health/runtime registers first so REST init completes even when the runtime base directory is refused, and the others register only once that base is available and ensure_runtime_wired() has run. /mcp is the one route registered elsewhere: Nodes_Mcp_Door::wire() hooks its own register_routes() on rest_api_init, as every MCP door does.

Route Method Permission
/workers/spawn POST Internal HMAC token, or MANAGE + WP nonce + rate limit
/auth POST A signed-in user + per-user mint budget
/oauth/register POST Anyone, over HTTPS + per-address rate limit
/oauth/token POST A registered client, over HTTPS + per-client rate limit; the code and verifier or the refresh token authenticate
/command POST A signed-in user + per-user burst limit, then a per-command signature and each command's declared role or WordPress capability
/mcp POST A Bearer <handle>.<key> live session admitted at this door's resource + per-session rate limit; each tool needs its command's declared role
/messages/stream GET READ
/health/runtime POST Internal HMAC token

A grid of the eight routes, /mcp among them, against the four gates a request crosses in order: WordPress's declared-argument check, the fleet gate that answers 403 newspack_nodes_not_fleet_site on a multisite subsite, each route's own permission callback with its refusal codes, and what the handler then refuses or does; beneath it, why /health/runtime registers first and /mcp registers through its door's wire(), and the entry points outside the namespace: the OAuth .well-known documents, the two settings actions on admin-post.php gated by the manage role, and the OAuth approval page with its admin-post.php decision, gated by a nonce and the scope the user can grant, which redirects to the client's registered URI.

The substrate is also its own client. HTTP_Out_Node POSTs batched JSONL command envelopes to a remote spoke's /command (COMMAND_PATH) and establishes the session that signs them at that spoke's /auth (AUTH_PATH); SSE_In_Node pulls /messages/stream over cURL. Both speak the shapes documented below, under bounds of their own — see The substrate as client.

For the full architecture and rationale, see architecture-guide.md.

Worker Spawn

POST  /wp-json/newspack-nodes/v1/workers/spawn

HMAC-validated worker spawn, used by every worker's _fleet peer scan, the WP-Cron cold-start pass, and the worker's own self_respawn() chain. Not for public callers — the token rotates every 10s and is per-site, so an external call without a valid one returns 403 Forbidden.

A five-lane sequence of one spawn POST across the spawner, WordPress, check_permission, spawn() and the worker: the argument check, the fleet gate, the two validators of the one nonce field, the three ordered refusals, the accepted spawn, the 200 written and the request finished before the worker runs inline for 595 seconds; below, the internal token formula for both purposes and the Perl mirror that copies it by hand.

Request

Field Type Required Description
type string yes Worker type, which is also the topology name (see wp nodes types). The route's validate_callback refuses a type that is not declared by any ACTIVE topology, so an unknown one never reaches the handler.
partition int yes Partition index, 0-based. Must be below both the type's active partition count and Spawn_Coordinator::MAX_PARTITIONS (8).
nonce string yes Either the internal HMAC token or a WordPress nonce for the newspack_nodes_spawn_worker action — one field, two validators.

The internal token is Internal_Request_Token::generate( 'spawn', $now, Spawn_Coordinator::spawn_key() ); the diagram carries the construction. The Perl side of that contract lives in services/gyrobase/sources/newspack-gyrobase/Gyrobase/Log.pm, and the two halves of for_window() are held together only by t/spawn-token.t in that separate repository.

Response

200 OK

{
  "spawned": true,
  "type": "job-worker",
  "partition": 0
}

Topology owners hook newspack_nodes/spawn_worker to build the right worker for $type and call ->execute(). The substrate registers Topology_Registry::spawn_worker on it at load, which spawns any worker in the active set, and the runtime ships five topologies under topologies/ — job-worker (the generic Job_Worker_Node pool, per-partition), job-intake (drains the large-write job ingress on substrate-only installs), settings-sync (a single-instance hub control plane, num_partitions = 1), topic-probe (the per-worker consumer-stats sweep, included by the others) and table-probe (the per-worker Table-stats sweep, included by a topology that declares Tables). Application plugins register the rest. Every active topology, builtin or application, spawns through this one hook — there is no separate control-plane spawn path.

Job_Worker_Node is generic async-job dispatch: applications register local and remote handlers through the newspack_nodes/job_handlers and newspack_nodes/remote_job_handlers filters, and the entry's k field picks the map. Each handler is called as ( string $id, array $parameters ) — $id is the entry's top-level id, '' when absent. The worker runs the newspack_nodes/job_worker/before_job FILTER ( $run, $handler, $id, $message ) and fires the …/after_job action ( $handler, $id, $outcome ) around each job, so applications can establish and tear down per-job request context. A before_job listener returning false DECLINES the job — the handler never runs — which is how a plugin refuses work addressed to another host; only an explicit false declines, so an action-style null return carries on. Shorter callables ignore the extra arguments.

400 Bad Request

{ "code": "invalid_partition", "message": "Partition out of range for worker type", "data": { "status": 400 } }

The handler refuses a partition that is negative, at or above MAX_PARTITIONS, or past the type's active partition count. This is the first of its three refusals.

403 Forbidden

{
  "code": "invalid_token",
  "message": "Invalid spawn token",
  "data": { "status": 403 }
}

This is the normal response for unauthenticated callers.

409 Conflict

{ "code": "fleet_held", "message": "fleet held since 2026-09-03T12:00:00+00:00; run `wp nodes start` to resume", "data": { "status": 409 } }

A deploy hold stands in Spawn_Coordinator::HOLD_OPTION, held at the endpoint rather than at each spawner because this is the one gate they all cross.

429 Too Many Requests

{ "code": "spawn_throttled", "message": "job-worker.p0 spawned less than 15s ago", "data": { "status": 429 } }

The {type}|{partition} pair is inside the shared throttle window; see Rate Limiting.

Authentication

This section covers the spawn endpoint only; /command's per-command signing model is Command Signing below.

The handshake above draws the spawn endpoint's dual-mode auth (Spawn_Controller::check_permission) as two validators of one nonce field. There is no env-var bypass — NEWSPACK_NODES_WORKER_TYPE and _PARTITION are written to $_SERVER after auth passes (see Worker Identity Tags) and are consulted nowhere on the permission path.

Application plugins adding their own endpoints should gate them through Capabilities::can() rather than a bare current_user_can( 'manage_options' ), so a site that installs the granular capabilities keeps its read-only callers read-only. See newspack-event-logger-nodes/docs/API.md for the application-side patterns.

Internal Runtime Health

POST  /wp-json/newspack-nodes/v1/health/runtime

Narrow internal loopback endpoint wp nodes doctor uses to read the two health rows whose answer depends on the PHP runtime asking: the cache backend the WEB runtime selects, and whether every sources:<name> its active topologies read resolves in its log-source registry, whose built-ins read the runtime's own ini and constants. A probe run under WP-CLI can report a posture that visitors and workers never get, because WP-CLI selects its own backend and reads its own php.ini. It is not a general cache API and not for public callers.

Request

Field Type Required Description
token string yes A lowercase 64-character HMAC-SHA256 token for the health-runtime purpose.

The body is form-encoded and carries only token, which is hash_hmac( 'sha256', "newspack_nodes_health-runtime:{$window}", wp_salt( 'nonce' ) ), built by Internal_Request_Token::generate() with PURPOSE_HEALTH_RUNTIME, where $window = floor( time() / 10 ). The endpoint accepts the current and immediately previous windows. Purpose separation means a spawn token cannot authorize this route, and a health-runtime token cannot authorize worker spawn.

WordPress REST enforces the required token argument. Omitting it returns HTTP 400 with code rest_missing_callback_param, before the permission callback and before Bootstrap::fleet_gate(), so controller validation never runs either.

When a token is supplied, the fleet gate applies before controller validation: on multisite, only the main fleet site may use the route, and a subsite receives 403 Forbidden. On the fleet site, a malformed, expired, future-window or wrong-purpose token receives 403 Forbidden with code invalid_health_token. Both refusals answer under that one code and echo nothing of what was presented, so a caller learns neither which check failed nor how close its token came.

The route accepts no caller-selected cache key or value. Extra key or value input is never used by the probe and never returned; the server generates and removes its own random probe entry.

Response

After permission succeeds, the route always returns HTTP 200 with exactly two canonical results, Health_Checks::runtime(), in this order:

[
  {
    "id": "cache-backend",
    "label": "Cache backend",
    "status": "good",
    "messages": [
      "Cache backend APCu add/read/delete round trip succeeded."
    ]
  },
  {
    "id": "log-sources",
    "label": "Log sources",
    "status": "good",
    "messages": [
      "Every file the active topologies' File_Tails follow resolves on this host."
    ]
  }
]

The four fields of each are fixed: id, label, status and messages, and messages holds one non-empty diagnostic string of at most Health_Checks::MESSAGE_BYTES (512) bytes holding no control, line- or paragraph-separator character. The cache row is good or critical. The log-sources row is critical when an active topology's File_Tail follows a file that does not resolve on this host, such as a sources:<name> the registry lacks, naming the topology and carrying the load's own refusal, and good otherwise. (wp nodes doctor synthesizes recommended for both rows when the loopback reply cannot be verified whole.) A failed check is a canonical critical result in the same HTTP 200 response: health severity is payload state, not a transport failure.

Worker Identity Tags

The spawn handler sets two $_SERVER keys once auth passes, before it fires newspack_nodes/spawn_worker:

$_SERVER['NEWSPACK_NODES_WORKER_TYPE']      = $type;        // e.g. "job-worker"
$_SERVER['NEWSPACK_NODES_WORKER_PARTITION'] = (string) $partition;  // e.g. "0"

Once Worker_Base::execute() holds that worker's lock, it fires newspack_nodes/worker_identified with the same pair, so a listener that read the keys before they were set — a request logger naming the process at its start — learns the worker's identity mid-process. A spawn that loses the lock race never becomes the worker and announces nothing, and neither does wp nodes run, which does not set a tag. A listener that needs the worker id spells it with CLI::worker_id( $type, $partition ), the inverse of CLI::parse_worker_id().

These are process-identity tags, not credentials. Three things read them:

  • Core::argv0() puts the worker type in every log line's midfix, so a firehose line names the process that wrote it instead of the SAPI.
  • Consumer_Node::checkpoint_frame_extra() stamps worker_type into each offsetlog checkpoint, which is how the dashboard labels a reader by the fleet it belongs to.
  • Consumer plugins keep worker self-traffic out of the global request counters. Log_Manager in event-logger-nodes allow-lists both keys in its environment capture and reads the type to tag its own process (start) frame; Request_Builder_Node reads the type back off that capture and marks the record is_worker, which Flame_Builder_Node files per URL but drops from the global roll-up. Nuclear-gyrobase reads the partition to build the lock lease it hands the Perl engine.

Bootstrap::reconcile_fleet() writes the same keys ('reconcile' and '0') so the WP-Cron reconciliation pass is tagged consistently with topology workers, and a consumer may tag a request of its own the same way — cache-cozy's mu-plugin writes 'cache-cozy' on its warm loopback so that deliberately expensive render lands outside the global stats. The value is a stats dimension, not a worker type: nothing compares against the literal. What every writer shares is that a client can never reach the key, since PHP exposes request headers under an HTTP_ prefix — which is why event-logger-nodes' Auto_Tuner_Node treats the type's mere presence as proof of worker context before it rewrites the ruleset.

Rate Limiting

The spawn endpoint's two limits are in the handshake above: one call per Spawn_Controller::RATE_LIMIT_S (2 seconds) per user on the WordPress-admin path, metered by Rate_Limit below and answering 429 rate_limited, and the 15-second {type}|{partition} throttle (Spawn_Coordinator::MIN_SPAWN_INTERVAL_S) every spawner shares, taken by Spawn_Coordinator::admit_spawn() with one atomic add() in the shared cache, so of concurrent spawns for one slot exactly one is admitted and the rest answer 429 spawn_throttled, or checked and then set in a transient where there is no shared cache. The key lives one second past the window, because memcached expires in whole seconds and a key stored for exactly 15 could lapse early. Only an admitted spawn takes it: a spawn refused for a bad partition or a hold leaves the slot free. Spawning is how the fleet revives itself, so where the shared cache cannot meter the admin path, the spawn is admitted unmetered, with one rate-limited warning, rather than refused.

The /command endpoint applies its own per-user limit (HTTP_In_Node::check_permission): RATE_LIMIT_BURST = 30 POSTs per RATE_LIMIT_WINDOW_S = 1 second per user, answering 429 rate_limited on overflow. Memcached counts expiry in whole seconds, so at one second a slot frees within the second it was claimed in, and the budget is close to one bucket per cache tick; what the limiter adds over a transient counter is the atomic claim. When neither memcached nor APCu answers, the door admits and logs one rate-limited warning, because every dashboard, Status poll and hub push rides /command, and a dead cache must not take them all down. The budget is tunable through the newspack_nodes/command_rate_limit filter, clamped to 1 through Rate_Limit::MAX_BURST (256). The capability is checked first, so an unauthenticated flood spends no slots.

The limit is Rate_Limit::claim( $name, $burst, $window_s ), which the spawn endpoint, the OAuth doors and an application door meter through too, as event-logger-nodes' MCP route does. Each admitted call claims one of $burst slots with the shared cache's atomic add() under a $window_s TTL, so two concurrent requests cannot both take the last slot. Each slot frees on its own clock, but in whole seconds: memcached frees it between $window_s - 1 and $window_s seconds after its claim, and APCu between $window_s and $window_s + 1. It answers Rate_Limit::ADMITTED, THROTTLED or UNAVAILABLE — the last when there is no shared backend or the slot read fails — and each door decides what an unmetered call gets: /command and spawn admit it through Rate_Limit::admit_unmetered(), which warns once per door, and the OAuth doors and MCP refuse it with a 503. A burst outside 1 to MAX_BURST, a window below 1 or a name holding whitespace throws.

Command Signing

Passing /command's permission callback authenticates the request; it signs nothing. Every command inside the batch must carry its own HMAC, stamped by the node that minted it, or the runtime refuses it. Ingress never signs on a caller's behalf, because that makes the boundary an oracle where arrival implies authority (ADR-15). Only the minter's own signature counts.

The auth request's three optional fields and the response drawn as five slots, the signed string drawn as four slots beside the four things that stay outside it, the VALUE.auth envelope, and the five verification checks in order with what each refusal writes to the drop audit and which ceiling it installs.

Establishing a session

POST  /wp-json/newspack-nodes/v1/auth

Issues a session: a random signing key under a random handle (Command_Auth::mint_session()). Gated by the fleet gate then a signed-in user — a scope is a ceiling, so a read-only user minting a manage session still gets a read-only one, and a user holding no role gets a none session, which covers no role and runs only the commands declaring a WordPress capability that user holds (see Service CIs). A labelled request from a user holding no role answers 403 invalid_scope, and Command_Auth::mint_session() refuses a labelled none session outright. Each user's mints claim Auth_Controller::RATE_LIMIT_BURST = 30 slots a minute (RATE_LIMIT_WINDOW_S = 60) under auth:<user>, raised by the newspack_nodes/auth_rate_limit filter and clamped to Rate_Limit::MAX_BURST; over budget answers 429 rate_limited, and with no cache to meter in the mint is admitted unmetered with one warning, as /command does. The handle and the key are both generated server-side; caller entropy is unverifiable, and a caller-chosen handle could collide with or fixate a live session.

Request (all optional)

Field Meaning
scope read, tune or manage. Defaults to manage, and is CLAMPED to the highest role the minting user holds, so the returned scope states authority rather than a request. An unrecognised value answers 400 invalid_scope. A request carrying a label is a session a person keeps, so it may hold only a Capabilities::LABELLED_SCOPES scope, read or tune: it defaults to tune, and manage answers 400 invalid_scope. An unlabelled request, a dashboard's or a hub's own session, may hold manage.
label How the session shows up in the Sessions tab. An empty label keeps it out of the listing. What is stored runs through sanitize_text_field(), loses every C0 and C1 control character and DEL, which that function keeps, and is then truncated to Sessions::MAX_LABEL = 64 characters, so a listing can come back shorter than what was sent, or stripped of markup — the cap is there so a listing cannot be used as storage. sessions create echoes the label as SENT, so the two can disagree the moment the next list runs.
ttl Lifetime in seconds, clamped to [ Command_Auth::SESSION_TTL_MIN_S, SESSION_TTL_MAX_S ] = [60, 86400]. Defaults to SESSION_TTL_S = 3600.

A session the store cannot hold answers 503 session_store_unavailable: Command_Auth::mint_session() throws Session_Store_Unavailable when the durable session store — a row of this install's nodes-sessions-<install> Table on the backend sessions_backend names, SQLite by default — will not open or refuses the row, and issue() catches that type alone, logging the server's reason and keeping it out of the response. mint_session() also throws InvalidArgumentException on a scope neither on the READ/TUNE/MANAGE ladder nor none, which issue() refuses first with the 400 above; a client cannot ask for none, which issue() grants alone. A hub whose HTTP_Out handshake fails logs auth failed at spoke: HTTP <status> <code>, naming the code the spoke's body carries — HTTP 503 session_store_unavailable for this outage — or the bare status when it carries none, and holds its batch for the next attempt.

Response

{
  "handle": "5f2b...(32 hex chars)",
  "secret": "9ac4...(64 hex chars)",
  "scope": "read",
  "expires_in": 3600,
  "now": 1735689600
}

The signing key is disclosed as secret because that is the field name Core::is_secret_property() recognises. Every redactor on both sides of the wire asks that one rule, so a reply rendered into a drop-audit line or persisted to a browser's transcript is masked by its name alone.

The key cannot be recovered from the Sessions listing. The session also records the minting user; without it, a credential presented outside a browser would authenticate and then act as nobody.

Signing a command

The minting node signs before the command leaves the process — the browser's Shell, a dashboard hook, or a PHP caller of Command_Auth::sign() / sign_for(). The result rides under VALUE.auth:

{ "auth": { "nonce": "b91e...(32 hex chars)", "sig": "7cd0...(64-char hex HMAC)", "handle": "5f2b...(session handle)" } }

Repointing an envelope at another handle only makes the signature stop matching, because the handle stays outside the signed string.

Verification

The /command request process installs Command_Auth::verifier() as every interpreter's authorize policy; the five checks, their order and the ceiling each outcome installs are in the diagram above. Beside the ceiling it installs Capabilities::$session_label: the session's label, '' for a session minted without one, and null for the per-site secret or a refusal; interpret() restores both when the command ends. A refused command replies TM_COMMAND|TM_ERROR carrying unauthorized: <verb> through the normal TO=FROM path, and the containing /command batch answers 401 instead of 202 or 200 (see Command Dispatch below). The drop audit is the rate-limited line Node::drop_message() writes as verification failed: …, which goes to the PHP error log alone, never into the response, an SSE stream, a worker's IPC output or the dmesg ring. When the command named a live site-wide session, the line ends handle: <handle>, so an operator can tell which client is misconfigured; it never carries the key, the signature, the nonce or the command's payload. The handle rides in the line's tail, outside the throttle key, so the log keeps one line per reason per process, so once per POST in the request scope, naming the first session refused for it, however many sessions a caller mints. A dead or grant handle is named nowhere. Every command the verifier judges and refuses — a wrong key, a stale or malformed envelope, a dead or grant handle — therefore answers the same bytes, naming neither the cause nor the process. An environment fault still reports to stderr, and so into the body: no nonce store, reached only after the signature verifies, and a session store that will not answer.

OAuth Authorization Server

The site is its own OAuth 2.1 authorization server, so an MCP client such as Claude signs in through the browser instead of being handed a session. Oauth_Server decides every answer and Rest\Oauth_Controller sends it; Oauth_Store keeps the clients, codes, grants and refresh tokens beside the sessions. Every endpoint refuses a request that did not arrive over HTTPS with 400 invalid_request, and none authenticates a user: on a multisite subsite the two routes answer 403 newspack_nodes_not_fleet_site and the documents are left to WordPress.

Metadata

A parse_request handler, Oauth_Controller::serve_well_known(), answers two paths outside the REST namespace as JSON, with Access-Control-Allow-Origin: * and Cache-Control: no-store, and ends the request; every other path passes through to WordPress untouched, and the hook tests for .well-known/oauth- before it loads either OAuth class. It reads WP::$request, which WordPress fills only under pretty permalinks, and a declared resource's trailing slash is ignored, because WordPress trims it.

Path Document
/.well-known/oauth-authorization-server RFC 8414: issuer (the home URL, no trailing slash), authorization_endpoint (wp-admin/admin.php?page=newspack-nodes-authorize), token_endpoint, registration_endpoint, response_types_supported: ["code"], grant_types_supported: ["authorization_code", "refresh_token"], code_challenge_methods_supported: ["S256"], token_endpoint_auth_methods_supported: ["none"], scopes_supported
/.well-known/oauth-protected-resource/<path> RFC 9728, for a resource whose URI path is <path>: resource, authorization_servers: [<issuer>], scopes_supported, bearer_methods_supported: ["header"]. A path that does not belong to any declared resource answers 404 invalid_target

scopes_supported is Oauth_Store::SCOPES, read, tune and manage. A door declares each resource it protects through the newspack_nodes/oauth_resources filter, and names the document's URL with Oauth_Server::resource_metadata_url( $resource ).

POST /oauth/register

Dynamic client registration (RFC 7591), for public clients only. The JSON body, and nothing else (a query string is not read), names client_name and redirect_uris: one to five URIs, each https:// or http:// on localhost or 127.0.0.1, with no userinfo, fragment or whitespace, and none that wp_sanitize_redirect(), which wp_redirect() runs, would alter: a quote, a brace or a non-ASCII byte is refused, because the approval page would match the URI and then send the browser elsewhere. A token_endpoint_auth_method other than none is refused. The answer is 201 with client_id, client_id_issued_at, client_name (kept as a session label is), redirect_uris, token_endpoint_auth_method: "none", grant_types and response_types: ["code"]; the server issues no secret. A missing name answers 400 invalid_client_metadata, a refused URI list 400 invalid_redirect_uri. A client lapses 180 days after registering.

Ten registrations an hour are admitted per caller address (oauth-register:<ip>); the eleventh answers 429 rate_limited. Claude Desktop and claude.ai register from Anthropic's servers, not from the person's machine, so every connector set up on one site shares that budget: an eleventh connection added within the hour is refused until the window passes.

Authorization: the approval page

authorization_endpoint is a wp-admin page, wp-admin/admin.php?page=newspack-nodes-authorize, hidden from the menu and open to any user holding read's capability (Admin\Oauth_Consent). wp-admin sends a visitor who is not logged in to wp-login.php and back. The query names:

Parameter Must be
response_type code
client_id A live registration
redirect_uri Byte for byte one the client registered
code_challenge_method S256
code_challenge Present
state Present
resource One an application declares through newspack_nodes/oauth_resources

The page also refuses plain HTTP, a multisite subsite, a parameter that would not come back unchanged through the form's hidden fields (an entity such as &amp; in a state, invalid UTF-8, or a control character such as a line break, which a redirect strips), and a user who can grant none of the scopes. A refusal is shown on the page and nothing is redirected, so an unregistered redirect URI is printed, never followed. Otherwise the page names the site, the client's self-declared name and, in large type, the redirect URI's host; a host other than claude.ai, claude.com, localhost or 127.0.0.1 carries the warning "This connection sends its credential to <host>, which is not Claude." It offers the scopes of Oauth_Store::SCOPES that Oauth_Server::can_grant() admits, each named and described by Capabilities::label(), as the Sessions tab describes it, the broadest checked, and asks for a connection name of up to Sessions::MAX_LABEL (64) bytes, which labels the grant and every session minted from it.

Approve and Deny post to admin-post.php with action=newspack_nodes_oauth_decide and the nonce field newspack_nodes_oauth_nonce. The handler dies on a missing or wrong nonce, then checks every parameter again and dies on any refusal, or on a scope the user cannot grant. Deny, or a post without a decision, redirects to the registered URI with error=access_denied&state=<state>. Approve naming no connection returns to the page, which asks for one; otherwise it issues a code that lives 60 seconds, bound to the client, the redirect URI, the challenge, the resource, the user, the scope and the name, and redirects to the registered URI with code=<code>&state=<state>. Both redirects go through wp_redirect(), because wp_safe_redirect() refuses every host but the site's host.

POST /oauth/token

A form-encoded token request naming grant_type and client_id. Only the form body is read: a credential in the query string would land in access logs, so a request carrying its fields there is answered as if it named none. A client_id that is not held by any live registration answers 401 invalid_client, which tells the client to register again; it spends no rate-limit slot. Thirty requests a minute are then admitted per client (oauth-token:<client>).

grant_type Also names Checks
authorization_code code, redirect_uri, code_verifier, resource The code is taken once, so a refused exchange spends it too. It must have been issued to this client for this exact redirect_uri and resource, and base64url( sha256( code_verifier ) ) must equal its challenge. A new thirty-day grant opens
refresh_token refresh_token, optionally resource The token rotates through Oauth_Store::rotate(): a retry within 60 s is answered with the same pair, and a token reused later revokes its grant. A token issued to another client, or a resource the grant was not issued for, is refused and spends nothing, a retry inside the 60 s included; a request without a resource is bound to the grant's resource

Before either mints a session, the grant's user is signed in for the one check that they still hold its scope (Oauth_Server::can_grant( $scope ), the check the approval page offers scopes by), then signed out again. A user who lost it ends the grant and every session it minted. The answer is 200:

{ "access_token": "<handle>.<secret>", "token_type": "Bearer", "expires_in": 3600, "refresh_token": "<64 hex>", "scope": "tune" }

expires_in is less than 3600 in a grant's last hour. Every answer carries Cache-Control: no-store and Pragma: no-cache.

Status error When
400 invalid_request A missing grant_type or field, or plain HTTP
400 unsupported_grant_type Any other grant
400 invalid_grant A code or refresh token unknown, expired, spent, or issued for another client, redirect URI or resource; a user who lost the scope
401 invalid_client No such client
429 rate_limited The client's budget is spent
503 rate_limit_unavailable Neither memcached nor APCu can meter the request, so it is refused, as the MCP door refuses one
503 temporarily_unavailable The session store did not answer; its cause goes to the log

The registration route answers the same 503s, and both routes carry the error's description in error_description.

MCP doors

A plugin serves an MCP server over commands it already declares by subclassing Mcp_Door and calling wire() (ADR-38). The route sits under the plugin's own namespace, so the substrate's own door, described below, answers at /newspack-nodes/v1/mcp and an application's at its namespace instead:

Route Method Permission
/<namespace>/mcp POST The fleet site; a Bearer <handle>.<key> live session admitted at this door's resource; a per-session, per-door rate limit

wire() hooks four things: register_routes() on rest_api_init, the body cap on rest_pre_dispatch, point_to_sign_in() on rest_post_dispatch, and the door's resource() on newspack_nodes/oauth_resources. The resource is the route's REST URL over https, the host lowercased and no trailing slash.

Authentication. check_permission() takes a command session, minted by hand or by an OAuth grant for this door's resource; a grant session for another resource is refused. It signs in as the session's minting user and installs the session's scope as the request's ceiling, so a tool is listed and called only where Capabilities::can() admits the role its command declares. It installs the session's label as Capabilities::$session_label, which a command reads to name the caller: '' for a session minted without one. Then, and only then, so an unauthenticated flood spends no slots, it claims one of RATE_LIMIT_BURST (20) slots named mcp:<namespace>:<handle> through Rate_Limit, an atomic add() on Cache_Backend::shared_first() under a RATE_LIMIT_WINDOW_S (10 s) TTL, which memcached frees 9 to 10 s after the claim and APCu 10 to 11 s.

Status code When
401 mcp_unauthorized No credential, a session unknown or expired, a wrong key, or a session issued for another resource; the response carries WWW-Authenticate: Bearer resource_metadata="<url>", which names where to sign in
403 newspack_nodes_not_fleet_site A multisite subsite
429 rate_limited The session spent this door's budget
503 rate_limit_unavailable Neither memcached nor APCu can meter the call

Protocol. One POST carries JSON-RPC 2.0, MCP revision 2025-06-18: initialize, tools/list and tools/call. A body to the door's own route over Command_Interpreter_Node::MAX_BODY_BYTES (1048576 bytes) is refused on rest_pre_dispatch, with a 413 carrying -32600 Request body exceeds 1048576 bytes. and a null id; one exactly at the cap is read. That filter runs before WordPress decodes a JSON body, which it does while validating the request's parameters, and before the permission check, so an oversized body answers 413 whether or not it carries a live session. A message with no id is a notification and is answered with a bodiless 202; a body that is not JSON-RPC 2.0 answers -32600 with a null id, and an unknown method or a tool the session may not call answers -32601. A tool's argument object rides to its command as --name=value tokens, a list as one token a member, and a command's refusal comes back as a tool result with isError true, in the words Core::caller_message() lets the session read (see Error words). Every reply leaves as JSON encoded with JSON_HEX_TAG inside one <site-data> fence, so a payload cannot close the tag, and initialize serves FENCE_NOTE, the sentence saying what the tag means, ahead of the door's own instructions and its caveat. The server's methods are final; a door declares, it does not override.

Deriving the tools. Mcp_Door::declared_tools( $node, $class, $prefix, $except = [], $text = [] ), final protected static, answers one tool per command $class declares but the $except verbs, for the node mounted as $node. A tool is named <prefix>_<verb>, summarized by the command's own description and offers every arg the command declares, its text {declared}, which tools/list fills with that arg's own description and throws \LogicException for an arg declaring none; $text, keyed by verb, overrides a tool's name, summary or args. An $except or $text entry naming a verb the class does not declare throws \LogicException, so a door cannot drift from the commands it fronts.

The substrate's door

Nodes_Mcp_Door answers at POST /wp-json/newspack-nodes/v1/mcp, its serverInfo naming newspack-nodes and the plugin's version. Its tools are declared_tools() over each node in Bootstrap::SUBSTRATE_CIS, the table the substrate mounts its service CIs from, with the hyphen in raw-logs written as an underscore, so a command added to one of those classes is a tool with no edit to the door. No compositions command is a tool, because a layout is page furniture, not data. A session sees the tools its scope covers:

Role Tools
read (16) status_get; workers_list, workers_dump_graph, workers_dump_cleanup, workers_inspect; topologies_dump, topologies_get, topologies_expand; raw_logs_list_logs, raw_logs_dump_log, raw_logs_read_message; classes_dump; settings_get; layouts_get; aggregator_summary, aggregator_list_servers
tune (2) settings_set, layouts_save
manage (12) workers_restart; topologies_save, topologies_delete, topologies_activate, topologies_deactivate; aggregator_probe; vault_list, vault_get, vault_delete, vault_test; sessions_list, sessions_revoke

A read session therefore lists 16 tools, a tune session 18 and a manage session all 30. manage reaches this door through an OAuth grant for it, or through the unlabelled session a dashboard mints; a session issued by hand holds at most tune. A manage grant mints no other session, and it carries its approver's whole authority over the fleet: topologies_save and topologies_activate run any graph, so a File_Tail can follow any file the PHP user can read, wp-config.php and its salts included, into a log raw_logs_dump_log reads back, and a Curl node can make requests to any host the site reaches. Revoking the grant ends its sessions and does not recall what it already read. The approval page checks the scope the client asked for when the user can grant it, and tune otherwise (ADR-36). declared_tools() leaves out every command declaring a secret arg, and Nodes_Mcp_Door::NOT_TOOLS five more; calling one answers -32601 Unknown tool: <name>:

Command Why it is not a tool
sessions create It mints a bearer key
vault add, vault update Each declares its password secret, which would pass through the agent's transcript
topologies connect_worker_input, topologies mount_tables Each mounts nodes into the request's graph, and an MCP call is one request that ends with its reply
workers heartbeat It refreshes a session's SSE slot, and an MCP session holds none
raw-logs read_block It answers through the envelope's FROM, and a door hands it no envelope; dump_log and read_message read logs

vault list and vault get answer through public_shape(), which drops the password. The door's caveat is one clause, and closes every tool description; the door's instructions name, once, the manage tools that change the running fleet — restarting workers, saving, activating, deactivating and deleting topologies, revoking sessions and deleting Vault spokes — so an agent calls them only on the operator's own request.

Command Dispatch

POST  /wp-json/newspack-nodes/v1/command

Unified non-streaming dispatch endpoint. The browser POSTs a TM_COMMAND envelope; the controller routes it through the request-scope _router to the named CI; the CI's reply walks back via TO=FROM through _output — an HTTP_In_Node, a double-duty class that is BOTH the /command REST controller and the egress Node registered as _output (Node_Names::OUTPUT) — whose fill() writes the packed Message directly to the HTTP response body. _output is the egress name inside an SSE stream process too, where an HTTP_Filter_Node holds it. In the browser graph the two names split: _http is the HttpOutNode that POSTs the batch, and _output is the local sink an unaddressed reply lands on.

The permission callback (HTTP_In_Node::check_permission) authenticates the request and does not sign any command inside it — see Command Signing. A signed-in user is the FLOOR every command behind the endpoint needs, because a command may declare a WordPress capability any user may hold, not the level any of them demands; who may call what is drawn under Service CIs.

The door routes a message only where the request graph meant it to go (ADR-23). A message whose TO's first segment names no node, an empty TO included, routes on: the base interpreter bounces a TM_PING or TM_EOF, and the Router answers the rest NOT_AVAILABLE. A request command may also go TO a bare name that holds an interpreter. Any other message, and a command whose TO runs past its first segment, may go only TO a node a gated mount built earlier in the same POST: a worker's input Partition through topologies connect_worker_input, or a Table through topologies mount_tables. A caller holding no role may send only a request command whose TYPE is TM_COMMAND and nothing else, to an interpreter. Each refused message is answered after the batch with a TM_RESPONSE|TM_ERROR frame carrying its ID — the door routes a command to an interpreter, or a message to a node this POST mounted, or for a roleless caller a caller holding no role sends a bare command to an interpreter alone — without the 401 a signature failure raises.

Request

The body is JSONL — one packed Message per line, where each line is the JSON of the substrate's 7-slot positional array [TYPE, TIMESTAMP, FROM, TO, ID, KEY, VALUE] (the wire form Message::packed() emits). Multiple lines in one POST batch through the request-scope graph serially, so an earlier command's side effect is visible to a later one — a client sending connect_worker_input or mount_tables ahead of the command or request it enables depends on that. Blank lines are skipped; every other line must decode to a 7-element positional array, and Message::unpacked() throws on the first that does not. A body carrying no parseable line throws as well.

That mount is not a pure graph edit. Bootstrap::register_worker_partition() reads the id through CLI::parse_worker_id(), which refuses any string CLI::worker_id() cannot write — a / or NUL in the type, a leading zero in the partition — then returns early when the partition is already mounted, and otherwise resolves the worker through Spawn_Coordinator::worker_channel(). A sleeping on-demand worker, which holds no lock dir, skips the ipc/{id}/input existence check, because it creates that directory only once it runs, and is woken through Spawn_Coordinator::wake_channel() once its input is mounted — so attaching a browser console to an idle on-demand slot STARTS a process. A mount refusal is silent, since the command answers an empty string, and the command behind it bounces NOT_AVAILABLE instead. A wake the spawn endpoint refuses is not: the command answers a TM_COMMAND|TM_ERROR to its FROM whose payload reads wake failed for {id}: HTTP {status}: {the endpoint's message}, or the transport's error when the POST never landed, in the same response body, and the commands behind it still land in the mounted input for when the worker starts. The endpoint's message is cut to its first line and 200 bytes, tags read as spaces. A 429 spawn_throttled is another process starting that worker, and answers nothing; any other 429 is a refusal like the rest.

topologies mount_tables <topology> also answers an empty string, but refuses out loud: an inactive topology draws a TM_ERROR reading mount_tables: <topology> is not active, and a Table whose backend cannot open fails the command naming the Table, unmounting whatever the call had mounted. In PHP, Bootstrap::mount_table() throws Table_Unavailable for that backend, and a plain \RuntimeException for a Table two active topologies declare differently or a TTL that is not a whole number of at least 1. Each mount is named {table}.p{N} and serves one read alone, GET (ADR-23); Other Node Primitives carries the Table protocol. A Row_Table mounts nowhere: mount_tables leaves it out, because only its worker answers ROWS, and a reader opens its files through Row_Table_Node::read( $schema, … ), naming the schema, which Bootstrap::row_table() resolves to the one table declaring it, an answer cached across processes for up to 300 seconds. Row_Table_Node::build( $schema, $name, … ) resolves the table afresh through Bootstrap::resolve_row_table(), as every path writing or deleting its files does, attaches the same files read-only into a connection writing a build file beside {base}/tables/<name>.sqlite, and renames it over that snapshot when the build answers anything but false, for a job publishing a derived file; Row_Table_Node::open_snapshot( $name ) reads the published one, immutable, and remove_snapshot( $name ) deletes it with any build a crash left (ADR-35).

Send that body as text/plain; charset=UTF-8 — the COMMAND_CONTENT_TYPE the browser transport declares — and never as application/json; the diagram under Response shows why.

Per-slot semantics (named here for documentation only — the wire is positional):

Slot Type Description
TYPE (index 0) int Bitmask. TM_COMMAND (8) for a dispatch.
TIMESTAMP (index 1) float Unix timestamp. The signature covers it truncated to whole seconds, and the freshness window is checked against it.
FROM (index 2) string Reply path. HTTP_In_Node stamps _output onto it on the way in, so a bare reply path (_output, _sse:{session}/…, or empty) walks back to this endpoint. The _sse form is minted by the browser's RemoteIpcNode, which rewrites a non-empty reply-node FROM to _sse:{session}/{node} before sending, naming the handle of the page's command session — that is how the server's HTTP_Filter_Node passes an async reply to the stream this session holds, across reconnects, rather than to another tab's.
TO (index 3) string CI node name (e.g. topologies, workers). Router peels the head off; subpaths flow through. Empty TO is dispatched by the base CI in-place.
ID (index 4) string Caller-chosen correlation id. The CI's reply carries the same ID.
KEY (index 5) string Routing and correlation metadata (e.g. 'completion' triggers REPL completion-list mode on help and ls).
VALUE (index 6) array The inner Command_Interpreter envelope {name, arguments} as a live JSON array. name is the verb; arguments is a flat token array (list<string> argv) — the Shell and the browser transport tokenize ONCE at the producer boundary, and the tokens ride verbatim through envelope, interpreter and make_node. Every command, scalar and structured alike, reads its data from that array ($args[0], $args[1], …). VALUE also carries auth, the HMAC envelope every minter stamps; see Command Signing.

The browser's command transport and the attached wp nodes cli both produce this exact wire shape via Message::packed().

Response

Seven stages of one /command POST in order, each with the outcome it can produce: the door's WordPress error objects, the uncaught-exception page for an unparseable body, the packed 500 for a missing graph, the FROM that overflows 1024 bytes, then inside interpret() the struct drop, the signature refusal that raises the 401 latch and the capability refusal that leaves the status at 200 or 202, the reply shapes, and the rule that the first write spends the status; beneath, the refusals the browser transport mints itself with undelivered: true.

Synchronous (in-process reply)

The CI's interpret() produced a reply, and HTTP_In_Node::fill() (the egress side) sent 200 and wrote the packed Message to the body.

202 Accepted (async / IPC)

The status line alone, with an empty body: the batch routed without the _output egress seeing a reply. Because Command_Auth decides signability from the TYPE bit rather than by sniffing for name, a malformed inner envelope signs cleanly and then vanishes into the struct drop, which runs before verification.

401 Unauthorized

The batch answers 401 when any command in it failed Command_Auth verification; the 401 is the fast signal a client checks before parsing the body. A capability refusal is not this 401: Capabilities::require_command() throws RuntimeException( "permission denied: <role> capability required" ) from Command_Interpreter_Node::dispatch(), after authorize has already passed.

500 Internal Server Error

Sent as a packed positional Message, the same wire shape as the request. Example body:

[288, 1735689600.5, "_command", "<request from>", "<request id>", "", "request-scope graph not initialized (missing _router or _output)"]

TYPE = 288 = TM_RESPONSE | TM_ERROR (256 | 32). HTTP_In_Node::emit_error() sends it; operational application errors never reach this path.

Refusals the client mints

A batch turned away at the door never reaches a command, so the browser transport answers its own caller rather than let a node wait out its deadline: postBatch() mints the undelivered: true refusals the diagram shows.

Service CIs

The substrate mounts eleven service CIs through newspack_nodes/request_graph_ready (newspack_nodes_mount_substrate_cis() in newspack-nodes.php), one per entry in Bootstrap::SUBSTRATE_CIS, the table of mount name to class that the substrate's MCP door reads too. Each is a Service_CI_Node declaring its commands once in node_schema(), and the base derives the dispatch table from that declaration, while dispatch() refuses each command below the role it declares (ADR-26). A command that declares no capability demands MANAGE, so silence is the strictest role rather than the loosest.

Node name Class Commands (role)
classes Classes_CI_Node dump (read)
layouts Layouts_CI_Node get (read), save (tune)
compositions Compositions_CI_Node get (read), save, restore (tune)
topologies Topologies_CI_Node dump (read), get (read), expand (read), save, delete, activate, deactivate, connect_worker_input, mount_tables (manage)
raw-logs Raw_Logs_CI_Node list_logs, dump_log, read_message, read_block (read; a log carrying a read-capability marker also needs its capability, ADR-39)
vault Vault_CI_Node list, get, add, update, delete, test (manage)
aggregator Aggregator_CI_Node summary (read), list_servers (read), probe (manage — on-demand per-spoke deep roll-up)
settings Settings_CI_Node get (read), set (tune)
status Status_CI_Node get (read)
sessions Sessions_CI_Node list, create, revoke (manage — issuing one hands out access)
workers Workers_CI_Node list, dump_graph, dump_cleanup, heartbeat, inspect (read), restart (manage)

Every service CI's commands as chips colored by the role each declares, read, tune or manage, with the help command each CI answers at read; the root interpreter's fifteen read-only commands beside its manage commands and their aliases; and three notes on the dispatch-time gate, the Shell builtins that mint nothing, and the three-level secure ladder.

The first column is the NODE name — make_node's second argument, and what a caller puts in TO. A CI's SHELL name is a different string: the class short name minus _Node, so Layouts_CI_Node is addressed as layouts and described as Layouts_CI. Command_Interpreter_Node::shell_name_for() derives that name; the classes CI's dump command reports it under shell_name, dump_metadata returns it as class, and the topology console's Inspector looks a node's commands up by it. It is also what help Layouts_CI renders a schema for, because Bootstrap registers the Newspack_Nodes\Rest\ prefix alongside Newspack_Nodes\.

workers inspect runs one read command inside a live worker. inspect <worker> <verb> [<arg>…] [--path=<node>] sends the command named by verb and its args into the worker <type>.p<n> and answers that command's reply. The command declares its args as worker, verb, args and path, in that order, so the variadic args takes every positional token after the verb and path is given only as --path=<node>. verb must have a READ base role, by Command_Interpreter_Node::base_role( $verb ), the rule dispatch() reads for a command every schema leaves undeclared (ls, dump_node, stats, dmesg and the rest); path must be empty, the worker's own interpreter, or name a <node>:config interpreter, a path holding no /. Any other verb, a different case or a trailing space included, and any other path are refused before a channel opens, so a read session reaches neither a worker's mutating commands nor a node that would store the command rather than interpret it. The command is signed with the site key for that one call (ADR-23, ADR-15). An unknown worker is refused, and a sleeping on-demand worker is woken as wp nodes cli wakes it.

The round trip is the CLI instance's ask_workers( $receiver, $asks, $timeout_s ), the one wp nodes tables makes, bounded by Workers_CI_Node::INSPECT_TIMEOUT_S (5 s). Each call mints its own reply session, _cli:<pid>-<suffix> with a random suffix, because in php-fpm one pid serves many requests and two calls under one head would read each other's replies. It reuses a worker's input Partition the request already registered, as topologies connect_worker_input registers one, takes _router and _command_interpreter through the get-or-build helper Bootstrap::mount_request_graph() uses, records each node it builds as it builds it, and removes those alone in a finally, whatever the outcome. A worker silent for 5 s is refused by name, and because a command whose answer is empty sends no reply, the refusal says so. A worker's own refusal comes back as the command's error, read by CLI::refusal(), which answers null for a reply that is not a TM_ERROR.

vault add and update take a group. A server's optional group option, held to the same id rule as its own id, groups it with any other server sharing the name; public_shape() echoes it back on list and get alongside url, auth_username, auth_header, auth_scheme and has_credentials. It is the field a Vault_Group node reads to decide its members.

vault add and update take a header and a scheme. An entry with a password, no username and a header sends <header>: <scheme> <password>, or <header>: <password> with no scheme, where an entry with a username and a password sends Authorization: Basic, and any other entry sends nothing. Vault::credential_header() is that one rule for every transport. The header is empty for none, or 1 to 64 letters, digits and hyphens, and never one of Vault::FRAMING_HEADERS (Host, Cookie, Content-Length, Transfer-Encoding, Connection, Expect, Upgrade, TE, Keep-Alive, Proxy-Connection, Proxy-Authorization) or Content-Type, in any case; the scheme is empty or 1 to 32 of the same. Each command refuses either by name, invalid header: <header> or invalid scheme: <scheme>, and an absent option leaves the stored one alone on update. has_credentials is true when the entry sends a line.

workers.dump_graph vs dump_metadata — different commands, different shapes. The workers CI's dump_graph returns the dashboard payload:

{ workers[], unreadable, consumers[], unparseable_lines, refused_producers,
  logs[], log_partitions, deadletter_segments, deadletter_by_reader,
  num_partitions, max_segments, segment_size, timestamp, heartbeat_interval_s,
  graph }

unreadable maps each active topology that will not read to its message, and graph omits it. refused_producers maps each registered log producer template that declares no dir under <config:logs_dir> to its message, and logs[] omits its dirs. Both are [] when nothing failed, and the Overview tab renders each entry as an error banner.

Where each dump_graph key comes from, one row per source: the lock dirs to workers[] with its seven fields, the last 512 KiB of the topicprobe log to consumers[] with its nine fields, the stale re-measure and the unparseable_lines count, the log catalog to logs[] and its join key, the logs root to log_partitions, the dead-letter dirs to the two dead-letter keys, config to the five scalars, and the active topologies to graph; beneath, the two readers of the one snapshot, the dashboard's dump_graph and Alerts::evaluate().

consumers[] is a report, not an inventory: CLI::consumer_rows() builds it from one Probe_Record per reader in the shared topicprobe log, so a reader that has not reported recently drops out instead of reading stale, and a record aged past Topic_Probe_Node::stale_after_s() is re-measured off disk. Every worker appends to that one log, and a short write can leave a torn line there; unparseable_lines counts the lines in the window that would not unpack and were skipped, and wp nodes status warns with the same count. Alerts::evaluate() reads the same snapshot, so an alert can never name a fleet the dashboard does not show.

The same log carries one Partition record per log a worker covers, its READER blank and its SOURCE the log's SSE stamp (ingest.p0, offsets:ingest.p0), or its path under the runtime base outside those roots, holding the log's byte length (END_BYTES) and the disk it takes (END_DISK_BYTES). consumers[] keys by READER and skips them; the Overview's Partition Size and On Disk panels read them off the stream.

workers dump_cleanup names candidates, not casualties. It answers { logs_dir, on_disk_basenames, expected_basenames, orphans } — every first-level dir under logs/, the set Log_Cleaner::declared_log_dirs() retains, and the difference between them. Both halves are the calls the sweep itself makes, but the diagnostic applies NO grace window, where the sweep and wp nodes gc spare any dir whose newest inner mtime is under Log_Cleaner::DELETE_GRACE_S (3600s). A dir an in-flight deploy created moments ago therefore lists as an orphan here while gc would still leave it alone, so reading this command as "what the next sweep deletes" overcounts.

sessions list — the labelled sessions and the OAuth connections, never a key or a token. It returns { sessions[], connections[], ttl_max, scopes{} }, each session { handle, label, scope, created, expires } and each connection { grant, name, client, scope, created, expires }: the grant id, the name its approver gave it, the name its client registered (empty once that registration has lapsed), the approved scope, when it was approved and when it expires, thirty days on. Oauth_Store::grants() lists a connection whether or not it holds a live session, reading the oauth:grants set, then every grant it names in one read, then their clients in one more; an ended grant is left out. ttl_max is Command_Auth::SESSION_TTL_MAX_S and scopes maps each of Capabilities::LABELLED_SCOPES, read and tune, to Capabilities::label(), the words the OAuth approval page shows for it, sent so a client draws its TTL bound, scope picker and scope descriptions from the substrate rather than a second copy that drifts into offering a scope the mint refuses. Every row is live: Sessions::listing() reads the session Table's labelled set, then every row it names in one read, takes each fact from the session's row, and leaves out a session whose row has gone — lapsed, a sessions revoke, or wp nodes session revoke --all. A store that does not answer, or a set holding more than 10,000 handles, refuses the command rather than listing part or nothing. Reading the listing writes nothing.

sessions create and sessions revoke. create [<label>] [<scope>] [<ttl>] binds each arg by position or by name, so create chris-claude tune 86400 and create chris-claude --scope=tune --ttl=86400 mint the same session; scope defaults to tune, refuses manage (only Capabilities::LABELLED_SCOPES scopes) and is clamped to what the issuing user holds, and ttl to Command_Auth::SESSION_TTL_MIN_S–SESSION_TTL_MAX_S. revoke <id> takes a session's handle or a connection's grant id and answers { id, revoked: true } only when it dropped a session's row or ended a grant. Revoking a connection ends its grant, so it refreshes no more, and every session it minted, as revoking one of those sessions does. A store that did not answer refuses session store did not answer; <id> may still be live and leaves the listing as it was. An id naming neither refuses no session or connection with id <id>, and when <id> is a session's LABEL — what an operator reading the Sessions tab types — the refusal names the handles carrying it: no session or connection with id <id>; the label <id> names <handle>, ….

topologies expand and topologies get compose different views. expand returns Topology_Analyzer::expand()'s shape for an include SET — { nodes, edges, tree, hulls }. Both commands take --group_children=false, which leaves each Vault_Group as written: no derived member, in nodes, edges or hulls. The topology console's editor seeds from that form; every other reader gets the members. Either form draws the group's own pair edges, the ones its child class declares, on the group. A node carries name, class, fans_out, args, commands, origin and via — each of commands a { verb, args } invocation the topology aims at the node; an edge carries from, to, origin, roles — connect, config, pair, or several — and, where a config role exists, config_slots naming the setter verbs that made it. A pair edge runs from a Remote_Source, or a Vault_Group of them, to the target one of its add_source lines name, which the broker writes through that source's reader; no connect_node or disconnect_node adds or removes one. origin is a LIST because a diamond include is provided by several directly-declared includes, while via is only the first path a node entered through. hulls maps every topology in the tree, at any depth, to the node names it declares, depth-first so an outer topology precedes what it brings and the canvas paints the nested hull on top. The whole payload is informational — the runtime is the Shell's include — and it is the topology console's edit-mode baseline.

topologies get <name> returns { name, source, tsl, includes, expanded, resolved_config_edges }, and its expanded is that same shape built from the file's DIRECT includes alone: it holds the borrowed members and none of the file's own lines, which is what lets the editor render a borrowed node as borrowed. --group_children governs expanded alone: resolved_config_edges comes from the member-bearing walk. resolved_config_edges covers the whole topology, because a config command pointed at a <namespace:key> token names an edge only the server can resolve. A client parsing a document that carries such a token must treat a response without that list as fatal rather than wire an edge to the literal token text; the console's document loader throws there, and augmentWithVirtualEdges() does not draw any edge at all for a token the server resolved to nothing. A node an owner builds for itself, such as a Crawler's {name}:seen Table, is not in any reply: a canvas never draws one, and it opens from the "…" on its owner's card.

aggregator — one id field, two meanings. list_servers returns one row per wired Remote_Source, keyed by the NODE name in id and carrying the Vault credential id separately as vault_id, the topology it is wired in, its partitions, each worker partition's status snapshot as the broker publishes it through Publishes_Status under status:<broker>:p<partition> and empty until the broker publishes one, and its readers: one { stamp, partition, cursor, distance } per stream reader the probe log reports, stamp as the spoke names the log, partition the hub worker's, cursor in the read-position grammar (a segment the reader does not know is :<offset>), and distance the bytes the spoke last reported past its own reader plus what the hub reader holds undrained, null until the spoke reports. list_servers alone reads the probe log; summary counts connections and never decodes it. probe <id> wants the VAULT id, answers server not found: <id> for anything else, and echoes it back as its own id. A client that carries id straight from one command into the other is refused. The probe's reply is the whitelisted roll-up { id, workers: { total, live, stale, held, idle, down }, worst_distance, deadletter_segments }, counting the spoke's workers by the state each workers[] row carries — the word wp nodes status prints for that slot — so a spoke reporting any other state fails the probe naming the worker, and worst_distance is the largest known distance across the spoke's consumers[], null when every reader's distance is unmeasurable. summary reduces the same snapshot to { connected, idle, total, server_now, unreadable } — unreadable maps each active topology that will not read to its message, and the Aggregator Status tab renders each as an error banner — and its per-server reading is best partition wins: connected while any partition is connected, idle while every partition is disconnected but one carries a scheduled_reconnect_at, and down otherwise — so a partition mid-handshake or in error backoff counts toward down.

settings — get and set do not cover the same keys. get answers the seven storage settings alone — num_partitions, segment_size, min_segments, num_segments, min_lifetime, lifetime, max_segments — as EFFECTIVE values through Config::value(), so a key never saved reports its schema default rather than an empty option. set reaches every int Field declaring a minimum: those seven plus the six remote_* spoke-geometry keys, the three alert_* thresholds and the four bounded sse_* limits, whether or not the settings page renders them. An int Field with no minimum (sse_idle_timeout, sse_max_lifetime, sse_retry_ms, on_demand_idle) and every Field of another type (base_directory, memcache_servers, log_sources, vault) is refused as unknown setting: <name>. The option name is accepted either way it is spelled, short key or full newspack_nodes_ option name, which is how Settings_Sync_Node pushing the full name and an operator typing the short key drive one command unmodified.

Two set behaviours differ from the settings page. A value outside the Field's bounds is REFUSED rather than clamped, throwing invalid value for setting: <key> as a TM_COMMAND|TM_ERROR reply. And a value already in place writes nothing, and neither resets the config nor requests a restart, which is what keeps the sync sweep's unchanged re-pushes from recycling the fleet every tick; a real change resets Config and asks Restart_Planner to recycle the topologies the Field's restart class names. The reply to set is the same seven-key snapshot get returns, so a caller that writes sse_max_streams or remote_segment_size never sees its own value come back — that reply is not evidence the write was ignored.

status get is a cheap probe. The snapshot is { status, runtime_version, num_partitions, topologies, cache_available, timestamp }. Its num_partitions comes from Bootstrap::global_num_partitions(), the count clamped into [1, Spawn_Coordinator::MAX_PARTITIONS] that the fleet spawns, while dump_graph's same-named field is the raw config value cast to int — a misconfigured value makes the two disagree, and only this one names what runs. cache_available asks whether a backend is SELECTABLE: a non-empty memcached server list reached Core::$memd, or APCu is loaded and enabled. Adding a server connects to nothing, so a configured but unreachable memcached still reports true; the add/read/delete round trip that catches it is /health/runtime and the cache-backend row of wp nodes doctor.

layouts backs one canvas. The Topology Console is its only caller, one layout per topology name. It seeds useCanvasLayout's position map from layouts get on load, for an edit-mode canvas or a worker scope whose label matches the topology, and writes back only when a person clicks Save Layout — dragging autosaves nothing. A layout's positions is a node-id map or the string "auto", which the canvas treats as a map holding a fresh autolayout of the graph it draws. "auto" is never stored: get answers it where the file is missing or unreadable or its map places no node, and save given it, or a map placing no node, removes the file, as Save Layout does while every card still sits where the canvas laid it out. save refuses any other value. Every other canvas in the substrate, the debug overlay's Inspector and each station tab graph included, persists positions to localStorage through the same hook and never reaches the server, so a dashboard wanting layout to survive a change of browser wires these two commands up itself.

compositions stores block layouts. A consumer declares the blocks and the contexts it composes on the newspack_nodes/compositions filter, and a layout is serialized block markup holding only those blocks' delimiters and their JSON attributes. Each context keeps one option, newspack_nodes_composition:<context>, not autoloaded, holding { markup, actor, ts } or, after restore, the tombstone { removed: true, actor, ts }. Every command answers Composition_Store::read()'s shape, { context, source, blocks, saved_ts, actor }: source is saved or default, blocks a tree of { name, attributes, inner } with each block's attributes laid over their declared defaults, and saved_ts and actor say when the record was written and by whom, null for a context with no record. Composition::validate() runs on every read, the default included, and on every save, so a stored layout a later version no longer reads is refused rather than replaced by the default. It refuses, naming the block and the field, markup past MAX_MARKUP_BYTES (65536), a block the context does not admit (its own blocks, the shared blocks it admits and the core blocks every context admits, as Composition::blocks_for( $context ) answers them), a block deeper than three, a list item excepted, since it is part of its list, HTML anywhere but a core block's own element, a block holding blocks that is not a container, a container anywhere but the top, a core/column or a core/group, a core/column outside core/columns, a core/list-item outside core/list, anything else in either, a second multiple: false block, an undeclared attribute or one its schema refuses, more blocks than the context's max_blocks, and anything the context's check refuses. The core blocks are core/heading (level 1 to 6), core/paragraph, core/list (ordered, reversed, start and type, a numbering style), core/list-item, core/group (tagName and layout, the keys core writes), core/columns (isStackedOnMobile, verticalAlignment and layout), core/column (verticalAlignment, width, a CSS length or percentage, and layout), core/separator (tagName, hr or div) and core/spacer (height and width, CSS lengths, and style.layout's flexSize and selfStretch); core/paragraph keeps direction. The text of a heading, a paragraph or a list item answers as its content attribute, filtered by wp_kses() to strong, em, b, i, code, br and a with an href, a target and a rel on every read, and refused whole when it holds a comment, which the filter could rebuild into a block delimiter; content in a block's delimiter is refused. The namespace core belongs to WordPress, and a declaration under it is refused. save stores the whole tree, every attribute written, with no whitespace between delimiters, a text block's text in its text element and every other core block with no element, the shape the editor writes, holding the escaped result to the same cap and refusing a tree that does not read back as itself; each attribute and each declared default takes its schema's type through rest_sanitize_value_from_schema(); and actor is the caller's login, else Capabilities::$session_label. Composition_Store::seed() writes only a context that has no record, after every rule but check, so neither a saved layout nor a restore is ever reset. A refusal can name a key the reader may not see, so read() throws it as worded only to a caller who Core::speaks_for_operator() — a logged-in manage user, or a worker's command under a manage session — and throws this layout could not be read to everyone else, the compositions get command included. Composition_Store::page_data( $contexts ) answers each context's read(), or { context, error } for one refused, error the refusal as Core::caller_message() words it for this caller, and enqueues core's wp-block-library styles, which the core blocks wear. Composition::editor( $context, $handle ) mounts the editor for a tune caller alone: it runs Block_Editor_Bootstrap::load_canvas( $context ) and Block_Editor_Bootstrap::load_content_blocks(), which registers the core blocks the canvas offers, enqueues the consumer's registered bundle $handle and localizes Composition::schemas( $context ) on it as NewspackNodesComposition; for any other caller it does none of it and answers false. Composition::enqueue_editor( $pages, $handle, $build_dir, $build_url ) loads the editor of the page $_GET['edit'] names, $pages mapping each page slug to its context. A slug $pages does not list, or a build dir without layout-editor.js, answers false and loads nothing; otherwise it registers $handle from layout-editor.js with the dependencies and version of layout-editor.asset.php, calls editor(), and when that loads, enqueues layout-editor.css where the build holds one. It answers what editor() answered.

The browser half lives in @newspack-nodes/shared/composition/, split so a reader's page loads no editor script. The reader half imports neither @wordpress/block-editor nor @wordpress/blocks: <CompositionView blocks views renderError /> renders a tree by each block's registered View, every node inside its own error boundary, a container's View handed its children; useComposition( group, context, onWritten ) reads the context fresh and answers { answer, error, loading, save( markup ), restore(), refusal, writing }; and <CompositionEditSlot context … /> renders whatever editor the context registered through registerCompositionEditor( context, Component ), a registry on the __newspackNodesCompositionEditors window key, and nothing before. The editor half, which a consumer builds as the second entry Composition::editor() enqueues, is registerCompositionBlocks( schemas, views ), registering each localized schema as a dynamic block whose edit is CompositionBlockEdit, which hands the block's Controls its attribute schemas as schema, and <CompositionEditor group context heading schemas preview onDone onDirty? />, the canvas over a fresh get, which saves serializeLayout( blocks, schemas ), each block's declared attributes alone and a core block's text in the tag its schema's text names, and calls onDone( answer, verb ): save or restore after a write, null after Cancel or Back. onDirty( dirty ) is told whether the layout holds a change not yet saved, as that turns, so a page asks before leaving only once there is one. The page shows the "Layout saved." snackbar, because the editor is gone by then. Given patterns, the list schemas() localizes, and at least one in it, the editor's header offers Start from: a chosen pattern's blocks, built from the tree the server validated, replace the canvas unsaved, after the discard confirmation when a change is pending. The layout counts as changed only while serializeLayout() writes it otherwise than the layout read, an attribute holding its declared default left out because the server lays it back, so a canvas that writes back what the layout held leaves it clean.

The substrate declares six shared blocks, through Composition::declare_substrate_blocks() at priority 5: newspack-nodes/stats, time-chart, table, log-stream, sparkline, and section, a container. Each of the first five reads the source its source attribute names, and a consumer admits whichever it wants on a context and spreads SUBSTRATE_VIEWS, from @newspack-nodes/shared/blocks, into the views it hands the frame. The frame renders the core blocks itself, in core's elements and classes, so a consumer declares no view for them.

The page frame in @newspack-nodes/shared/composition/page carries both halves. Its reader half, which page/index.js exports, includes providedViews( views ), each View wrapped to provide its editing prop as the Editing context nested write controls read, which ComposedLayout applies to the views it is given. Its editor half is page/mountComposedEditor, imported by path: mountComposedEditor( localized, views, { heading, wrap?, preview? } ) takes the NewspackNodesComposition object, registers its blocks against the provided views, leaving the core blocks to core's own registration and confining the canvas to what a layout keeps (confineCanvas(): core's blocks lose the supports and styles whose attributes a save drops, and every format outside schemas()'s inline elements is unregistered, while CompositionEditor hands the provider CANVAS_FEATURES, every colour, typography, spacing, border and dimension setting off, and refuses an indented list item), throwing on a localized block that has no view, and registers an editor that renders CompositionEditor inside wrap( children, slotProps ), sending for the group of the ComposedPage it reads there, with <preview { ...slotProps } /> beside the buttons.

Every Command_Interpreter_Node separately exposes dump_metadata for the per-node canvas snapshot the topology console renders, keyed by node name:

{ class, counter, sink, target, targets, debug_state, arguments, lgst_msg,
  bytes_read, bytes_written, accepts_fill, has_target, has_config }

target is the ROUTING value, Node::target() verbatim, a scalar unless the node fans out; targets is the DISPLAY union Node::display_targets() returns, always a list, the routing target plus any destination the node declares through extra_targets() (ADR-19). accepts_fill and has_target come from the node's node_schema() and tell the canvas which ports to draw. has_target: false is also enforced: Node::refuse_target_on() refuses a target set on such a class, through target() or connect_node, a Vault_Group refuses one its child class would refuse, and Topology_Analyzer refuses a topology carrying one. A node another node published adds owner, the publisher's name, and the console offers no delete or rename for it, because remove_node and move_node refuse it. A node with registered listeners adds registrations, and a node's own dump_metadata() may add further keys — never clobbering a fixed one.

A full snapshot (no node named) leaves out every Node::owned() node — one with a patron — before building its schema, then every schema flagged hidden; the browser producer dumpMetadataPayload() applies the same two rules, a browser node having a patron and no publisher. An owner's row lists its owned siblings under sidecars, [ { name, class } ] sorted by code unit, from Node::sidecars(): every published sibling with a patron but its :config interpreter and any schema-hidden one, so a Crawler lists its {name}:curl and its {name}:seen Table, and a Remote_Source its per-stream Remote_Consumer readers and its channel. The key is absent from a row owning none. A published sibling with no patron, such as a Vault_Group's child, draws on the canvas instead. A patron keeps a node's :config, so a sidecar's commands reach it there as a canvas node's do. Naming a node, dump_metadata <name>, answers it unless its schema is hidden, owned or not, carrying owner as above. dump_metadata -o <name>... answers each node named the same way, then the row of every sidecar it lists, so a client asks for an owner's sidecars by the owner's name however many it owns: that is how the console polls every grid of live cards an owner's "…" opens, while it is open, and the sidecar its Inspector shows, Time Travel and Triage included, in one command. A full snapshot also carries a _header entry holding profiling and, when the command supplied one, pwd.

registrations comes from Node::registered_listeners(), which walks the registration table and keeps only the entries whose callback is null — the Node-NAME listeners. A closure listener has no name to draw an edge to, so it is skipped, and an event left with no name listeners drops out of the map rather than reporting empty. Timer_Node, which registers itself by name on the router's TIMER channel, therefore shows a registration edge, while Remote_Broker_Node, which registers a closure on the fleet's RELOAD, shows none: a missing edge is not evidence that nothing is listening.

A composite node reports its PATRONS' I/O, not its own. Once a Remote_Source_Node's patrons exist, counter is SSE_In's frame count, bytes_read and lgst_msg come from SSE_In, bytes_written from HTTP_Out, and the link's own relay increments are masked — so a rate or throughput figure read off a Remote_Source row measures the wire, not the node. Before the patrons are built, on a fresh drop or a spoke with no Vault entry, both byte counts read 0 and counter falls back to the node's own tally.

Address dump_graph to workers for the dashboard shape; address dump_metadata with empty TO for the canvas shape.

The root (empty-TO) base Command_Interpreter_Node's own vocabulary, the Shell builtins that precede it and the secure ladder are in the command chart above. help <NodeType> renders a class's node_schema() / nodeSchema() identically in PHP and browser-local JS. Addressing a command with empty TO dispatches against the root table; a non-empty TO routes to the named CI.

node_schema() shape. A CI's node_schema() returns a Service-category schema: { category, description, arguments, commands }, where each commands entry is { name, description, capability, args, handler } plus the optional console flags multiple, hidden and action. A command with none is configuration the editor writes into a .tsl and also a command a live node takes; multiple makes it one row per call, as every add_* is; action keeps it out of the editor and in the live panel and the REPL, as every remove_* is; hidden drops its button from the Inspector, though the REPL and the Inspector's own controls still send it. Each arg is { name, type, required } plus an optional default (for example, workers restart's partition, defaulting to -1) and an optional secret. A node schema may also carry requests, registrations, accepts_fill, has_target, owns and hidden; owns names the siblings a class builds that the analyzer must know of, suffix to class, such as a Crawler's seen Table, and only a class on the palette may declare it. A REST reply never carries it. hidden keeps a class off every console surface: the palette, the canvas, a sidecars list and the class catalog. A requests entry is { name, description, reply_shape } plus an optional args, in the shape a command's take, and an optional handler( static $node ): array; classes dump carries every field but the handler, and nothing undeclared. It contributes no dispatch entry: the addressed node answers a TM_REQUEST itself, by calling Schema_Reflection::answer_request( $message ) first in its fill() and returning when that answers true. The trait takes VALUE's first word, upper-cased, as the verb and runs the matching entry's handler for the reply's data, answering TM_STRUCT | TM_RESPONSE with VALUE { verb, data }. Any other request, a handlerless entry included, is refused with TM_ERROR and VALUE "unknown request verb: <VERB>\n", the error plane the interpreter refuses a command on. Either reply goes from the node TO the request's FROM, with ID and KEY echoed, and a request reaching a node with no sink throws fill requires a wired sink. reply_shape documents data.

An arg's declaration is enforced. Command_Interpreter_Node::dispatch() binds a command's tokens against its args before the handler runs (ADR-25): a missing required arg, an unknown --option, a surplus positional, a name given twice or both ways, and a token not of its declared int or float type each answer the command's TM_ERROR, and an absent arg takes its default. One arg may declare variadic, collecting the positional tail, or every --name= repeat, as a list of typed members. A bool takes 1, true, yes, on, 0, false, no or off and refuses any other word, and a blank for a required arg is missing. A Node constructor's arguments() is bound the same way by parse_schema_args() (ADR-11), except that a constructor's variadic argument must be last and is never bound: the node reads the tail itself.

An arg declaring 'secret' => true is a credential. It is named, never positional — the binder refuses <name> must be named: write --<name>=<value>, echoing nothing, because the browser masks only --name= tokens in its history and transcript — and the command line Command_Interpreter_Node::$around_dispatch hands a wrapper renders it --<name>=<redacted>; every other token reaches the wrapper verbatim. Core::$around_span is the same kind of slot for substrate work outside a command: Core::span( $label, $work, $describe ) runs $work through it, handing it the label, the work and a $describe that turns the work's result into the span's closing detail, and runs $work bare when it is null. A Table's purge pass (table purge) and WAL checkpoint (table checkpoint) go through it. An assigner composes with what is already there, as with $around_dispatch, and the slot survives Core::reset(). vault add and vault update declare it on password, the only secret arguments in the system: the Vault is the one place a credential enters, and Vault::url_carries_credentials() refuses a url carrying userinfo — judged after esc_url_raw(), and refusing one that will not then parse — so none rides in beside it.

What help renders of that is thinner. The COMMANDS and REQUESTS tables Node_Schema_Help::render() builds carry name and description alone: a command's capability, its declared args and the multiple / hidden / action flags never appear, and a hidden command prints like any other. So help Workers_CI will not tell an operator what restart takes or which of the three roles it needs. The live-mode Inspector's command modals and the Service-CI tables above are where arguments and capabilities surface; help is a summary, not the command reference.

Classes_CI's dump command inlines the serializable half of every concrete Node class's schema for the topology-editor palette and the live-mode Inspector: each class declaring a category and no hidden flag. A Hidden-category class, such as a broker's Remote_Consumer, is listed for the Inspector, which reads a sidecar's arguments and commands here, and the palette leaves it out. It returns { classes[], formatters[] }, each class carrying shell_name, fqcn, category, description, arguments, commands, requests, registrations, accepts_fill, has_target, is_interpreter and fans_out. The non-serializable handler — a command's and a request's — and the server-enforced capability are stripped on the way out. formatters[] is Formatters::list_names() sorted, and it is the option list the console renders for any node_schema() argument declared type: formatter_name — today only the one with_index takes. The substrate registers no formatters of its own, so on an install carrying no consumer plugin that calls Formatters::register() the array is empty, the console falls back to a free-text input, and a with_index command raises unknown formatter: <name> for whatever is typed into it. Discovery reads the composer classmap (ADR-10), so a class added or renamed without composer dump-autoload -o is absent from the palette.

Every command reads its bound arguments by name. A producer sends the flat arguments token array — each arg by position in declared order or as --name=value — and the handler receives ( Command_Interpreter_Node $interpreter, array $args, array $envelope = [] ), where $args is array<string,mixed>, one key per declared arg: the typed value, null for an absent optional, or a list of typed members for a variadic one. So command_node topologies get Home and command_node topologies get --name=Home reach cmd_get() alike, as [ 'name' => 'Home' ]. raw-logs dump_log answers { log_id, segments: [ { id, size } ], segment_count, total_size } for the one partition dir or sources:<name> registry source it inspects: a segmented source answers its segments and their summed total_size, and a file source one segment, { id: <inode>, size: <bytes> }, the slot its stream's ID breadcrumbs carry, or none while the file is absent or unreadable. An unknown or empty log is refused with an error. raw-logs read_message binds the log key, then the position — the single-step grammar <segment>:<offset>[:<length>], the segment-less :<offset> a file source's stream states before it has seen the file's inode, or one of start, recent and end, never the positions JSON the SSE route takes; the two vocabularies share those three words and nothing else. A segment-less position reads at that offset in the file the path holds now, and a partition dir refuses it as a malformed position. It reads a partition dir or a sources:<name> registry source, and an unknown or empty log is refused with an error. See Log Sources for the read model and the struct it answers. raw-logs read_block binds <log> <position> [--multi_writer=<bool>]: the log and position as read_message takes them, and a bool defaulting to false that holds a segment boundary until Consumer_Node::SEAL_GRACE_SECONDS have passed since the later of the older segment's mtime plus one second and the newer segment's first record TIMESTAMP, and while the newer segment is empty; a segmented Log holds bare values with no stamp, so it seals on the mtime alone. It does not answer with a command reply: the reader's raw block goes to the asker's FROM as two messages, its header echoing the position as ASKED <position> and its TM_BATCH sent even when empty, so the asker settles on it. The ownership-fenced workers heartbeat binds exactly [ slot, owner ], both declared int and read through Core::canonical_decimal(), from the current SSE connected handshake, and the server — never the client — owns the lease TTL. topologies save and layouts save bind [ name, body ], where the whole TSL body or positions JSON, newlines included, rides as one token, with no rest-of-line splitting to guess at. workers restart binds <type>… [--partition=<n>]: types is variadic, so a partition is reachable by name alone, and an absent partition restarts every one. A command whose schema carries no args key — the base interpreter's own vocabulary — receives its raw token list. The $envelope is the full 7-field positional Message; both save commands use it to enforce a 1 MiB body cap via Message::packed_size( $envelope ).

KEY='completion' mode. A help or ls command carrying KEY='completion' returns a bare newline-separated candidate list — sorted verb names, or bare node names across the whole registry — instead of the tabulated output. The KEY stands in for Tachikoma's TM_COMPLETION type flag, which this runtime does not carry, and REPL tab-completion is built on it. See architecture-guide.md → REPL.

Per-command args, return shapes and error semantics are declared on each CI's node_schema() under includes/rest/class-{classes,layouts,topologies,raw-logs,workers,vault,aggregator,settings,status,sessions}-ci-node.php; the palette and the Inspector consume the same schema. Auth gating is uniform: the endpoint requires a signed-in user AND a valid command signature, and each command's declared role decides the rest. A command may declare a WordPress capability in place of a role, 'capability' => 'wp:edit_posts': the signed-in user's own current_user_can( 'edit_posts' ) decides it, neither the session's ceiling nor the allowed_users list narrowing it. It runs only under an unlabelled session — a dashboard's none, or a role holder's — and is refused permission denied: edit_posts refuses a labelled session under a labelled one, read and write alike, permission denied: edit_posts requires a verified command under the ceiling a refused verification leaves, and permission denied: edit_posts requires a signed-in user in a worker, so such a command belongs on a CI the request graph mounts. No MCP door lists one (ADR-26). A refusal THROWS — Command_Interpreter_Node::interpret() wraps it as TM_COMMAND|TM_ERROR.

Error words. The reply's text is Core::caller_message( $e, <command> ). A deliberate refusal, an \Exception, reads as thrown to every caller. An engine \Error (a TypeError, ValueError, ArgumentCountError, DivisionByZeroError, …) or a driver's exception (PDOException, mysqli_sql_exception), anywhere down the getPrevious() chain or among a Failures' members, can name files, lines and database paths, so its own words reach only a caller who may run a MANAGE command (Capabilities::command_holds()): a logged-in user holding manage under the session scope in force, or a worker's command whose session allows it. Every other caller, a manage user under a read or tune session and an anonymous request outside a worker included, reads <command>: internal error; the reason is in the server's log, and the class, message, file and line of each link go to stderr — a worker's log, or PHP's error log in a request — once per command while print_less_often()'s window holds. A Worker_Should_Stop and a TM_NOREPLY command's failure re-throw instead. The same rule answers an MCP tool error, a Log_Sources unavailable row and the unreadable rows of workers and aggregator summary. raw-logs read_message and read_block refuse a malformed position that way too, and read_message answers a position holding no record as a result whose message is null (see Log Sources).

A Table's stats command (read, no arguments) answers one map per counted operation, GET to CHECKPOINT, each { calls, asked, answered, bytes, total_ms, max_ms, errors }; a Row_Table's carry no bytes, because its verbs encode no keyed value. errors counts the requests the Table answered with a TM_ERROR: a refused request, or a read its backend failed. A write that lands fewer keys than it asked adds no error; asked less answered shows it. dump_node and dump_metadata carry the same map as verb_stats, and reset_stats (manage) answers it and zeroes it. The calls since the last probe sweep carry across the reset, so the next tablestats.p0 record still counts them.

Test mode

HTTP_In_Node::set_test_mode( true ) makes dispatch() return instead of exit(), so PHPUnit can capture stdout through ob_start().

SSE Stream

GET   /wp-json/newspack-nodes/v1/messages/stream

Server-sent-events drain endpoint backed by SSE_Out_Node, which is both the _sse egress Node and the REST controller, mirroring HTTP_In_Node's double-duty pattern. One endpoint covers every subscription a dashboard needs: log partitions and worker IPC partitions both surface as Consumer_Node instances drained in the same loop. A log or source reader sends raw blocks, each going out as two msg events; a worker IPC reader sends each reply, which _output must see to gate it by session, as a msg event carrying the packed Message.

A stream's life in order: the slot acquire before any header, the headers, the retry event, the graph build, the connected handshake with its 4096-byte padding flush, then the five checks of every drain tick, the finally that releases the slot, the four Closure seams and when each is called, the pool's four numbers, and the client heartbeat that alone keeps a lease alive.

Permission: the fleet gate, then the READ role. No nonce — that would break the cross-server SSE pull, which is the aggregator's whole job, and it is why workers heartbeat (the slot keepalive) is read too: MANAGE there would expire every read-only stream after one slot TTL. A session the stream presents is checked after that, before any slot is taken: it must name a live command session minted by the user the request authenticated as, or the stream answers:

{ "code": "sse_session_refused", "message": "The command session is not live, or belongs to another user.", "data": { "status": 401 } }

The subscriptions are resolved first, before the session and the slot. A prefix that is not carried by any stamp (logs:x, secrets:x), a name the stream's guard refuses (offsets:Kea-1) and a sources:<name> the registry lacks refuse the stream with the reason; a registry that will not read is a server error:

{ "code": "sse_subscription_invalid", "message": "invalid subscription: logs/kea-7713.p3", "data": { "status": 400 } }

Query parameters

Field Type Required Description
subscribe string yes CSV of subscription names, described below. Blank entries between commas are dropped.
session string no The handle of the command session the client signs with. Its attached commands head their FROM with _sse:{session}, and this stream passes only the replies headed with it; a stream presenting none passes none, which is what a server-to-server pull does.
stream string no The client's own id for this stream, 1-64 characters of [A-Za-z0-9_.:-]; the browser sends its SseInNode's name. With session it keys the slot lease, so a reconnect takes its own live lease over instead of claiming a second slot, while two streams on one page keep two leases. A malformed id answers 400 sse_stream_invalid. Because the session is resolved first, a request carrying a deliberately malformed id is a probe: it answers 401 sse_session_refused for a dead session and 400 for a live one, and opens nothing.
positions string no Optional resume positions: a JSON object keyed by the STAMP each subscription resolves to. A value is either an exact {segment, offset} object or a seek sentinel — 0 start, -1 end (live tail), -2 recent — Tachikoma's vocabulary (Consumer.pm: "valid offsets: start (0), recent (-2), end (-1)"), mirrored as Consumer_Node::SEEK_START / SEEK_END / SEEK_RECENT. The words start, recent and end are accepted aliases. Stating the seek as a number is what makes {segment: 0, offset: 0} mean the START of the log rather than an absent value: a Remote_Source states a position for every live reader it asks for, so a reader resuming a 0:0 checkpoint replays its backlog instead of being tail-seeked past it. -1 / end seeks the newest segment's current size; -2 / recent seeks byte 0 of the SECOND-newest segment, or of the only segment when there is just one, so it can replay up to a whole segment. Malformed JSON, or a value decoding to anything but an array, is treated as omitted. A per-key value is handled the opposite way: SSE_Out_Node::position_arg() reads a word of Log_Position::WORDS as its number and any other string through Log_Position::parse(), so "<segment>:<offset>" seeks that place, and any string that is neither seeks to start, so ONE typo'd position word replays that subscription's whole log rather than tail-seeking it. Omitting the parameter tail-seeks every subscription, which is what a browser dashboard does on a first connect. A value of skip (SSE_Out_Node::SKIP) keeps that stamp out of the stream: a reader never opens for it, and under a glob the stamp stays closed through every rescan while the glob's other matches open.
multi_writer boolean no Read the subscribed logs with the multi-writer seal-grace (Consumer_Node::SEAL_GRACE_SECONDS). Set it for a log every request process on this server appends to — the firehose — where a peer can keep writing to segment N for up to Partition_Node::DRIFT_RESCAN_INTERVAL_SECONDS after N+1 appears; without it the reader advances off N on sight and orphans that straggler, typically a request's terminal process (complete). The CLIENT asserts it, because nothing on disk records which logs are shared. It applies to log Consumers only — a worker IPC attach has one writer and never takes the grace. Default off: a single-writer log seals N the instant it creates N+1, so the grace would be pure added latency. Remote_Source_Node sends it through its set_multi_writer config command; browser dashboards leave it unset.

Subscription grammar

How one subscribe name resolves: the group split against Log_Discovery::GROUPS, the character-class check that leaves * as the only wildcard, the worker-IPC path and the partition-dir glob with the multi-writer seal grace; the connected envelope drawn as seven KEY VALUE slots with what SSE_In_Node rejects; and how a client advances each cursor from the ID breadcrumb of every record and sends it back as positions, with the three seek sentinels.

Response

A standard SSE stream, Content-Type: text/event-stream, with every buffering layer between PHP and the browser disabled and each tick's output chased with the FLUSH_SIZE (4096) padding comment the lifecycle diagram shows.

Six events go out, in this order of first appearance. Every one but msg is a TM_INFO frame stamped FROM=_stream, so the table names only what tells them apart:

Event Message Meaning
retry KEY=retry, VALUE the sse_retry_ms config value (default 5000), or 0 at a lifetime close The reopen schedule, sent first because an idle close relies on it, and again as 0 just before the lifetime close of a stream that delivered records. An EVENT rather than the protocol retry: field, since the client owns reconnect and needs the interval as data it can read. The VALUE is a canonical decimal; the last one a connection carries sets the delay after its close, and 0 means at once. Set sse_retry_ms to 0 to advertise nothing at open: an idle close then leaves the client on its own backoff.
connected KEY=connected, VALUE the flat envelope below The session handshake; see below.
msg the packed Message the egress received A raw block or its header, which the TM_BATCH bit marks and the client splits into records (see Raw blocks), or one reply from a worker's IPC channel. These count as data, which is what defers the idle close.
heartbeat KEY=heartbeat, VALUE the tick's wall seconds Liveness every HEARTBEAT_MS = 2000ms. Deliberately not data: a heartbeat never defers the idle close. A hub's reader takes it as word that its stream has nothing more to send, which keeps the backlog the last header gave from going stale.
unparseable_lines KEY=unparseable_lines, VALUE COUNT <n> COUNTS <stamp>=<n>,… CURSORS <pairs> Lines a worker IPC reader skipped since the last such frame because they would not unpack, sent on the tick that skipped them; a log or source reader does not read lines, and its client counts what it cannot split. The IPC reader keeps no cursor to replay a torn line from, so each is skipped rather than ending the stream on it; CURSORS is the connected envelope's token, restating every reader's resume point past the skip so a reopen does not read the line again. COUNTS lists only the stamps that skipped this tick, each with its own count, and leaves out a stamp holding a space or a comma, as CURSORS does. Not data: it never defers the idle close. The browser's SseInNode adds COUNT to a running total it publishes as UNPARSEABLE_LINES, and re-seats each subscription's resume point from CURSORS; a hub pulls no IPC stream, so its SSE_In_Node ignores the event.
disconnect KEY=slot_lease_lost, VALUE SSE slot lease lost; or KEY=superseded, VALUE SSE stream superseded by its reconnect The terminal frame for a stream whose lease is gone. slot_lease_lost is a failure and writes a diagnostic line; superseded means the stream's own reconnect took the lease over, and writes nothing.

The stream closes itself after sse_idle_timeout seconds (default 5) with no msg event, and after sse_max_lifetime seconds open (default 30) however busy it is. Either is disabled at 0. The lifetime runs on the wall clock, from Core::$now against the instant the stream opened, so a session never holds one PHP-FPM child for its whole life: the client resumes from its positions and takes its own slot over. After an idle close it waits out the retry schedule, which is what gives the child back to a page that is not being read. A lifetime close on a stream that delivered records sends retry 0 first, so a busy stream reopens at once; one that delivered nothing keeps the gap. Neither close sends a terminal event; a disconnect frame always means failure, and SSE_In_Node is the reference implementation for consuming one.

The connected envelope

SESSION <handle> SLOT <slot> OWNER <owner> SUBSCRIPTIONS <csv> INTERVAL <ms> CURSORS <csv> DISTANCES <csv>

SESSION echoes the session the stream presented and is omitted when it presented none. DISTANCES rides with CURSORS, one stamp=<bytes> pair for each, the bytes that reader has yet to send: a raw block's DISTANCE stated before any block, because a stream starting caught up sends none, and a hub reader would otherwise read its backlog as unknown until the log grew. A file source whose named generation has rotated counts the whole current file, which its first read rewinds to.

A flat space-separated KEY VALUE string rather than JSON, because a TM_INFO VALUE is never an array; the slots, their validation by SSE_In_Node through Core::canonical_decimal(), and the resume loop they seed are in the subscription diagram above.

Raw blocks

A log or source reader does not parse what it reads. Each read takes up to 1 MiB (Log_Sources::BLOCK_BYTES), or 4 MiB for raw-logs read_block (Log_Sources::FETCH_BYTES), of one segment from wherever the reader stands, mid-line included, through Consumer_Node::raw_block(), and Consumer_Node::block_messages() sends it as two messages:

Message TYPE ID VALUE
header TM_INFO|TM_BATCH empty OFFSET <segment>:<offset> DISTANCE <bytes>[ RESET 1]
batch TM_BATCH <segment>:<offset> the raw bytes

OFFSET, which the batch's ID repeats, is where the bytes begin; a seek word, a rolled segment or a rewind can move it from where the reader was asked to stand. DISTANCE is what the log holds from there, the block's own bytes included, so 0 is the end. RESET 1 says retention deleted the segment the reader stood on, and the block begins at offset 0 of the oldest segment left. A stream sends a block only when it holds bytes; raw-logs read_block sends one even empty and adds ASKED <position> to the header.

The client splits the bytes at their newlines. A record's breadcrumb is the block's segment, its offset plus the line's byte position in the block, and the line's byte length with its newline. A packed line is unpacked and stamped FROM the stream; a source's line becomes a TM_BYTESTREAM carrying it. What follows the last newline is carried and joined to the next block that continues it, so a record longer than a block arrives whole; one the next block does not continue was torn, as is a line that will not unpack. A batch that was not placed by any header is dropped. A hub reader dead-letters a torn line at its place and moves past it; the browser's SseInNode, which has no quarantine, counts it under UNPARSEABLE_LINES. A hub reader's backlog is the latest DISTANCE less how far its cursor has drained past OFFSET.

Slot gating

The application controls concurrency through four optional Closure seams on SSE_Out_Node, installed by SSE_Slot_Pool::wire() from Bootstrap::register_rest_routes():

Seam Signature Called
$acquire_slot function ( int $partition, ?array $session ): array{slot:int,owner:positive-int}|false Once per stream, before any header, so false can still answer 429 too_many_connections. $session is {key, ttl} — <handle>:<stream> and the seconds the session has left — or null. The shipped pool hands a live lease recorded under the key to the new stream and rotates the owner, so the old process's next check fails; it never takes a recorded slot past max_streams or inside a browser's reserved tail, and records the index for ttl seconds at most.
$check_slot function ( array $lease, int $partition ): bool Every drain tick; false takes the disconnect close. It only READS — refreshing the TTL belongs to the client heartbeat, and refreshing it here would let an unread stream hold its slot forever.
$release_slot function ( array $lease, int $partition, ?string $session ): void Once per stream, from the drain's finally or from a shutdown function, whichever runs first, so a close, a throw, a fatal or a client abort never leaves the slot held until its TTL expires. $session is the lease key the stream acquired under, whose takeover index the pool forgets with the lease.
$inspect_slot function ( array $lease, int $partition ): array<string,int|string> Only once a check has already failed, to name the backend and lease state in the diagnostic line. The healthy path never pays it.

The stream draws on one host-wide pool, sized by sse_max_streams (6), sse_max_slots (3), sse_reserved_slots (0) and sse_slot_ttl (60 seconds). sse-host-budget.md carries the arithmetic behind each number.

Log Sources

GET   /wp-json/newspack-nodes/v1/messages/stream?subscribe=sources:<name>

A Log_Sources registry entry streams from /messages/stream as the subscription sources:<name>, beside partitions on the same stream. The name is a fixed registry NAME, opened as a Tail_Node reader instead of a Consumer; a caller can never supply a path, so there is no traversal surface. On the wire a source is a partition: same msg events, retry and connected envelopes, heartbeat cadence, flush framing, idle close and slot pool, with each frame's FROM sources:<name>. Its blocks are a TM_BATCH like a partition's, and the sources: stamp says their lines are text: a client wraps each as the TM_BYTESTREAM a Tail emits, VALUE the line with its newline.

The Log_Sources registry: the three families merged in priority order with the tail mode each carries, the picker row Log_Sources::catalog() builds and the footprint dump_log answers, the two classes Log_Sources::open_tail() maps the mode token to, the one read model behind raw-logs read_message and read_block with the struct read_message answers, the malformed position it refuses and the empty read it answers with a null message.

The same registry backs raw-logs. list_logs lists every source as a sources:<name> row after the partition dirs, each row carrying available; dump_log sources:<name> sizes one, as { log_id, segments, segment_count, total_size }; read_message sources:<name> <position> single-steps it; and read_block sources:<name> <position> answers one raw block of up to 4 MiB. A source whose segments will not list takes an unavailable row carrying error beside every readable source, so one failure never blanks the picker. An active topology that will not read takes one too, labelled for the topology and keyed without a stamp, because it names no log to stream. Every reader of a stamp resolves it one way (ADR-29), so read_message, read_block and dump_log refuse what the stream refuses, with its message.

Permission: inherited from /messages/stream — the fleet gate, then the READ role, with no nonce. A dir carrying a .read-capability marker — a log whose topology declared set_read_capability <cap>, its write quarantine, and every reader's cursor and dead letters over it — also needs current_user_can( <cap> ) (ADR-39): dump_log, read_message, read_block and an exact subscription refuse such a log with permission denied: log "<stamp>" needs the <cap> capability, a glob leaves it out, and list_logs omits it. The marker stays after the declaration goes, and a marker that will not read refuses everyone.

Subscription parameters

Field Type Required Description
subscribe string yes CSV of subscriptions, any mix of partitions and sources:<name>. A sources: name the registry lacks is refused before the stream opens, as 400 sse_subscription_invalid carrying the teaching unknown log source error that lists the names that exist. A registry that will not read — an active topology with no matching .tsl, a broken include — is the server's fault: the stream logs it as a stream: <subscribe>: <cause> stderr line and answers 500 sse_subscription_unavailable with a message naming no internals. An \Error is not caught. No globs — registry sources are fixed for the life of a stream, and Tail's missing-file grace covers a source that appears, rotates or truncates mid-stream.
positions string no Optional resume positions, same vocabulary as the partition subscriptions: a JSON object keyed by the whole sources:<name> stamp, each value a {segment, offset} object or a seek sentinel (0 start, -1 end, -2 recent), with start, recent and end accepted as aliases. File_Tail_Node folds recent to the start, because one file has no previous segment to fall back to. The shape round-trips unchanged from the client's perspective: for a segmented source segment is the segment id; for a file-mode source the file's inode occupies the same slot, and the cursor self-validates against the live file and degrades to 0 on mismatch. That validation happens on the FIRST POLL, not at subscribe time — open_subscription() seeks before the file is open, so the pair is held as a candidate, and cursor_position() echoes the client's own unvalidated value straight back into the connected envelope's CURSORS. An unchanged echo is therefore not an acknowledgement that the server accepted it. A candidate counts toward compute_lag() only while it still names the live generation and sits within the file: honouring a stale pre-rotation position would declare the new generation caught up and let the stream idle-close on its first tick without ever delivering it. Omit to start at end (live tail).
multi_writer boolean no Inert for a source: the seal-grace belongs to a Consumer, and a source resolves to a Tail_Node.

The substrate as client

Two nodes call these endpoints on a remote server instead of serving them: a hub POSTs commands to its spokes through HTTP_Out_Node and pulls their streams back through SSE_In_Node. Both bound themselves, and an operator diagnosing a flapping spoke reads it against the numbers below.

A hub with many spokes builds many such nodes from one line: Vault_Group (class-vault-group-node.php) keeps one child — an HTTP_Out, a Remote_Source, or any other type — per server in a Vault group, running update_graph() on the fleet's RELOAD as a spoke is added to or dropped from the group. Each child is a normal client of the endpoints below, published as an owned sibling named <group>:<vault id> rather than hand-written per spoke.

HTTP_Out_Node — /command and /auth

The push side in four steps: fill() buffering under a one-shot timer with the three in-tree minters, fire() dropping an unaddressable batch or holding one while /auth runs, the JSONL POST with its 15-second timeout and 8 MiB reply cap, and on_transfer_done() branching on transport error, 401, 202, other non-200 and 200; beneath, the blocking probe_command() bounds and what an operator reads off a flapping spoke.

A batch lost past fire() has no retry and no dead letter (ADR-3); a minter finding Command_Auth::has_session() false calls ensure_session() rather than skip the push, because a skipped push deadlocks both sides: HTTP_Out handshakes only inside fire(), and only fill() arms that tick.

Constant Value Bounds
REQUEST_TIMEOUT 15 seconds One non-blocking POST, as CURLOPT_TIMEOUT.
MAX_REPLY_BYTES 8388608 (8 MiB) One spoke's reply body by default, capped because the write callback buffers it into the PHP heap. A broker raises its HTTP_Out's cap with set_reply_cap(), never below this.

The blocking path, probe_command() behind vault test and aggregator probe, is in the diagram: it returns the reply envelope's payload array and nothing else, so a caller never has to read a verdict out of a returned value.

SSE_In_Node — /messages/stream

The pull side as a state machine: CONNECTING, CONNECTED, DISCONNECTING, ERROR, DISCONNECTED and RECONNECTING with the event behind each edge and the backoff rail back to CONNECTING, the four bounds, what the node sends on every connect, who states the cursor, with SSE_In holding none: the live readers' positions, the broker's stream_request() that collects them into streams(), and the connected frame's CURSORS handed back to each reader by stamp, the eight keys of connection(), and the browser mirror's own watchdog, silence and backoff numbers with its five states.

A patron (Remote_Source_Node) drives maybe_connect() and check_stale(). The node keeps no cursor. The patron states what to pull through streams( $subscribe, $positions ) from its on_connecting seam, each time a request is about to go out: the subscriptions in order, and a position for each stamp, either { segment?, offset } or a seek sentinel. A position naming no segment is a file source whose generation is not yet known. A broker builds the request in Remote_Source_Node::stream_request(), asking each live Remote_Consumer_Node for its connect_position() and stating skip for a paused reader under a glob, so the position stated to the spoke changes only at a connect, though a live reader's cursor moves on every drain.

Constant Value Bounds
CONNECT_TIMEOUT 5 seconds The connect, as CURLOPT_CONNECTTIMEOUT. The transfer itself is untimed (CURLOPT_TIMEOUT 0), which is what check_stale() covers instead.
HEARTBEAT_TIMEOUT 45 seconds The silence check_stale() reads as a dead stream. SSE_Out_Node heartbeats every 2 seconds, so only a broken link reaches it.
INITIAL_BACKOFF / MAX_BACKOFF 1 / 30 seconds The reconnect delay, measured from the last connect attempt, doubling on each failure and reset to the floor by any received event. A clean 200 close on a stream that advertised a reopen delay — the retry event, or a plain server's retry: field — takes that delay instead, counted from the close, capped at MAX_BACKOFF and leaving the failure state untouched; a delay of 0 reopens on the next tick. A delay of 0 on a stream that delivered a msg and lived at least INITIAL_BACKOFF opens nothing inside the completion either: SSE_In calls the patron's on_reopen_due, and Remote_Source reopens from the timer scan that follows the completion, in the same drain iteration, instead of on its next 1 s tick.
MAX_BUFFER_SIZE 33554432 (32 MiB) Received bytes holding no newline.
MAX_EVENT_SIZE 33554432 (32 MiB) One event's accumulated data:.

The browser mirror, src/runtime/sse-in-node.js, opens the same endpoint through fetch(), parsing the events itself; its numbers, states and seekMap() precedence are in the diagram. That precedence is what keeps a chart asking to replay from the start of the log from being collapsed to a single live point by a stream the slot pool refused before its first frame, and a glob's dirs tail-seek on the first connect and resume from the positions they themselves reported thereafter.

Curl_Node — any http(s) url

Curl_Node calls no endpoint of ours: it fetches the url a TM_BYTESTREAM carries with a GET on the same shared multi, through the Curl_Transfer mechanism HTTP_Out_Node runs on. make_node Curl <name> [vault_group]. A TM_STRUCT carries a request instead, { url, method, body, headers, proxy, response_headers, connect_timeout, timeout, public_only }: method is GET, POST or DELETE, a POST sends its body, a string of at most the node's body cap ('' when absent), as Content-Type: application/json, and a GET or a DELETE sends none. The cap is DEFAULT_BODY_CAP (64 KiB) until set_body_cap( $bytes ), called in code on the node, raises it for that node to at most 1 MiB; a value outside that range throws \InvalidArgumentException. headers, when present, is a list of Name: value lines sent after that type and before the vault credential, a Content-Type among them replacing the JSON one. A line must carry a value, since libcurl reads an empty one as removing the header, and may not name one of Vault::FRAMING_HEADERS (Host, Cookie, Content-Length, Transfer-Encoding, Connection, Expect, Upgrade, TE, Keep-Alive, Proxy-Connection, Proxy-Authorization), in any case, nor Authorization or the header the credential rides in where the vault group lends the origin one. proxy, when present, is the <scheme>://<host>:<port> the fetch goes through, set as CURLOPT_PROXY with CURLOPT_NOPROXY empty, so a no_proxy variable cannot bypass it, its TLS verified as vault_verify_ssl says: its scheme socks5h, socks5, http or https, a trailing slash allowed, and nothing else, userinfo above all; beside a credential, the vault's or a header line of the request's own, the url must be https. response_headers, when present, names at most eight response headers to read; the success then answers a TM_STRUCT copy whose VALUE is { body, headers }, each named header the final response's head sent, a trailer never, under its lowercased name, its first value trimmed and cut to 256 bytes. timeout, when present, is the whole seconds, from 1 to MAX_REQUEST_TIMEOUT (120), the whole request may take, REQUEST_TIMEOUT (30) when absent. It is a deadline set when the request starts: each transfer gets the milliseconds that remain, read from the live clock as it starts, as CURLOPT_TIMEOUT_MS, so a public-only request's hops, and the lookup before each on a PHP without CURLOPT_PREREQFUNCTION, spend one budget. The deadline passing between transfers answers timed out <url>, and passing inside one answers libcurl's curl error 28 (Timeout was reached) <url>. connect_timeout, when present, is the whole seconds, from 1 to that timeout, the connection may take, set as CURLOPT_CONNECTTIMEOUT; timeout is checked first. public_only, when present, is a boolean; true marks a url a response chose, a discovered link or a Location, and holds the request to public addresses (below). A header name is 1–64 letters, digits and hyphens (Vault::HEADER_NAME) wherever one is read. Both forms share the vault group, vault_require_ssl, MAX_IN_FLIGHT and the answer. The completion emits a copy of the input message, FROM, ID and KEY intact and TO stamped from target, with its whole TYPE replaced: TM_BYTESTREAM carrying the body below status 400, and otherwise a TM_ERROR whose VALUE opens with a fixed prefix a consumer can match.

A fetch follows redirects, up to MAX_REDIRECTS. follow_redirects( false ) turns that off for one node, and is a method rather than a positional because the optional vault_group has no empty placeholder. With redirects off, a fetch sets CURLOPT_FOLLOWLOCATION off, and a 3xx naming a Location answers a TM_RESPONSE copy whose VALUE is that Location, absolute as libcurl reports it (CURLINFO_REDIRECT_URL).

A request naming public_only true reaches public addresses alone. Before any transfer starts, its host is refused when it is an IP literal outside the global ranges RFC 6890 marks (FILTER_FLAG_GLOBAL_RANGE: private, loopback, link-local, shared, reserved, and an IPv4-mapped form of any; besides the filter, the site-local fec0::/10 and the local-use NAT64 64:ff9b:1::/48, and a well-known NAT64 address in 64:ff9b::/96 judged by the IPv4 address it carries), a single label (intranet, localhost., 2130706433), or a name whose last label is numeric (127.1, 0x7f.0.0.1), which libcurl's resolver reads as an address; a request naming a proxy is refused, and the transfer sets an empty CURLOPT_PROXY, which turns off a proxy the worker's environment names (http_proxy, https_proxy, ALL_PROXY), because libcurl would report the proxy's address rather than the host's. The lookup and the pin below run last, after every other refusal, the busy and event-loop checks included. Where PHP defines CURLOPT_PREREQFUNCTION (8.4 and later), libcurl resolves the name itself, without blocking the worker, and once connected hands the address it reached to a check that aborts the transfer, before a byte of the request is sent, when that address is outside the global ranges; the check runs on every connection libcurl makes or reuses, so a name that rebinds cannot slip past it. Where PHP does not (8.2 and 8.3), the name is looked up once, through the Curl_Node::$resolve seam (A and AAAA records, dns_get_record(), when null), and refused before anything is sent when it resolves to nothing or to any address outside the global ranges; the transfer is then pinned to the first address checked (CURLOPT_RESOLVE +host:port:address, an IPv6 address in brackets, and a rooted name pinned under both its spellings, since libcurl matches an entry by the url's own), on a fresh connection, because libcurl reuses a connection by host and port rather than by pinned address; the + keeps the entry from outliving the transfer in the shared multi's DNS cache. So a name that rebinds between the check and the connection never reaches a private address. That lookup blocks the worker for as long as DNS takes. Curl_Node::$prereq_available is the seam choosing the path. Either way, once the transfer completes, the address it connected to (CURLINFO_PRIMARY_IP) is checked again as a second line, answering a TM_ERROR rather than the body, the headers or the Location. While the node follows redirects, libcurl follows none of a public-only request's: the node fetches each Location itself as the next hop, held to the same guard, its own lookup and pin included; as libcurl does, a hop off the first url's origin carries no vault credential, and a 301, 302 or 303 sends a POST on as a GET with no body. The answer is a copy of the first request's message. Past MAX_REDIRECTS hops it answers a TM_ERROR. Following none, a 3xx answers its Location as any 3xx does, and the sender marks the Location public_only when it fetches it. A request not naming public_only is not guarded, and libcurl follows its redirects.

TYPE VALUE When
TM_RESPONSE <absolute url> With redirects off, the status is 300-399 and names a Location.
VALUE When
HTTP <code> <url> The response's status is 400 or more, or, with redirects off, 300-399 with no Location.
curl error <n> (<curl_strerror>) <url> The transfer failed: a timeout, a refused connection, a redirect past the limit or a scheme outside it, or the body cap's short write. A request's deadline passing while a transfer runs answers curl error 28 (Timeout was reached) <url>, the libcurl text, unchanged; one passing between transfers answers timed out <url>.
invalid url <value> VALUE, or a request's url, is not an absolute http(s) url with a host.
redirect not followed: credential in <header> <url>, redirect not followed: credential in request headers <url>, redirect not followed: a DELETE <url> The node follows redirects, the response is a 3xx naming a Location, and the fetch followed none: the url's credential rides in a header other than Authorization, the request carries header lines of its own, or it is a DELETE. The value is never shown.
invalid method <method> <url> A request's method is not GET, POST or DELETE, or is absent.
body on a GET <url>, body on a DELETE <url> A GET or DELETE request carries a body, even an empty one.
invalid body <url> A POST request's body is not a string.
body over <cap> bytes <url> A POST request's body is longer than the node's body cap, DEFAULT_BODY_CAP (65536) until set_body_cap() raises it.
invalid headers <url> A request's headers is not a list, or one of its lines is not a string opening with a name of 1–64 letters, digits and hyphens and a colon.
header carries CR or LF <url> A request's header line holds a carriage return or a line feed; the line is left out.
refused header <name> <url> A request's header line names one of Vault::FRAMING_HEADERS; the value is left out.
refused header <name> beside a vault credential <url> A request's header line names Authorization, or the header the credential rides in, and an entry of the vault group sits on the url's origin.
empty header <name> <url> A request's header line carries no value but whitespace.
refused proxy beside a credential on a plain url <url> A request names a proxy and carries a credential, an entry of the vault group on the url's origin or header lines of its own, and the url is not https.
invalid proxy <url> A request's proxy is not a string naming one of the four schemes, a host and a port, and nothing more but a trailing slash; the proxy is left out.
invalid proxy carrying userinfo <url> A request's proxy holds an @; the proxy, which may hold a credential, is left out.
invalid response headers <url> A request's response_headers is not a list of at most eight names of 1–64 letters, digits and hyphens.
refused response header <name> <url> A request's response_headers names one of Vault::FRAMING_HEADERS, Set-Cookie or Authorization, in any case.
refused response header <name> beside a vault credential <url> A request's response_headers names the header the vault's credential for the url's origin rides in.
invalid timeout <url> A request's timeout is not an integer from 1 to MAX_REQUEST_TIMEOUT.
invalid connect timeout <url> A request's connect_timeout is not an integer from 1 to its timeout, REQUEST_TIMEOUT when it names none.
timed out <url> The request's deadline passed between transfers, so the transfer about to start never starts: a public-only request's next hop due after it, or, on a PHP without CURLOPT_PREREQFUNCTION, the lookup before a public-only transfer outlasting it. <url> is the url that would have been fetched. The deadline passing inside a transfer answers curl error 28 instead.
invalid public_only <url> A request's public_only is not a boolean.
refused private address <url> A public-only request's host is an IP literal outside the global ranges, or its transfer connected to such an address before sending, or, on a PHP without CURLOPT_PREREQFUNCTION, a name any of whose addresses is: nothing is sent. Or the address the completed transfer reports is one, and nothing of the response is answered.
refused single-label host <url> A public-only request's host is one label, a trailing dot ignored.
unresolved host <url> On a PHP without CURLOPT_PREREQFUNCTION, a public-only request's host is a name that does not resolve to any address; nothing is sent.
refused numeric host <url> A public-only request's host is not an IP literal and its last label is digits or 0x hex.
refused proxy on a public-only request <url> A public-only request names a proxy.
redirects past 5 <url> A public-only request on a node following redirects was redirected again after MAX_REDIRECTS hops; <url> is the last hop.
proxy carries CR or LF <url> A request's proxy holds a carriage return or a line feed; the proxy is left out.
invalid url carrying userinfo VALUE carries userinfo before its host, as Vault::url_carries_credentials() reads it; the url, which may hold a credential, is left out. PHP and libcurl split an authority holding two @ at different ones, so the origin read here need not be the host fetched.
vault entries <id>, <id>… share origin <origin> <url> Two or more entries of the vault group sit on the url's origin, so the fetch cannot choose between their credentials.
vault_require_ssl set but url is not https <url> The url is plaintext, an entry of the vault group sits on its origin, and vault_require_ssl is set.
no event loop <url> The process is not inside a drain loop, so a transfer could never complete.
busy: <N> requests in flight <url> MAX_IN_FLIGHT transfers are already running; nothing is queued.
curl_init failed <url> libcurl yielded no handle, so the transfer never started.

The output goes to target like any forwarded message, never back along FROM. Anything but a TM_BYTESTREAM or a TM_STRUCT is dropped with one rate-limited line. A transfer still in flight when the node is removed is lost, and one rate-limited line counts the discards (ADR-3).

Under a vault group each fetch looks up the group's entries through Vault::in_group() and keeps those whose Vault::url_of() has the url's origin, by Curl_Node::origin_of(): the same scheme, host and effective port. With no matching entry, the url is fetched with no credential; an entry with no url has no origin. With one, the fetch sends the header line its Vault::credential_header() answers, and under vault_require_ssl refuses a plaintext url and takes https alone, redirects included. With two or more, the fetch refuses, naming every one. The url is fetched as written; a vault entry's path plays no part. A credential starts on its own origin and stays there: the transfer sets CURLOPT_UNRESTRICTED_AUTH off, so libcurl (7.83.1 and later) drops an Authorization header when a redirect changes host, port or scheme. libcurl sends every other header past a redirect, so a fetch whose credential rides in another header, a request carrying header lines of its own, any of which may be a credential, and a DELETE, whose method libcurl would carry along, do not follow redirects: on a node following redirects its 3xx naming a Location answers the TM_ERROR redirect not followed, and on one with redirects off it answers the Location as any 3xx does. A node with redirects off sends nothing past the first response. Every fetch, vault group or not, verifies TLS as vault_verify_ssl says, through Vault::tls_opts().

Curl_Node::origin_of() and DEFAULT_PORTS are public: Crawler_Node reads a link's origin and default port through them, so a link is kept by the rule a fetch is held to. transfers_in_flight() is public too: the crawler sizes each refill by Curl's own count.

Constant Value Bounds
MAX_IN_FLIGHT 16 Transfers one node runs at once.
DEFAULT_BODY_CAP 65536 Bytes a POST request's body may carry until set_body_cap() raises the node's cap, to at most 1 MiB: room for the largest real body, a 65,536-byte item body JSON-encoded twice (about 460 KB). A node at that cap holds up to 16 bodies, each in PHP and in libcurl's copy, 32 MiB, beside its reply buffers.
REQUEST_TIMEOUT 30 seconds One request that names no timeout, redirects and hops included, as CURLOPT_TIMEOUT_MS.
MAX_REQUEST_TIMEOUT 120 seconds The longest timeout a request may name.
MAX_REDIRECTS 5 CURLOPT_MAXREDIRS, with CURLOPT_PROTOCOLS and CURLOPT_REDIR_PROTOCOLS both http and https alone, or https alone for a fetch carrying a credential while vault_require_ssl is set; and the hops the node follows itself for a public-only request.
MAX_REPLY_BYTES 8388608 (8 MiB) One response body, buffered into the PHP heap.

Crawler_Node — each seed's site, link by link

Crawler_Node crawls the site of each seed. It fetches each url through a Curl_Node it owns, which does not follow redirects, follows the same-origin links every page carries and the same-origin Location every redirect names, and sends each answer to target. make_node Crawler <name> <ttl> [vault_group] [delay_ms] [concurrency]: ttl is a whole number of seconds, at least 1, that a url counts as seen, so a url may be crawled again once it expires; vault_group goes to the Curl sibling. A seed may be on any origin, and each url is fetched with the credential of the group's entry on its own origin, if any, by Curl's rule. delay_ms, default 0, is the least number of milliseconds between two fetch starts, and 0 leaves the crawl unthrottled; any other delay is at least MIN_DELAY_MS (100). concurrency, default 1, is the most fetches in flight at once, from 1 to Curl_Node::MAX_IN_FLIGHT (16). make_node refuses a non-numeric or negative token for either, a delay_ms from 1 to 99, and a concurrency outside 1 to 16, with Bad arguments for Crawler '<name>': …. It extends Timer_Node and refills on a recurring timer: the one-second tick with no delay, every delay_ms with one.

The crawler declares no commands. Its pacing is set by make_node alone, because a worker rebuilds the graph from its .tsl at least every ten minutes and a change made at runtime would revert without a word. A replay of its arguments that changes only delay_ms or concurrency keeps both siblings and their transfers, and the timer re-arms only when its interval changes; one changing ttl or vault_group rebuilds both.

The crawler owns two siblings, built when its arguments arrive and torn down with it:

Sibling Node Wiring
{name}:curl Curl_Node [vault_group], follow_redirects( false ) Its sink is the crawler's own sink; its target is the crawler, so every answer comes back TO the crawler's name.
{name}:seen Table {name}:seen <name> <ttl> sqlite Its sink is the crawler's own sink; it answers the crawler's Table_Client. The second <name> is the Table's namespace and its file stem.

The Table keeps a key per url seen, written with ADD under the ttl, and the crawler's Frontier: pending, the urls discovered, and inflight, the urls handed to Curl and not yet answered. A url in either set lives Frontier::MEMBER_TTL, one year, however short the ttl, so a frontier waiting longer than the ttl keeps its urls. When pending takes none of the urls an ADD reported new, their seen keys are removed again with RM, and one rate-limited line says so, so a later sighting finds them new. Its sqlite file is {base}/tables/{name}.p{N}.sqlite, so the crawler must be built where a partition is bound, as a topology load binds it; built anywhere else it throws Table_Unavailable, Table <name>: a sqlite backend needs a bound partition.

Message What the crawler does
A TM_BYTESTREAM whose FROM is not the crawler's name (a seed) Its trimmed VALUE must be an absolute http(s) url with a host and no whitespace. A url that is not draws the TM_ERROR below. A valid one is normalized exactly as a link is, resolved against itself, so HTTP://Example.com#top and a page's link to / are one url; it is ADDed and joins pending when the Table reports it new; one seen within the ttl draws the TM_INFO below. Then the crawler refills.
A TM_BYTESTREAM, TM_RESPONSE or TM_ERROR whose FROM is the crawler's name (Curl's answer) The url fetched is its KEY. A body's same-origin links, or a redirect's Location resolved against KEY and kept only on KEY's origin, are ADDed and the new ones join pending; the answer goes to target unchanged, an off-site redirect included; the url leaves inflight, last, so a forward that throws leaves it for the next start to fetch again; and the crawler refills.
A reply from the {name}:seen Table Collected by the crawler's Table_Client.
Anything else Dropped with one rate-limited line.

A refill takes up to concurrency less Curl's transfers_in_flight() from the frontier, one SMOVE from pending to inflight. The window fills only as fast as the frontier does: a seed is the one url in pending until its body is parsed, so a crawl starts with one fetch in flight whatever its concurrency, and a redirect queues only its one Location. An answer Curl sends synchronously, a refusal such as curl_init failed, refills nothing, so its slot waits for the next tick. With fetches in flight, an answer or a tick refills only once half of concurrency is free, so one SMOVE starts several fetches; an empty window refills at once. Under a delay_ms above 0 only the timer refills: a seed or an answer queues what it found and starts nothing, and each fire moves one url. A delay below the Router's tick holds a slot of its own and measures each interval from its last fire. A delay of at least the tick rides that tick on the grid, and no start comes within the delay, less a tenth of a tick for jitter, of the last: a fire short of it is held, and the first tick that clears it starts the url. It hands each url moved to Curl as a TM_BYTESTREAM whose VALUE and KEY are the url and whose FROM is the crawler's name, which Curl copies into its answer. It runs only inside a drain loop, and moves nothing while a recovery is owed. The first refill after the node is built moves everything left in inflight back to pending, Table_Node::MAX_MEMBERS_LIMIT at a time, so a restart fetches again what the last process had in flight; a move the Table does not answer retries on the next tick. A rename builds a new Curl sibling, since the old one's answers carry the old name, and recovery fetches its urls again.

A 3xx answers a TM_RESPONSE whose VALUE is the absolute Location, KEY the url fetched. The crawler discovers that Location by the rule a link meets, resolved, normalized and kept only on the fetched url's origin, so a redirect to another host, scheme or port is forwarded to target and not followed. A seed on http:// that redirects to https:// leaves its origin, so seed the final url.

The crawler answers a seed itself in three cases:

TYPE VALUE When
TM_ERROR invalid url <value> A seed's trimmed VALUE is not an absolute http(s) url with a host, holds whitespace or carries userinfo (https://u:p@host/).
TM_INFO already seen <url> The Table's ADD reports the seed's normalized url seen within the ttl. KEY is that url, and nothing is fetched. A link or Location already seen is skipped without a word.
TM_ERROR ADD to <name>:seen failed <url> The {name}:seen Table refused the seed's ADD or left it unanswered, so the url is not recorded as seen. KEY is that url, and nothing is fetched.

Each is a copy of the seed with its whole TYPE and its VALUE replaced and its TO cleared, sent to target, so FROM and ID stay as the seeder set them. Every TM_ERROR Curl answers (see Curl_Node) reaches target as well, KEY set to the url. An answered url is done, TM_ERROR included: it is never retried, so a permanent 404 cannot loop, and it is not fetched again until its ttl expires.

Crawler_Node::links( $html, $page_url ) reads a page's links. It parses the body with \DOMDocument::loadHTML() and takes every <a href>, resolving each against the first <base href> or else the page url by RFC 3986 section 5.2: dot segments removed, the fragment dropped, an empty path read as / and a default port dropped. A link is kept when its origin, from the public Curl_Node::origin_of(), equals the page's: the same scheme, host and effective port. A link that is not http(s), carries userinfo or holds whitespace is skipped, and each url is returned once, in document order.

One Crawler runs per name per host: its Table's namespace and file are its name, so two workers running one crawler share the file and re-queue each other's inflight. Topology_Analyzer claims that file in the write set, so two active topologies declaring one crawler conflict, and lists the Table in declared_tables(). The vault group adds nothing to either: Curl reads it per fetch, and it builds no node. The crawler does not honor robots.txt, keep a depth limit or check the content type, and it retries nothing; the ttl and the same-origin rule bound the crawl.

Constant Value Bounds
TICK_MS 1000 The refill tick with no delay_ms, which also carries the first tick's recovery.
MIN_DELAY_MS 100 The least positive delay_ms. Every fire asks the Table for a url whether or not one is pending, so a shorter delay costs an idle crawl a Table round trip that often.

Frontier — durable work a node has yet to finish

Frontier keeps a node's work where neither a recycle nor a crash loses it, in a durable Table sibling the node owns and asks through its own Table_Client. A node builds it beside that client, new Frontier( $this->tables, fn (): string => $this->sibling_name( 'seen' ) ), so the Table's name is read at each ask and follows a rename. It holds no Table and no reply of its own: the node's fill() hands every message to the client first, as any asker does. Crawler_Node is its first user.

Method What it does
add( array $members ): bool SADD member => value into pending for MEMBER_TTL; a member already there takes the new value and expiry. Answers whether pending took them.
take( int $count ): array Up to $count members, lowest first, moved from pending to inflight in one SMOVE, as [ member, value ] pairs. A count below 1 takes none.
settle( array $members ): void Take the members out of inflight: their work is done. The next take(), restart(), drain() or count() removes every member settled since in one SREM.
restart(): void Owe a recovery, as a new process does, for an owner that dropped the work it had in flight.
drain(): array Empty both sets, pending in take()s of MAX_MEMBERS_LIMIT and inflight in one SREM, and answer every member they held.
flush(): void Remove what was settled since in one SREM, as an owner's shutdown_sweep() does on a clean stop. An SREM the Table refuses or leaves unanswered keeps those members owed to the next flush.
count( string $state, string $prefix = '' ): ?int How many members of PENDING or INFLIGHT start with $prefix, every member with none, once what was settled is removed and any recovery owed has finished, as before a take(), so a successor never counts its predecessor's members as in flight: one SCARD of the Table through Table_Client::count_members(), or a count of the process's own sets for a frontier with no Table. Null when the Table does not answer the count or the recovery; any other state throws \InvalidArgumentException.

Built with no arguments, new Frontier() keeps both sets in the process, in the order added rather than lowest first, behind the same calls, for an owner whose work cannot outlive it.

A member is consumed only when it is settled, so a crash costs an unsettled member a second attempt and nothing more. The first take() a process makes, or the first after restart(), returns everything in inflight to pending, Table_Node::MAX_MEMBERS_LIMIT at a time; a move the Table does not answer leaves the recovery owed, and nothing is taken until it finishes, or it would return live members to pending. Once a take() moves fewer members than it asked for, take() stops asking the Table until an add() or a recovery puts members in pending, so an idle owner's tick costs no round trip. One partition has one writer, so each member is taken once; two workers sharing one Table would re-queue each other's inflight.

Members share the Table's key space with whatever else the owner keeps there, as the crawler's seen keys do. A member added while it is in flight sits in both sets until it is settled. A member settled but not yet removed is in inflight still, so a crash before the next take() costs it a second attempt too; a clean stop flushes it first.

Constant Value Bounds
PENDING pending The set of members added and not yet taken.
INFLIGHT inflight The set of members taken and not yet settled.
MEMBER_TTL 31536000 (one year) How long a member waits in either set.

Topology dirs

Topology_Registry resolves a name through the dirs registered with it: each stock dir, the last registered first, then the user dir. A deployment shadows what a plugin ships by registering a dir holding the same name.

Method What it does
resolve( string $name ): ?string The path of <name>.tsl or <name>.tsl.php, through resolve_file().
resolve_file( string $filename ): ?string The path of the first file of that name, or of that name plus .php, in those dirs, or null when neither file exists in any of them. Within a dir the exact name wins. Each dir answers from one listing, read again when its mtime or inode moves, so a lookup stats no file a dir does not hold, and a dir that will not list throws naming it. A directory of that name is skipped; an unreadable file still answers, so its reader fails loud. A name carrying /, \ or .. answers null, since it could reach outside the dir.
stats(): array<string,string> Every registered dir in resolution order as mtime:inode, then each topology in it as mtime:size:inode, reading each dir once and stat'ing only what it lists. Bootstrap::row_table() keys its shared cache on it.

Extensibility hooks

Every hook the substrate fires, and the ones a consumer plugin is expected to answer. Every newspack_nodes/* name and signature is frozen surface — see stability.md.

Four rails, read left to right: a command request from rest_api_init through declare_config_keys, the capability_map and command_rate_limit filters, request_graph_ready and the per-command capability check to the stderr action; the reconcile pass from the cron event through before_reconcile, the topologies filter, retention's registered_log_producers and periodic to after_reconcile in a finally; a spawn to spawn_worker; and a job from the two handler filters through before_job, the handler, after_job and batch_complete; with the settings, admin and topology-fired hooks beneath.

Actions

Hook Arguments Fired from
newspack_nodes/request_graph_ready Command_Interpreter_Node $base_interpreter Bootstrap::mount_request_graph(), on every command door. Mount your service CIs here — see below.
newspack_nodes/worker_identified string $type, int $partition Worker_Base::execute(), once the worker holds its lock, and only where Spawn_Controller tagged the process as that worker. The identity is the validated pair, never the raw request; a refused spawn, a spawn that loses the lock race and wp nodes run fire nothing. A request logger that named the process at its start renames it here. The announcement runs inside the topology load path, so a listener's RuntimeException ends the run as a malformed .tsl does: the lock is released, the throw escapes execute(), and a successor is never spawned.
newspack_nodes/spawn_worker string $type, int $partition Spawn_Controller::spawn(). Build the worker for $type and ->execute() it. Topology_Registry::spawn_worker handles every active topology already.
newspack_nodes/reconcile — The WP-Cron event itself, on the registered 60-second newspack_nodes_minute schedule. Bootstrap::reconcile_fleet() is its handler.
newspack_nodes/before_reconcile — Bootstrap::reconcile_fleet(), before the pass.
newspack_nodes/periodic — Bootstrap::reconcile_fleet(). The minute-cadence tick for work that needs no worker, fired as the last of the reconciliation steps. The substrate's own minute work — Alerts::emit() and Job_Delay::sweep() — runs as steps of their own ahead of it, not as subscribers. The whole action is ONE step, so a subscriber that throws takes every later subscriber's window in that tick with it; the other steps all run regardless, and the failure is then written to stderr beside theirs as periodic: <cause>, behind the pass's reconcile[<pid>] process label.
newspack_nodes/after_reconcile — Bootstrap::reconcile_fleet(), after the pass's last step, whatever the steps threw, and after their failures are written to stderr, so a subscriber closing a log context closes it after them. Its own failure is written last, as after: <cause>, and the callback returns.
newspack_nodes/restart_fleet string $name Topologies_CI_Node, once per AFFECTED fleet on a topology save or delete — a saved child restarts every parent that composes it, transitively. Worker_CLI_Command::restart_fleet_by_name() is registered on it at load and restarts every partition of the named fleet, so a substrate-side restart already happens; a name that is not carried by any active worker restarts nothing.
newspack_nodes/declare_config_keys — Config, on the first key check of the process and again on any miss. Call Config::register_keys() here and nothing else — see below.
newspack_nodes/config_reset — Config::reset(). Drop anything memoized from config: the substrate drops log-dir scans, parsed TSL and vault credentials here.
newspack_nodes/job_worker/after_job string $handler, string $id, ?array $outcome Job_Worker_Node, always — after a success, a throw, or a decline. Tear down per-job request context here. A listener that throws fails the job: its failure joins the job's own and propagates to the Consumer.
newspack_nodes/job_worker/batch_complete string $batch Job_Worker_Node, when a batch's last job settles.
newspack_nodes/vault/changed string $id, string $action, string $previous Vault_CI_Node, on any credential write, a group edit included. $action is added, updated, renamed or removed; $previous carries the id a rename moved away from, else ''. Bootstrap::reload_vault_consumers() listens here to signal RELOAD to every topology holding a Remote_Source, HTTP_Source or Vault_Group, and to reset the analyzer's caches so a planning read later in the same process sees the write.
newspack_nodes/stderr string $text, string $keyword Core::_stderr(), beside the stderr handler and under the same re-entry guard. A listener that throws cannot break the last-resort diagnostic path, and one that calls stderr() itself short-circuits to error_log rather than recursing. $keyword is what a logger files the line under: Core::STDERR_KEYWORD, stderr, for every stderr() line. Core::stderr_to_seam( $text, $keyword ) fires it with the line stderr() would compose under a keyword of its own and does not run a handler, so the line never reaches error_log; a worker writes its routine stops (stop requested, restart requested, timeout, idle: no message for <n>s) this way, under worker (stop).
newspack_nodes/settings_after_form — Admin, below the settings form.

Hook_Node fires whatever name its required hook_name argument carries — do_action( $hook_name, $value ) by default, apply_filters in filter mode — so a topology can add arbitrary names to this list at runtime. A listener is handed the VALUE alone as a single argument, never the 7-field envelope, so it can read and (in filter mode) rewrite the payload but cannot re-address the message: TO, FROM, ID, STREAM and TIMESTAMP never reach it, and Hook mints nothing, so FROM crosses untouched and still names the source that stamped it. Registering with accepted_args above 1 gains nothing. Action mode leaves VALUE and TYPE as they stand; filter mode adopts the return AND restamps TYPE from its shape — a list array becomes TM_STRUCT and everything else, an associative array included, becomes TM_BYTESTREAM, which a consumer gating on TM_STRUCT then refuses.

Newspack_Log_Node fires newspack_log with ( string $code, string $text, array $params ) — Newspack Manager's hook, not the substrate's own. Every entry rides at 'type' => 'debug' and 'log_level' => 2, with 'data' carrying the message VALUE when it is an array and [] otherwise. The level is the contract, not a detail: Manager ships an entry to WPCOM and logstash fire-and-forget at log_level 2 or above, and debug is what keeps it out of the paging path.

Filters

Hook Filtered value Applied from
newspack_nodes/topologies array $topologies — name => entry Bootstrap::get_topology_catalog() and Topologies_CI_Node. Topology_Registry::publish_catalog populates it from every registered .tsl dir, and raises a dir that will not list, naming it, rather than publish a catalog short of its topologies. It runs at the default priority 10 and skips any name already in the array, so a consumer pinning a topology's num_partitions, stale_timeout, on_demand_idle or max_runtime must hook BELOW 10 for its entry to stand; above 10 it receives the synthesized entry and replaces it. Never return [] transiently — see below.
newspack_nodes/job_handlers array $handlers — name => callable Job_Worker_Node::load_handlers_from_filters(). A name failing HANDLER_NAME_PATTERN or a non-callable value is skipped, never refused, so one bad registration cannot cost a worker every other handler.
newspack_nodes/remote_job_handlers array $handlers The same, for entries whose k selects the remote map.
newspack_nodes/compositions array $declarations — namespace => { blocks, contexts } Composition::declarations(), once a process on first use. blocks is name => { title, contexts: list<string>, attributes: { name => JSON schema with a default }, container?: bool, multiple?: bool }, where shared: true replaces contexts; contexts is name => { default: string, max_blocks: int, check?: \Closure, admits?: list<string>, patterns?: { slug => { title: string, file: string } } }, default the absolute path of the consumer's markup file, admits the shared blocks, from any namespace, that stand on the context beside its own, and check( string $block, array $attributes, string $context ): void throwing \RuntimeException to refuse. A patterns entry offers a named layout: its slug matches [a-z0-9-]+, its file is the absolute path of an existing markup file, and the markup must pass Composition::validate_shape() for the context once every namespace is read, so it may name an admitted shared block; check is not run on it. Composition::schemas() lists each as { name, title, blocks } in declared order, blocks the tree validate_shape() answers for its markup, [] for a context with none. Every name sits under <namespace>/. A container holds blocks and stands only at the top, in a core/column or in a core/group; multiple: false lets a block stand once. A shared block declares no contexts and does not stand on any context of its own, and each admits name is checked once every namespace is read: it must be a declared shared block. A declaration outside its namespace, an attribute with no type or default or a default its own schema refuses, a block naming an undeclared context, a block declaring shared and contexts or neither, a context admitting a block that is undeclared or not shared, a context whose default file is missing, a pattern with a slug or title that breaks its rule, patterns declared as a list, a pattern that is not a map, and a pattern whose file is missing or whose markup the context refuses each throw \LogicException naming the namespace and the key, refusing the whole set; a throw memoizes nothing, so the next call applies the filter again.
newspack_nodes/job_worker/before_job bool $run, string $handler, string $id, array $message Job_Worker_Node. Return false to DECLINE the job.
newspack_nodes/capability_map array $map — role => WP capability Capabilities::cap_for(). The baseline is Roles::defaults(): all three roles map to manage_options until a site installs the granular capabilities, and then to newspack_nodes_{read,tune,manage}. Return all three roles — the union form fn ( $map ) => [ 'read' => 'edit_posts' ] + $map, never a bare [ 'read' => … ]; see below.
newspack_nodes/oauth_resources array $resources — canonical resource URIs Oauth_Server. Declare each resource the OAuth server issues sessions for, such as an MCP door's URL; its RFC 9728 metadata is served for a declared resource alone. Non-string entries are dropped, and a non-array result declares nothing.
newspack_nodes/command_rate_limit int $burst HTTP_In_Node::check_permission(): /command POSTs per user per one-second window. Clamped to 1 through Rate_Limit::MAX_BURST (256).
newspack_nodes/auth_rate_limit int $burst Auth_Controller::check_permission(): /auth mints per user per 60-second window, 30 by default. Clamped to 1 through Rate_Limit::MAX_BURST (256).
newspack_nodes/registered_log_producers array<int,string> $producers — path templates Log_Cleaner. Declare a log dir the retention sweep must know about. Non-string and empty entries are dropped and duplicates collapse, but a well-formed template that does not resolve to a directory under <config:logs_dir> anywhere in the partition range is NOT dropped: it refuses the whole sweep, offsets/ included, raising log producer <template> declares no dir under the logs root <root> before any delete, and dump_graph lists every other log and names the template under refused_producers. Core::resolve_partition_template expands each template over Bootstrap::global_num_partitions() — the global num_partitions clamped to Spawn_Coordinator::MAX_PARTITIONS, never the declaring topology's own count — so a producer writing past that range leaves undeclared dirs the sweep will take. A template carrying no {partition} token collapses to one directory pinned across the fleet, which is how alerts.p0 is declared. One log dir escapes this rule: Log_Cleaner reads Settings_Event_Writer::SETTINGS_LOG_DIR directly and seeds settings.p0 at partition 0 into an already non-empty declared set, because the settings writer has no .tsl write-set entry and does not register a template. Do not copy the settings writer as the model for a new PHP producer — one that declares nothing here is reaped.
newspack_nodes/segment_size_overrides array<string,int> $overrides — basename => bytes Workers_CI_Node. Declare the geometry of a Partition built in PHP rather than by a make_node line, which has no literal size to read. The union keeps the left side, so the filter fills gaps and never restates.
newspack_nodes/settings_sync/value mixed $value, string $option Settings_Sync_Node. Resolve the value a hub pushes to its spokes — a requirement for any option that can be absent, not an optional refinement. push() reads \get_option( $local ) with NO default and never goes through Config::value(), register_setting() defaults apply only inside is_admin() requests and a worker is never one, and Reset_Gate DELETES the row on reset-to-default. An unsaved or freshly-reset option therefore resolves to false, Core::as_string() makes that '', and the spoke's typed receiver refuses it with invalid value for setting: …. A hub syncing the substrate's own remote_* geometry needs a resolver mapping absent to the owning config's default; the filter runs on every push, the periodic sweep included. The substrate does not register a handler of its own.
newspack_nodes/settings_audit_values_allowlist array $options Settings_Event_Writer, over the Settings_Schema option names. Options whose old and new values may ride in a settings-audit record; everything else is logged by NAME only. The encrypted vault option is refused BEFORE the filter runs, so a filter cannot opt the credential store back in.
newspack_nodes/station_tab_bundles array $bundles Admin. Register a station tab bundle: handle, dir and url are required, localize and lazy optional. A malformed entry is dropped whole, so one bad contribution cannot break the others. Leave lazy off: it skips the enqueue and puts the load recipe on the station handle, but the fetch is driven by placeholders lazyTabs.js registers for the four substrate handles it names, so a bundle contributed under any other handle would simply never load.
newspack_nodes/overlay_pages array $pages Admin::overlay_pages(). Admin page slugs the debug overlay should mount on; non-strings are filtered out.

A capability_map callback REPLACES the map. cap_for() reads $map[ $role ] and throws InvalidArgumentException( "unknown capability role: <role>" ) when the value is missing or empty — and when the filter returned anything but an array. So a callback written the usual WordPress way, returning only the key it means to change, breaks every tune and manage check. The consequence differs by call path, and neither is a permission denial. Nothing catches the throw on a REST permission callback (HTTP_In_Node::check_permission, Auth_Controller, Spawn_Controller) or at admin-menu registration, where Admin passes Capabilities::cap_for( Capabilities::MANAGE ) straight into add_menu_page(), so a partial map is a 500 and a dead wp-admin; inside a CI command Command_Interpreter_Node::interpret() catches it and returns it as TM_COMMAND|TM_ERROR. wp nodes caps status resolves all three roles and throws too.

An empty topologies result retires the WHOLE fleet. When Bootstrap::expand_workers() comes back empty on a pass following one that saw workers, Fleet_Node::refresh_active_set() calls drain_all_workers(), which writes Lock_Node::RESTART_FLAG into every *.lock.d under locks/ — every worker in the tree, not only the types that vanished — and each holder exits with its self-respawn refused, because its type is no longer in the active set. Only a worker's first scan is exempt, by the observed-not-assumed guard on the previously-read set. Deactivating the last topology is the intended trigger; a provider returning [] on an internal error, or a deploy that removes the .tsl files under a live fleet (Topology_Registry::resolve() hits the disk on every call), takes the same path.

newspack_nodes/request_graph_ready

Fires from Bootstrap::mount_request_graph() once the request-scope graph has been built or confirmed already-built. That builder is shared by every command door, not just /command, so a door never ends up with a different command surface behind it. At this point Core's node map holds _router and _command_interpreter (the base CI); HTTP_In_Node names itself _output immediately after.

Signature:

do_action( 'newspack_nodes/request_graph_ready', \Newspack_Nodes\Command_Interpreter_Node $base_interpreter );

Canonical usage — applications mount their service CIs through the base CI's make_node():

function my_app_mount_service_cis( \Newspack_Nodes\Command_Interpreter_Node $base_interpreter ): void {
    $base_interpreter->make_node( 'My_Service_CI', 'my-service' );
    // ... more service CIs ...
}
\add_action( 'newspack_nodes/request_graph_ready', 'my_app_mount_service_cis' );

make_node( string $type, string $name, ...$args ): ?Node does four things and returns the node:

  1. Resolves and instantiates, via a no-arg new $fqcn(), the first {$prefix}{$type}_Node that exists and is a concrete Node subclass, looping the prefixes registered through Command_Interpreter_Node::register_namespace() at plugin load. So make_node( 'My_Service_CI', … ) resolves My_App\My_Service_CI_Node once My_App\ is registered. There is no per-class registry — a plugin registers its namespace prefix once (ADR-10). It returns null when every prefix fails to yield a concrete class.
  2. Calls $node->name( $name ) so Router can find it.
  3. Calls $node->arguments( $arg_tokens ) — the scalar positional args, cast to strings and re-indexed, as a flat token array (arguments() takes and returns list<string>, never a space-joined string). They map onto the node's declared node_schema()['arguments'] properties, so config round-trips through dump_config(), which re-joins the tokens via Node::serialize_args(). A non-scalar argument is dropped with a rate-limited warning; assign object dependencies as public properties after make_node returns, as the substrate does for Workers_CI's CLI.
  4. Calls $node->sink( $this ) so the node's reply routes back through the base CI to _router and out through _output.

Redeclaring a name with the SAME class and the same tokens returns the node already registered, so re-mounting on every request is idempotent; a genuine collision — same name, different class or different tokens — throws. A constructor or name() that rejects is cleaned up, never left orphaned.

Skipping make_node() — constructing and name()-ing a node by hand — leaves it unwired. Its command responses walk back via TO=FROM (ADR-7), cannot find a path to _output, and drop on the floor. Always go through make_node().

Because the hook fires on every command request, keep CIs stateless: pure command dispatchers with their dependencies injected. For the application-side build-out, read the per-CI node_schema() declarations under newspack-event-logger-nodes/includes/app/.

newspack_nodes/declare_config_keys

Config::value() refuses an undeclared key: it throws unknown config key '<key>' — not declared by any registered schema rather than answer null. The substrate derives the declared set on the first key check of the process — registering its own Settings_Schema overlay keys, then firing this action so every consumer plugin declares its own — and Config::is_declared() fires it again on a miss, which is how a plugin that loads after that first read still gets its keys accepted for the rest of the request.

Signature:

do_action( 'newspack_nodes/declare_config_keys' );

No arguments. A callback calls Config::register_keys( array $keys ) with UNPREFIXED key names, and nothing else:

function my_app_declare_config_keys(): void {
    \Newspack_Nodes\Config::register_keys( \My_App\Settings_Schema::get()->overlay_keys() );
}
\add_action( 'newspack_nodes/declare_config_keys', 'my_app_declare_config_keys' );

Registration is VALIDATION, not resolution. Config::load_config() builds its map from Settings_Schema — the substrate's own schema — plus the two config files plus the newspack_nodes_-prefixed option overlay, so a consumer's registered key is never in that map and \Newspack_Nodes\Config::value( 'my_key' ) answers null, silently and forever — the ?? default failure this whole action exists to prevent. A consumer owns its own Config whose value() calls Newspack_Nodes\Config::is_declared() to validate the key and then reads its OWN merged config — its schema defaults, its own option overlay, and the substrate config layered underneath — as Newspack_Event_Logger_Nodes\Config::value() and its load_config() do. The one exception is a consumer key an operator writes into LOCAL_NEWSPACK_NODES_CONF, which is deliberately free-form and does land in the substrate's map.

Declare from CODE — a schema, or a literal defaults array — and never from a config file's keys, which is the rule Config::declare_keys() follows for the substrate's own. Deriving from the file makes an operator's typo self-declaring: the misspelling becomes valid, the real key falls back to its default, and nothing says so. It also leaves an install whose file predates a key unable to read that key at all. A plugin whose schema names only its overlay keys registers the union, as nuclear-gyrobase does with its code defaults and its WP option schema.

Register the hook at PLUGIN FILE SCOPE, not from a deferred plugins_loaded loader. The substrate PULLS the declaration from inside a read, and that read can precede the loader: event-logger-nodes builds its logger at plugins_loaded:-11001, well ahead of a loader at priority 11, and value() would throw on a real key. A plugin whose slug sorts before newspack-nodes loads before any substrate class exists, so it hooks the literal action name rather than the Config::DECLARE_ACTION constant.

Do nothing else in the callback. It runs from inside a config read, on any request and in any process, so a config read, an option write or I/O from here fires at an unpredictable point in the request. A read of a still-undeclared key from a callback is re-entrant — the declaring guard bounds it, but it cannot see keys a later callback declares.

Declarations accumulate and are never pruned. Config::reset() drops the cached config and re-arms the derive, and the declared set survives it, so a dropped callback cannot un-declare keys that already resolve. The registry is flat and shared: every plugin's keys land in the one set, and Config::is_declared() is the primitive a consumer plugin's own value() accessor calls to validate a key before reading its own merged config.