Commit efb1e1a
feat(agent,cli,publisher): OT-RFC-38 LU-6 — opaque SWM hosting on cores + member host-catchup fallback
Lands the last Phase A milestone for SPEC_CG_HOSTING_MEMBERSHIP: cores
opaquely host a curated CG's encrypted SWM substrate without ever
holding the chain key, and a member that was offline (but whose
sender-key state survived on disk) can recover the missed history from
cores when the curator AND every other member is also offline.
The §1.1 user-visible promise — "any member can recover the full
history as long as one peer that has the bytes is reachable" — now
holds for the "every member is offline; only cores are reachable"
case. The §5.2 invariant ("cores never possess plaintext or keys") is
preserved: cores store the wire bytes verbatim, never attempt
decryption, and `SwmHostModeStore` rejects zero-length envelopes.
Source surfaces
- `packages/agent/src/swm/host-mode-store.ts` (new) — file-backed
append-only per-CG log of opaque ciphertext envelopes. Separate
TTL + per-CG byte cap for unregistered (default 6h / 1 MiB —
pre-registration staging per §1.2) vs registered (30d / 64 MiB).
Eight-byte-BE timestamp + seqno + four-byte-BE len framing; one
file per CG, named by `sha256(cgId)` base64url so user-supplied
ids stay filesystem-safe.
- `packages/agent/src/swm/host-catchup-wire.ts` (new) — JSON wire
format for the new libp2p request/response protocol
`/dkg/10.0.1/swm-host-catchup`. `denied` vs empty `entries`
distinguishes "I refuse to serve" from "you're up-to-date / I have
nothing"; envelope bytes base64-encoded so the same JSON works as a
debug-tool target.
- `packages/agent/src/dkg-agent.ts` — host-mode reconciler that
subscribes cores to a curated CG's SWM topic in HOST MODE (drops
bytes that aren't ciphertext, accepts and stores those that are),
the `/dkg/10.0.1/swm-host-catchup` request handler, the member-side
`catchupSwmFromHost` / `catchupSwmFromConnectedHosts` clients, and
the `enableSwmHostModeFor` operator surface for explicit
designation.
- `packages/publisher/src/workspace-handler.ts` — new
`{ trustedReplay: true }` option on `SharedMemoryHandler.handle()`
that skips the two pubsub-transport-layer peer assertions
(`publisherPeerId === fromPeerId`, peer-allowlist gate). The
cryptographic chain — gossip-envelope signature verification +
sender-key AEAD decryption — is still enforced for every replayed
envelope, so a host can't forge or tamper with what it stored
opaquely, only relay it.
- `packages/cli/src/daemon/routes/memory.ts`:
- `POST /api/shared-memory/host-mode/subscribe` — operator-driven
designation (Phase A surface that the future sharding-table
auto-subscribe plugs into).
- `POST /api/shared-memory/host-catchup` — dedicated member-side
catchup endpoint for debugging a specific peer's hosting.
- `GET /api/shared-memory/host-mode/stats` — per-daemon diagnostics
(cgCount / totalBytes / totalEntries / subscribedCgIds).
- Auto-fallback in `POST /api/shared-memory/catchup`: when the
standard sync path inserts 0 triples, transparently invokes
`catchupSwmFromConnectedHosts` against the same peer set. Opt
out with `{ hostCatchupFallback: false }`.
- `packages/core/src/constants.ts` — `PROTOCOL_SWM_HOST_CATCHUP`
string.
Devnet validation (`scripts/devnet-test-rfc38-late-joiner.sh`)
- SCENARIO D (new, LU-6 happy path): curated CG with `[curator,
member]`; cores explicitly designated via the new
`host-mode/subscribe` endpoint (note: cores are NOT pre-created on
CG_D — gossiped meta would otherwise expand the allowlist union
and shortcut past the host-mode path). Curator handshake + 1
triple → member receives chain key. Member killed. Curator writes
5 more (ciphertext flows to cores). Curator killed. Member
restarted (chain key persists on disk). Catchup endpoint returns
`hostCatchup.ranFallback: true` and member ends with all 6
triples. The test asserts on `totalEntries > 0` across cores AND
on `hostCatchup.ranFallback === true` so we'd catch a silent
regression where the host-mode path stops running.
- SCENARIO C updated: now correctly documents that cores DO serve
ciphertext, and the outsider's 0-inserted result is the
confidentiality invariant (no chain key → AEAD decrypt fails on
apply), not a missing-hosting gap.
Documentation
- `docs/specs/SPEC_CG_HOSTING_MEMBERSHIP.md` §7.1.1 — LU-6 moved
from "deferred" to "landed" with a description of the operator-
driven designation surface, the wire protocol, the auto-fallback
in catchup, and the explicit Phase B carry-overs (sharding-table
auto-subscribe, per-wallet rate limits, cross-core ciphertext
re-gossip).
Run instructions
./scripts/devnet.sh start 6
./scripts/devnet-test-rfc38-late-joiner.sh # ~2 min, 4 scenarios
Tested
- All 4 late-joiner scenarios PASS on a fresh 6-node devnet (4 cores
+ 2 edges). SCENARIO D output shows
`totalEntries: 2` × 4 cores = 8 stored envelopes,
`hostCatchup.ranFallback: true`, `applied: 1` (the SWM batch
envelope; the sender-key setup envelope was already processed
pre-kill so its replay is correctly skipped), and `N_D_POST: 6`
via SPARQL.
- `packages/agent/test/swm/host-mode-store.test.ts` — 8 new unit
tests covering monotonic seqno, persistence across restarts, TTL
pruning, byte-cap enforcement, registered-vs-unregistered limit
switching, zero-length-envelope rejection.
- `packages/publisher` test suite — 965 passed / 1 skipped (the
`trustedReplay` option change is additive; no regressions).
Co-authored-by: Cursor <cursoragent@cursor.com>1 parent 94c96bd commit efb1e1a
10 files changed
Lines changed: 1724 additions & 25 deletions
File tree
- docs/specs
- packages
- agent
- src
- swm
- test/swm
- cli/src/daemon/routes
- core/src
- publisher/src
- scripts
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
837 | 837 | | |
838 | 838 | | |
839 | 839 | | |
840 | | - | |
841 | | - | |
842 | | - | |
843 | | - | |
844 | | - | |
845 | | - | |
846 | | - | |
847 | | - | |
| 840 | + | |
| 841 | + | |
| 842 | + | |
| 843 | + | |
| 844 | + | |
| 845 | + | |
| 846 | + | |
| 847 | + | |
| 848 | + | |
| 849 | + | |
| 850 | + | |
| 851 | + | |
848 | 852 | | |
849 | 853 | | |
850 | 854 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
38 | 38 | | |
39 | 39 | | |
40 | 40 | | |
| 41 | + | |
41 | 42 | | |
42 | 43 | | |
43 | 44 | | |
| |||
600 | 601 | | |
601 | 602 | | |
602 | 603 | | |
| 604 | + | |
| 605 | + | |
| 606 | + | |
| 607 | + | |
| 608 | + | |
| 609 | + | |
| 610 | + | |
| 611 | + | |
| 612 | + | |
| 613 | + | |
| 614 | + | |
| 615 | + | |
| 616 | + | |
| 617 | + | |
| 618 | + | |
| 619 | + | |
| 620 | + | |
| 621 | + | |
| 622 | + | |
603 | 623 | | |
604 | 624 | | |
605 | 625 | | |
| |||
0 commit comments