Skip to content

Commit 5dd395d

Browse files
Branimir Rakiccursoragent
authored andcommitted
fix(agent,cli,publisher,scripts): OT-RFC-38 LU-6 — address Codex PR #610 R2
Three follow-ups from the second Codex pass on commit 730e58a: 1. memory.ts:765 — host-catchup applied counter was conflating envelope counts with triple counts. One SWM envelope can carry many quads, so summing the boolean `applied` flag undercounted recoveries whenever a publisher batched > 1 triple per share. Threaded `appliedTriples` through SharedMemoryApplyOutcome (`insertedTriples` from `quads.length` on the success path) → `catchupSwmFromHost` → `catchupSwmFromConnectedHosts` → `/api/shared-memory/catchup` + `/api/shared-memory/host-catchup`. `appliedTotal` now reports triples (the user-facing unit); `appliedEnvelopes` is exposed alongside for operators who want the envelope count. `totalInsertedTriples` is rolled up from the correct unit (verified end-to-end on the late-joiner devnet: appliedTriples=5 vs appliedEnvelopes=1 for a 5-quad batch). 2. dkg-agent.ts:8497 — reconciler re-entry was returning before `maybeMarkRegisteredForHostMode()` ran, so a core that subscribed while a CG was unregistered stayed on the 6h/1MiB pre-registration limits forever — even after the CG was later registered on chain — and pruned ciphertext from registered CGs much earlier than intended. Mirror the same fix R1 applied to `enableSwmHostModeFor()` on the periodic reconcile path. 3. workspace-handler.ts trustedReplay — the R1 bypass of the two transport identity checks had no focused coverage (only the SwmHostModeStore was unit-tested). New test file `workspace-handler-trusted-replay.test.ts` covers: a. valid host replay applies (publisher ≠ fromPeerId, allowlist only includes publisher, signature + CG binding all valid) b. control: same wire bytes WITHOUT trustedReplay are rejected (proves the bypass is the only thing letting (a) through) c. tampered envelope signature still rejects under trustedReplay (agent-gate verification runs first; bypass MUST NOT defeat cryptographic identity) d. encrypted payload bound to a different CG still rejects (CG binding check runs before decryption) e. missing decryptor state still rejects as retryable (agent- gated analogue of "no sender-key state yet") Bonus: tests assert `insertedTriples` is set on the apply outcome from (a), locking in the R2-1 contract. Devnet hardening: scripts/devnet-test-rfc38-late-joiner.sh now calls `wait_for_peer_link` before SCENARIO B's curator write to both other members. SCENARIO A→B transition was flaking with "All multiaddr dials failed" when run from a cold-started devnet because stale dial cache entries from A weren't expired before B's 3-way sender-key handshake fan-out. Verification: - All 5 new trustedReplay tests pass - All 21 agent-gate tests still pass - 9 host-mode-store tests still pass (incl. R1's seqno recovery) - Pre-existing strict-equality test in workspace.test.ts updated to include the new `insertedTriples` field - All 11 RFC-38 scenarios pass on a clean devnet (lu5-pub, lu5-cur, lu7..lu10, e2e, xcg, mm, scale, lj) - Late-joiner SCENARIO D explicitly observes the R2-1 fix in hostCatchup.appliedTotal=5 (triples) vs appliedEnvelopes=1 Co-authored-by: Cursor <cursoragent@cursor.com>
1 parent 730e58a commit 5dd395d

6 files changed

Lines changed: 425 additions & 7 deletions

File tree

packages/agent/src/dkg-agent.ts

Lines changed: 33 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -8494,7 +8494,17 @@ export class DKGAgent {
84948494
// local consumption; no need to also opaquely store.
84958495
return;
84968496
}
8497-
if (this.swmHostModeSubscribed.has(contextGraphId)) return;
8497+
if (this.swmHostModeSubscribed.has(contextGraphId)) {
8498+
// Codex PR #610 R2: idempotent re-entry on the periodic
8499+
// reconcile path must still re-probe on-chain registration
8500+
// state. Without this, a core that subscribed while the CG
8501+
// was unregistered stays on the 6h/1MiB pre-registration
8502+
// limits forever — even after the CG is registered — and
8503+
// ciphertext gets pruned much earlier than intended.
8504+
// Mirrors the same safeguard in `enableSwmHostModeFor`.
8505+
await this.maybeMarkRegisteredForHostMode(contextGraphId);
8506+
return;
8507+
}
84988508

84998509
// Only host curated CGs. Public CGs already have plaintext SWM
85008510
// distribution and don't need an opaque ciphertext custodian.
@@ -8726,7 +8736,20 @@ export class DKGAgent {
87268736
): Promise<{
87278737
rounds: number;
87288738
fetched: number;
8739+
/**
8740+
* Number of envelopes whose apply path returned `applied: true`.
8741+
* NOT the same as triples — one envelope can carry many quads.
8742+
* For triples-applied accounting, callers MUST sum
8743+
* {@link appliedTriples}. Codex PR #610 R2 caught the previous
8744+
* conflation where `memory.ts` aggregated this count into a
8745+
* field named `totalInsertedTriples`.
8746+
*/
87298747
applied: number;
8748+
/**
8749+
* Total triples (N-Quads) inserted by successful replays.
8750+
* Summed from `SharedMemoryApplyOutcome.insertedTriples`.
8751+
*/
8752+
appliedTriples: number;
87308753
skipped: number;
87318754
nextSeqno: number;
87328755
denied?: string;
@@ -8738,6 +8761,7 @@ export class DKGAgent {
87388761
let rounds = 0;
87398762
let fetched = 0;
87408763
let applied = 0;
8764+
let appliedTriples = 0;
87418765
let skipped = 0;
87428766
let lastDenied: string | undefined;
87438767
while (rounds < maxRounds) {
@@ -8787,6 +8811,10 @@ export class DKGAgent {
87878811
);
87888812
if (outcome.applied) {
87898813
applied += 1;
8814+
// Triples per envelope is variable; track it separately
8815+
// from the envelope count so callers reporting a
8816+
// triples total don't undercount. Codex PR #610 R2.
8817+
appliedTriples += outcome.insertedTriples ?? 0;
87908818
} else {
87918819
skipped += 1;
87928820
const reason = 'reason' in outcome ? outcome.reason : 'unknown';
@@ -8804,7 +8832,7 @@ export class DKGAgent {
88048832
sinceSeqno = resp.nextSeqno;
88058833
if (!resp.truncated) break;
88068834
}
8807-
return { rounds, fetched, applied, skipped, nextSeqno: sinceSeqno, ...(lastDenied ? { denied: lastDenied } : {}) };
8835+
return { rounds, fetched, applied, appliedTriples, skipped, nextSeqno: sinceSeqno, ...(lastDenied ? { denied: lastDenied } : {}) };
88088836
}
88098837

88108838
/**
@@ -8823,6 +8851,7 @@ export class DKGAgent {
88238851
rounds: number;
88248852
fetched: number;
88258853
applied: number;
8854+
appliedTriples: number;
88268855
skipped: number;
88278856
nextSeqno: number;
88288857
denied?: string;
@@ -8845,6 +8874,7 @@ export class DKGAgent {
88458874
rounds: number;
88468875
fetched: number;
88478876
applied: number;
8877+
appliedTriples: number;
88488878
skipped: number;
88498879
nextSeqno: number;
88508880
denied?: string;
@@ -8861,7 +8891,7 @@ export class DKGAgent {
88618891
} catch (err) {
88628892
const reason = err instanceof Error ? err.message : String(err);
88638893
this.log.warn(ctx, `host-catchup peer=${peerId} cg=${contextGraphId} failed: ${reason}`);
8864-
results.push({ peerId, rounds: 0, fetched: 0, applied: 0, skipped: 0, nextSeqno: options?.sinceSeqno ?? 0, error: reason });
8894+
results.push({ peerId, rounds: 0, fetched: 0, applied: 0, appliedTriples: 0, skipped: 0, nextSeqno: options?.sinceSeqno ?? 0, error: reason });
88658895
}
88668896
}
88678897
return results;

packages/cli/src/daemon/routes/memory.ts

Lines changed: 28 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -750,6 +750,9 @@ WHERE {
750750
type HostCatchupLeg = {
751751
contextGraphId: string;
752752
peers: Awaited<ReturnType<typeof agent.catchupSwmFromConnectedHosts>>;
753+
/** Envelope-level counter from the replay path. NOT a triples count. */
754+
appliedEnvelopes: number;
755+
/** Triples (N-Quads) inserted by successful replays. Maps to the public `appliedTotal`. */
753756
appliedTotal: number;
754757
error?: string;
755758
};
@@ -762,19 +765,30 @@ WHERE {
762765
peers: peerIdParam ? [peerIdParam] : undefined,
763766
maxRounds: 8,
764767
});
765-
const appliedTotal = peerResults.reduce((sum: number, r: any) => sum + (r.applied ?? 0), 0);
766-
hostCatchup.push({ contextGraphId: cg.contextGraphId, peers: peerResults, appliedTotal });
768+
// Codex PR #610 R2: `r.applied` is the count of replayed
769+
// envelopes (booleans), NOT inserted triples. One envelope
770+
// can carry many quads, so summing `r.applied` here would
771+
// undercount whenever a publisher batched > 1 triple per
772+
// share. Use `r.appliedTriples` (threaded through
773+
// `catchupSwmFromHost` / `SharedMemoryApplyOutcome`) for
774+
// the triples total surfaced as `appliedTotal` and rolled
775+
// into the top-level `totalInsertedTriples`.
776+
const appliedTotal = peerResults.reduce((sum: number, r: any) => sum + (r.appliedTriples ?? 0), 0);
777+
const appliedEnvelopes = peerResults.reduce((sum: number, r: any) => sum + (r.applied ?? 0), 0);
778+
hostCatchup.push({ contextGraphId: cg.contextGraphId, peers: peerResults, appliedEnvelopes, appliedTotal });
767779
} catch (err: any) {
768780
hostCatchup.push({
769781
contextGraphId: cg.contextGraphId,
770782
peers: [],
783+
appliedEnvelopes: 0,
771784
appliedTotal: 0,
772785
error: err?.message ?? String(err),
773786
});
774787
}
775788
}
776789
}
777790
const hostCatchupAppliedTotal = hostCatchup.reduce((sum, h) => sum + h.appliedTotal, 0);
791+
const hostCatchupEnvelopesTotal = hostCatchup.reduce((sum, h) => sum + h.appliedEnvelopes, 0);
778792

779793
// Codex PR #610 R1 comment 2: `totalInsertedTriples` must cover
780794
// BOTH the standard sync leg and the LU-6 host-catchup leg so
@@ -830,9 +844,13 @@ WHERE {
830844
hostCatchup: hostCatchupOpted ? {
831845
ranFallback: hostCatchup.length > 0,
832846
triggeredForContextGraphIds: hostCatchup.map((h) => h.contextGraphId),
847+
// `appliedTotal` is triples (the user-facing unit); the
848+
// separate `appliedEnvelopes` is exposed for operators who
849+
// want to know how many discrete shares were replayed.
833850
appliedTotal: hostCatchupAppliedTotal,
851+
appliedEnvelopes: hostCatchupEnvelopesTotal,
834852
perContextGraph: hostCatchup,
835-
} : { ranFallback: false, triggeredForContextGraphIds: [], appliedTotal: 0, perContextGraph: [] },
853+
} : { ranFallback: false, triggeredForContextGraphIds: [], appliedTotal: 0, appliedEnvelopes: 0, perContextGraph: [] },
836854
});
837855
}
838856

@@ -868,12 +886,18 @@ WHERE {
868886
sinceSeqno,
869887
maxRounds,
870888
});
871-
const appliedTotal = peerResults.reduce((sum: number, r: any) => sum + (r.applied ?? 0), 0);
889+
// Codex PR #610 R2: report triples (`appliedTriples`) as the
890+
// user-facing total; keep envelope count alongside as
891+
// `appliedEnvelopes` for diagnostics. Same fix as the
892+
// `/catchup` fallback leg above.
893+
const appliedTotal = peerResults.reduce((sum: number, r: any) => sum + (r.appliedTriples ?? 0), 0);
894+
const appliedEnvelopes = peerResults.reduce((sum: number, r: any) => sum + (r.applied ?? 0), 0);
872895
const fetchedTotal = peerResults.reduce((sum: number, r: any) => sum + (r.fetched ?? 0), 0);
873896
return jsonResponse(res, 200, {
874897
contextGraphId: cgId,
875898
peers: peerResults,
876899
appliedTotal,
900+
appliedEnvelopes,
877901
fetchedTotal,
878902
});
879903
} catch (err: any) {

packages/publisher/src/workspace-handler.ts

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -111,6 +111,22 @@ export type SharedMemoryApplyOutcome =
111111
* Caller addresses the PROTOCOL_SWM_SHARE_ACK to this peer.
112112
*/
113113
publisherPeerId?: string;
114+
/**
115+
* Number of N-Quads that were inserted into the SWM graph
116+
* by this apply (i.e., `quads.length` after decode + decrypt).
117+
*
118+
* Distinct from "envelope was applied" (the boolean
119+
* `applied` flag): one envelope can carry many quads, so
120+
* callers aggregating per-triple counts (e.g. the LU-6
121+
* host-catchup endpoint that reports `totalInsertedTriples`)
122+
* MUST consume this field rather than counting `applied`
123+
* booleans. Codex PR #610 R2 caught the undercount when
124+
* `hostCatchup.appliedTotal` was being summed from envelope
125+
* counts. Optional only because the false variant cannot
126+
* carry it; the true variant always sets it from the same
127+
* `quads.length` used in the success-path log line.
128+
*/
129+
insertedTriples?: number;
114130
}
115131
| { applied: false; reason: string; retryable: boolean };
116132

@@ -1020,6 +1036,7 @@ export class SharedMemoryHandler {
10201036
cgId: contextGraphId,
10211037
shareOperationId,
10221038
publisherPeerId,
1039+
insertedTriples: quads.length,
10231040
};
10241041
}
10251042
// `applied === false` from the withWriteLocks closure. PR-C

0 commit comments

Comments
 (0)