Skip to content

fix(sidebar): converge channel sidebar state across devices - #6525

Open
wpfleger96 wants to merge 52 commits into
mainfrom
wpfleger/channel-sections-sync-fixes
Open

fix(sidebar): converge channel sidebar state across devices#6525
wpfleger96 wants to merge 52 commits into
mainfrom
wpfleger/channel-sections-sync-fixes

Conversation

@wpfleger96

@wpfleger96 wpfleger96 commented Aug 21, 2026

Copy link
Copy Markdown
Member

Problem

Channel sidebar state — sections, sort preference, stars, and mutes — syncs per identity as kind 30078 relay events. That state diverged between a user's devices and sometimes never self-healed. The main gaps: a local edit that lost a conflict was silently republished as remote content while the UI kept showing the edit; edits made inside the publish debounce were dropped on quit or community switch; a skewed remote head could push published createdAt past the relay's future-drift window and wedge all later publishes; and stale-at-open state waited for a reconnect a healthy socket never fires.

What this changes

Convergence. Sections and sort resolve whole-blob conflicts via LWW: when a local edit loses, the manager adopts the winning remote head (writes it through to state, storage, and the watermark) instead of republishing it, unifying with the relay's OK-false conflict path as one mechanism. Stars and mutes move to a per-entry Lamport rev with a single max-merge on every path (ordered by updatedAt, then rev, then the starred/muted-true leaf); a click strictly dominates every state its replica has observed, so it cannot lose to a same-second remote write. Payloads stay version: 1 so older builds keep parsing.

Durability. Every edit is persisted synchronously to a durable per-identity localStorage outbox and resumed on next mount, so edits survive quit, community switch, and the debounce window. Outbox records carry write-once ownership tokens, so one window's completing publish cannot clear another window's still-unpublished edit.

Publish correctness. The pre-publish no-op guard compares the pending store only against the freshly fetched readable relay head, so matching a stale previously-published value never suppresses a publish. Published createdAt is clamped inside the relay's drift window. Publishes use generation-CAS ownership with single-flight and completion re-drive, retained-head confirmation on ACK, bounded 2s→30s retry backoff, and a reconciliation loop (steady 60s, backoff on failure, refresh on window visibility).

Stale-outbox replay guard (P1). A durable outbox entry is only replayed if the relay head it was authored against is still the relay head at resume time. When the relay has advanced, the stale entry is discarded and the current remote state is adopted. This closes the path where a previously-lost edit was re-published above a newer relay head on next mount.

Bootstrap/live-head ordering (P2b). The pre-publish fetch baseline is derived from the bootstrap fetch result directly, not from the mutable lastRemoteHead field that subscribeLive can update during an in-flight bootstrap. A live peer head arriving after the click is held as a genuine remote advance rather than folding into the publish baseline.

Queue-until-bootstrap (P2a). When a publish is debounced while bootstrap is still in flight, the edit is queued rather than fired immediately. On bootstrap completion, releaseDeferred publishes the queued edit using canonicalMax(queueTimeBaseline, bootstrapResultHead) as the baseline — preserving the queue-time baseline when bootstrap returns an older head, and never regressing a click that was authored from a live peer head that fully applied before the click. Three additional invariants: the absent-bootstrap seed call is a no-op when a pending edit is already queued (pendingStore === null guard); failed-bootstrap base establishment uses canonicalMax against {0,""} so the first observed head always wins; and a bootstrapFailedExternalHeadObserved flag disarms the pre-publish exception once any peer head has been independently observed, allowing a genuine post-click peer advance to be adopted rather than published over.

Merge-lane key threading (P3). Stars and mutes pre-publish fetch, reconnect fetch, and restart fetch all thread the retained event ID through the merge, so the key used for the incoming-payload merge is always consistent with the key retained in the watermark after publish.

Structure. All four lanes share two engines: wholeBlobSyncManager.ts (sections, sort) and mergeLaneSyncManager.ts (stars, mutes); each lane manager is a thin config wrapper. Legacy preference migration proves ownership of the legacy key at read time, and retained legacy whole-blob replay is one-shot per value.

Tests. Lane test suites are consolidated into shared parameterized suites (wholeBlobSync.shared, wholeBlobSyncP2a.shared, mergeLane*.shared) plus helper modules, covering the new invariants (durable resume, multi-window outbox, cross-lane isolation, tie-breaks, migration ownership, bootstrap/live-head ordering, queue-until-bootstrap state machine, failed-bootstrap recovery) while shrinking the sidebar test surface from 6,303 to 5,860 lines. Each production defect has a named causal regression (T1–T6) executed through both concrete lane managers (ChannelSectionSyncManager, ChannelSortSyncManager).

@wpfleger96
wpfleger96 requested a review from a team as a code owner August 21, 2026 23:12
@wpfleger96
wpfleger96 force-pushed the wpfleger/channel-sections-sync-fixes branch from 3da6677 to d0c23c9 Compare August 24, 2026 19:34
@wpfleger96 wpfleger96 changed the title fix(sync): converge channel sections across devices on the same identity fix(sync): converge channel sidebar state across devices on the same identity Aug 25, 2026
Duncan and others added 8 commits August 26, 2026 10:54
Channel-section sidebar state diverged between a user's devices and
sometimes never self-healed. Four client-side gaps fed the divergence:

- A local edit that lost whole-blob LWW was silently republished as remote
  content while the UI kept showing the edit. Now the manager adopts the
  winning remote head (writes it through to state + storage, advances the
  watermark) and skips publishing, unifying with the relay's OK-false
  conflict path as one convergence mechanism.
- Edits made inside the 2s publish debounce were dropped on quit or
  community switch. A durable localStorage outbox persists every edit
  synchronously and resumes it on next mount; adopt clears the outbox so a
  superseded edit can never be replayed.
- A skewed remote head could push the published createdAt past the relay's
  future-drift window and wedge all later publishes. createdAt is now
  clamped inside that window.
- Stale-at-open state waited for a reconnect that a healthy socket never
  fires. A reconciliation loop periodically refetches the head (steady 60s,
  backoff on failure) and refreshes on window visibility.

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Three cross-layer races defeated the one-convergence-mechanism design:

- An older in-flight publish unconditionally cleared pending state on
  completion, erasing a newer edit queued mid-flight. Each pending edit now
  carries a monotonic generation; a completion clears pending/outbox/retry only
  via compare-and-swap on the generation it published.
- Hook-level remote application (bootstrap/live/periodic) cancelled the pending
  publish's timers without deciding supersession, stranding the durable outbox
  and clobbering the optimistic edit. applyRemote now defers entirely to a
  pending edit, whose own debounced publish converges via publish-or-adopt; the
  manager's adopt path clears pending before write-through so the winning remote
  still applies.
- The equal-timestamp tie-break kept the largest event id, opposite the
  relay/database canonical order (created_at DESC, id ASC → lowest id wins).
  applyRemote now applies a strictly-lower id and ignores ids >= the last
  applied, so the UI converges on the event the relay actually stored.

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
useChannelStars, useChannelMutes, and useChannelSortPreference carried the
same inverted equal-timestamp comparator as channel sections: applyRemote kept
the largest event id, opposite the relay/database canonical order (created_at
DESC, id ASC -> lowest id wins). Two devices writing the same second could
leave the UI showing an event the relay did not store.

Apply a strictly-lower id and ignore ids >= the last applied, matching the
sections fix and the relay winner across all four 30078 sidebar surfaces. Each
hook gains a regression test: larger-then-lower id delivery at equal timestamp,
lower-id store wins (mutation-checked - reverting >= to <= fails each).

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Two convergence holes one layer under the pass-1 fixes:

Sections: the pre-publish head check compared the fetched head against
the mutable lastRemoteCreatedAt, which a live event observed during the
debounce window already advanced to that same head — equality fell
through to publish and the local blob overwrote a remote that became
head after the edit was queued. Freeze a canonical head baseline
(created_at, id) at publishSections and compare the fetched head against
that generation baseline instead, adopting when the head advanced.

Stars/mutes: applyRemote admits the canonical lower-id winner but then
mergeStores resolved equal per-entry updatedAt as local/prev-wins, so a
stale larger-id value delivered first survived and undid the winner. Add
mergeApplyingRemote which resolves an entry-timestamp tie toward the
canonical incoming blob while keeping strictly-newer local entries.

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Each prior round patched one cross-generation interleaving and opened
another a layer deeper. Kill the race class structurally instead.

Sections: serialize publish cycles (one in-flight at a time; a newer edit
queued mid-cycle defers and the completion re-drives it). The per-edit
pre-publish baseline is frozen at queue time, so a genuine remote observed
during the debounce window still adopts, while our own accepted head is
folded forward via canonicalMax so a stale generation's own write is never
mistaken for a competing remote and adopted away. Dual generation guards in
doPublish (post-fetch and pre-publish) stop a stale generation signing or
publishing after a newer edit exists.

Stars/mutes: scope mergeApplyingRemote (remote-wins on entry-tie) and the
pending-publish cancel to fire only on a canonical supersession of an
already-applied same-timestamp larger-id head. Every other application
(bootstrap/live/newer-timestamp) keeps local-wins mergeStores and does not
cancel the pending publish, so a later same-second local click is no longer
clobbered by an older remote entry that decrypts late.

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
…biguous-ACK heads

Round-4 client-side convergence fixes for channel-sections/stars/mutes sync,
closing two silent edit-loss variants that survived publish serialization.

Ambiguous-ACK fold: a publish whose ACK is lost may still have been accepted
by the relay. Retain each attempt's signed id; when a later cycle's pre-publish
fetch returns a head whose id matches a prior attempt, fold it forward as our
own accepted predecessor and publish above it instead of adopting it away and
erasing the queued edit. A head the relay never accepted can never surface by
id, so the fold is proof-gated on an exact id match.

Canonical-supersession dirty overlay (stars + mutes): a lower-id canonical
correction that arrives after a same-second local click must not clobber the
click. Apply the correction to the prior remote layer, then overlay entries
changed locally since that layer; never cancel a pending publish merely because
a correction arrived.

Client-only: loss discovery relies on the existing pre-publish fetch, live
subscription, and reconcile loop rather than a relay conflict signal.

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Replace the LWW register plus ownership/dirty-set/canonical-supersession
machinery with a per-entry Lamport `rev` and a single max-merge on every
path, mirroring the read-state data model. Each entry carries an additive
optional `rev` (missing implies 0; payload stays `version: 1` so older
builds keep parsing our blobs). One `mergeStores` orders by
updatedAt then rev then the starred/muted-true leaf, and ends in the
500-entry bound.

Clicks stamp `updatedAt = max(now, localEntry?.updatedAt ?? 0,
maxUpdatedAtSeen(id))` and mint `rev = max(localEntry.rev, maxRevSeen(id))
+ 1`, so a click strictly dominates every state its replica has observed
and cannot lose to a same-second remote. The sync managers hold a
per-channel two-field high-water map fed by a single `observe()` on every
ingest path. Stars sync keeps the generation-CAS + single-flight lane and
bounded-backoff retry plus a durable outbox so an in-flight publish can
never clear a newer pending edit; mutes sync mirrors it. Remote ingestion
never touches the pending lane. Deleted: mergeApplyingRemote,
mergeStoresWithTie, mergeCanonicalSupersession, dirtyChannelIds,
lastAppliedRemoteTs/lastAppliedEventId, and the event-clock branch.

Sections and sort are untouched.

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
… contract

The hard-eviction-branch test asserted only the rev tuple outcome on two
one-entry stores while its comment claimed the >500-entry eviction/remount
setup. Build the real fixture: >500 equal-updatedAt entries so the bound
evicts the target by the id tiebreak, then a remounted rev-1 click merged
against the retained rev-100 remote, asserting the deterministic rev-100
outcome in both merge orders. Stars and mutes.

The unobserved-future mixed-fleet residual was stated but never exercised
directly: the fast-clock suites cover only the observed-future fix. Add a
click with an empty high-water at t, then a genuinely unobserved
opposite-value head at t+300 that wins on the primary updatedAt key. Both
hook suites.

Test-only; no production source changes.

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>

@wesbillman wesbillman left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Carl, an automated reviewer, commenting via Wes’s GitHub account.

Requesting changes at exact head 0d797b550e405eea9557e053f884533ad3dd891a. The relay ordering and the per-entry max-merge are coherent, but the lifecycle still has reproducible edit-loss paths.

[P1] The durable outboxes are not safe across desktop windows

Sections, stars, and mutes share one localStorage outbox key per identity + relay, but each window owns only an in-memory generation. A write in one window therefore replaces another window's pending payload, and a completion in either window removes the shared key without proving that it still owns the persisted value.

For sections, window A can start publishing edit A, window B can replace the outbox with newer edit B, and A's ACK then clears B's outbox. If B closes before its debounce fires, the next bootstrap applies relay head A before discovering there is no outbox, so B is permanently lost. See channelSectionsStorage.ts:211-245, channelSectionsSync.ts:262-281,284-313,522, and useChannelSections.ts:138-156.

Stars and mutes can lose independent clicks even earlier: two windows starting from the same store can click different channels before receiving each other's asynchronous storage event. Their whole-store main/outbox writes race, teardown cancels both timers, and remount resumes only the last blob. The storage handler merges only into React state; it does not durably merge the main store or outbox. See channelStarsStorage.ts:206-242, useChannelStars.ts:57-72,95-113, and channelStarsSync.ts:256-260,383-393 (mutes mirror these paths).

Please give persisted attempts cross-window ownership, not only manager-local generations. A per-operation outbox whose owner deletes only its own record, or an actual cross-window serialization mechanism, would close both overwrite and stale-clear races. Add multi-window tests that interleave write, ACK, storage delivery, teardown, and remount.

[P1] Sort preferences still drop pending edits on ordinary lifecycle and failure paths

publishSortPrefs keeps intent only in memory (channelSortSync.ts:113-121). destroy() deliberately cancels and discards it (:252-262), while a failed publish only logs and never retries (:167-210). A live remote also cancels the debounce while leaving pendingStore stranded (useChannelSortPreference.ts:82-103).

Reproduction: change a sort mode and quit/switch communities within two seconds, or let one publish time out and remount. Bootstrap then whole-blob-replaces the local cache with the relay head (useChannelSortPreference.ts:108-122), visibly reverting the user's choice. Please carry the durable outbox, generation ownership, serialized retry, and pending-aware remote application used by sections over to sort, with lifecycle tests.

[P2] Future relay heads can wedge stars, mutes, and sort publishing

Sections clamps created_at inside the relay's future-drift window (channelSectionsSync.ts:463-472), but stars (channelStarsSync.ts:298-301), mutes (matching code), and sort (channelSortSync.ts:183-186) stamp lastRemoteCreatedAt + 1 without a cap. If a self-authored head was accepted near the relay's +900s boundary, a correctly clocked second device emits +901s; the relay rejects it, and retries continue deriving from the same head until wall time catches up. Apply the same bounded timestamp rule across all four sidebar sync surfaces.

CI is green at this head. I did not duplicate the CI-equivalent suite locally; these failures are source-reproduced interleavings absent from the current tests.

@wpfleger96
wpfleger96 force-pushed the wpfleger/channel-sections-sync-fixes branch from 0d797b5 to 16aeab6 Compare August 26, 2026 14:55

@wesbillman wesbillman left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Carl, an automated reviewer, commenting via Wes’s GitHub account.

Re-reviewed exact current head 16aeab68b094baaba508a0f2fc91c06b0c430c1c against base ef0d2025683869418e8eee22ac5b5ac16c5198b7 after the force-rebase. Changes are still required.

All 19 files in this PR have the same Git blob IDs as at previously reviewed head 0d797b550e405eea9557e053f884533ad3dd891a, including all production sync files and tests. The only sidebar differences between the old and new repository trees are three UI files inherited from the new base; they do not touch the NIP-78 sync producers, consumers, or their direct dependencies. Consequently, the previous review's blockers survive unchanged:

  1. P1: sections/stars/mutes still lack cross-window ownership for the shared durable outboxes. Manager-local generation fencing cannot stop one desktop window from overwriting another window's persisted payload or from unconditionally clearing the newer payload after its own older ACK/no-op/adopt completion. The existing tests remain single-manager and do not cover two windows sharing the key.
  2. P1: sort still drops pending intent. It still has only an in-memory pendingStore, no durable outbox or failure retry, destroy() still cancels and nulls the edit, and a live remote can still cancel the timer while leaving the intent stranded.
  3. P2: stars, mutes, and sort still derive created_at = max(now, lastRemoteCreatedAt + 1) without the future-drift clamp used by sections. A self-authored head accepted near the relay's +900s limit can therefore wedge subsequent writes until wall time catches up.

Please address the concrete reproductions and repair boundaries in the review on 0d797b550: #6525 (review)

Current CI is not green: Desktop Smoke E2E (2) failed at this head while several desktop jobs remain in progress. I am not using that still-unclassified failure as a separate code finding; the source blockers above independently require changes.

Duncan and others added 2 commits August 26, 2026 11:35
Carl's re-review found three edit-loss paths remaining after the
stars/mutes rev-merge landed.

[P2] Stars, mutes, and sort stamped createdAt = lastRemoteCreatedAt + 1
uncapped, so a self-authored head accepted near the relay's +900s drift
boundary wedged every later publish until wall time caught up. Extract
the sections clamp into a shared clampPublishCreatedAt in
sidebarSyncWatermark.ts (all four surfaces already import that module)
and wire sections/stars/mutes/sort to it.

[P1] Sort preferences never got the durable lane: doPublish only logged
on failure, destroy() discarded the pending edit, and a live remote
cancelled the debounce with the edit stranded. Port the sections lane —
durable outbox + bootstrap resume, generation/CAS ownership, single-
flight + completion re-drive, 2s->30s backoff, 60s reconcile loop, and
pending-aware remote application. Sort stays whole-blob LWW; a lost head
is adopted at pre-publish rather than republished.

Tests: a clamp test per surface, and sort lifecycle coverage — outbox
resume after destroy-inside-debounce, retry without a later edit,
overlapping-generation safety, live-remote-during-debounce adopt, and a
hook-level pending-defer test.

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Sections, stars, mutes, and sort each persist an unpublished edit under one
localStorage outbox key per identity+relay, but generation ownership is only
in-memory per window. Without cross-window ownership, one window's completing
publish could clear a peer window's still-unpublished edit, and (on the merge
lanes) a peer's write could overwrite an edit before it published.

Every outbox write now mints an ownership token and stores a {store, token}
envelope; a completing publish compare-and-clears only when the stored token
still matches its own, so a peer's newer write survives an older window's ACK.
Stars and mutes additionally read-merge-write both the durable outbox and the
main store via their per-entry mergeStores, so two windows editing different
channels both survive; sort and sections replace whole-blob with LWW resolution
matching the relay. Sort also gains the durable outbox + bounded retry it
previously lacked, and stars/mutes/sort now clamp publish created_at inside the
relay's future-drift window like sections. The envelope reader tolerates a
legacy token-less entry so an outbox written by a prior build still resumes.

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>

@wesbillman wesbillman left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Carl, an automated reviewer, commenting via Wes’s GitHub account.

Requesting changes at exact head 9141fe292268e2681e65664bbeb708b732af2535.

[P1] The multi-window outbox operations are still racy

The new ownership token does not make the localStorage operations atomic. clearOutboxEntry reads and validates the stored token, then calls removeItem separately (sidebarSyncWatermark.ts:141-162). Window A can read token A, window B can write its newer {store, tokenB} envelope, and A can then remove B's value using the stale read. That still loses an unpublished edit on quit/remount.

Stars and mutes have the matching write-side race: writeChannelStarsOutbox reads the shared entry, merges in memory, then writes separately (channelStarsStorage.ts:226-237; mutes mirrors it). If two windows read before either writes, each computes a one-sided merge and the later setItem drops the other pending click. The main-store read/merge/write has the same shape (channelStarsStorage.ts:135-144).

The added multi-window tests execute whole operations sequentially, such as A write, B write, then A clear (multiWindowOutbox.test.mjs:62-167), so they cannot exercise either read/write or read/remove interleaving. localStorage provides no compare-and-delete or transactional read-modify-write primitive; the token proves what was read, not what is still stored at the destructive operation.

Please move this durability boundary to a design that does not depend on atomicity localStorage lacks, such as per-operation append-only records with owner-specific deletion, or an appropriately serialized cross-window store. Add tests that pause operations between their read and write/remove steps and verify teardown/remount preserves every unpublished intent.

The LWW comparator itself matches the relay's created_at DESC, id ASC rule; this review is blocking only on the durability race above. I reviewed read-only GitHub metadata and diff and did not check out or execute PR code.

localStorage has no atomic compare-and-delete or transactional
read-modify-write, so a single outbox key shared across every window
could not be mutated safely: one window's read-then-write or
read-then-remove races a peer's write in the gap and drops its
still-unpublished edit. A per-write ownership token narrowed that window
but could not close it — the token proves what was read, not what is
still stored at the destructive op.

Key the outbox per window instead: <prefix>:<pubkey>:<relay>:<nonce>,
where the nonce is minted once and parked in sessionStorage. Each window
is the sole writer of its own key, so a hot-path write is one
unconditional setItem — the write race is designed out, not guarded.
Resume enumerates every window's key: merge lanes (stars/mutes) fold all
records order-independently; whole-blob lanes (sort/sections) replay the
max-queuedAt record with a nonce tiebreak. Redundant foreign keys are
reclaimed at boot, gated on durable relay evidence (merge: head subsumes;
whole-blob: head created_at supersedes) and re-read immediately before
removal so a live peer's fresh write in the recheck gap survives.
Reclamation runs only on a successful head fetch, never on a failed one.
The token contract is deleted entirely — ownership is the key.

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
…ce-free

The per-window token/recheck reclamation still performed a non-atomic
compare-then-delete on a mutable foreign key: a live owner could rewrite
its key between the reclaim decision-read and the removeItem, and the
whole-blob `queuedAt <= head.created_at` gate dropped same-second and
legacy `queuedAt=0` records that had not provably lost LWW.

Records are now write-once: a key is `<prefix>:<pubkey>:<relay>:<nonce>:<seq>`
and is never rewritten. A new edit writes a new key (next zero-padded seq)
then deletes its own older keys (write-before-delete, so a crash leaves at
least one record). Foreign reclamation reads an immutable record, proves it
reclaimable against durable relay evidence, and deletes it with no recheck.
Whole-blob supersession is strict (`queuedAt < head.created_at`); replay runs
before reclamation in every hook so a same-second record is consumed into
pending first; the legacy v1 shared key is only ever replayed, never deleted.

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
The legacy v1 shared outbox key is never deleted (it is mutable and a
concurrently-live old build may still rewrite it), so the whole-blob
resume path re-read it on every boot and republished the stale blob above
the current relay head forever. A found relay head never stopped it: a
fresh manager has no lastPublishedStore and queueing the replay freezes
publishBaseline to the just-fetched head.

Distinguish compatibility replay from permanently pending intent with a
durable per-value consumption marker, whole-blob lanes only. resumeWholeBlobOutbox
excludes the legacy record when its exact raw matches the stored marker;
a live old build rewriting the key stores a different raw and is replayed
again. The hook transfers the intent into its own v2 key (synchronous
publish) BEFORE writing the marker, so a crash in that gap replays the
blob once more rather than losing it.

Merge lanes (stars/mutes) need no marker but gained a head-subsumed gate
so a lingering legacy key does not re-drive an identical boot-time publish.

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>

@wesbillman wesbillman left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Carl, an automated reviewer, commenting via Wes’s GitHub account.

Requesting changes at exact head 3712e6fb2c71f03587a476c36081b1ac831a60c7 against base ef0d2025683869418e8eee22ac5b5ac16c5198b7.

Product direction: the relay is authoritative, while sidebar sections/sort use whole-blob LWW and stars/mutes use per-entry max-merge. The compatibility contract is that all four lanes converge across concurrently running builds/windows without dropping an unpublished user edit. I reviewed the four surfaces across bootstrap, live delivery, storage events, reconnect, periodic/visibility reconciliation, local edits, publish failure/ACK, destroy, remount, and community switch. I found these blockers:

[P1] Preserve the original whole-blob baseline when reconnect wakes a pending edit

useChannelSections.ts:264-276 and useChannelSortPreference.ts:257-269 fetch on reconnect and then call the public publish*() method for an already-pending edit. Those methods increment pendingGeneration and reset publishBaseline to the now-current lastRemoteHead (channelSectionsSync.ts:277-295, channelSortSync.ts:273-289).

Reproduction: window A queues an edit against H0; window B publishes H1 before A reconnects. A's reconnect fetch records H1, and applyRemote correctly defers because A is pending. The reconnect handler then re-queues A's old store, replacing its frozen H0 baseline with H1. The pre-publish fetch sees equality rather than remoteAdvancedSince, so A publishes above H1 instead of adopting the remote winner. This reverses the intended LWW decision on both whole-blob lanes.

Wake/retry the existing generation without resetting its baseline, and cover remote advancement plus reconnect while an edit is pending for both sections and sort.

[P1] Do not clear a merge-lane outbox merely because its EVENT received OK

channelStarsSync.ts:279-330 and channelMutesSync.ts:279-330 clear their own durable outbox after publishEvent resolves. But the relay returns accepted OK for a superseded NIP-33 write as a no-op, and two windows can also prefetch the same head and publish different whole blobs at the same second. Only one blob is retained.

The shared cache does not close this race: writeChannelStarsStore / writeChannelMutesStore at storage lines 140-151 overwrite the shared key without reading it, while the storage handlers at useChannelStars.ts:67-74 and useChannelMutes.ts:67-74 ignore e.newValue and reread whichever snapshot currently occupies that key. Two windows can therefore create different-channel snapshots from the same base, overwrite the cache before either storage event is processed, and independently receive successful OKs. The non-retained window then deletes the only durable record of its click. The relay, cache, and all outboxes can end with that edit absent.

After publication, clear an own outbox only after an authoritative retained-head fetch proves that head subsumes the attempted store; otherwise merge and retry. Add a two-window test where distinct edits race from the same head and both OK paths complete in either NIP-33 order.

[P1] A later old-build click can lose forever to an earlier new-build rev in the same second

channelStarsStorage.ts:59-77,188-195 and the identical mutes code normalize a missing rev from an old build to 0, then rank updatedAt before rev. If a new build publishes {updatedAt:T, rev:1, starred:true}, an old build observes it and the user unstars during the same second T, that later old-build payload has no rev and parses as rev:0. New builds deterministically retain the earlier true/rev:1; the user's later action cannot win until another click occurs in a later second.

Keeping payload version: 1 allows parsing but does not make the causal model backward-compatible. Use an explicit migration/version strategy that handles old writers, or define and test a product-accepted mixed-fleet limitation rather than claiming observed later intent cannot be lost.

[P1] An unreadable current head must block whole-blob overwrite, not fall through to publish

channelSectionsSync.ts:353-359 and channelSortSync.ts:341-347 record an existing head, but if decryption or parsing fails they return publish. A pending local blob is then signed at lastRemoteCreatedAt + 1 and replaces the only current head, even though the client could not inspect it or determine whether it should be adopted. A future schema, transient keychain/decrypt fault, or malformed payload therefore becomes destructive data loss on the next local edit.

Treat an unreadable/future head as a failed precondition: retain the durable pending edit and retry/surface the incompatibility, but do not overwrite unknown authoritative state. Add pre-publish coverage for decryption failure and unsupported payload version in sections and sort.

…sidual

Close the pass-3 review blocker and Carl's four P1s on the channel
sections/sort/stars/mutes relay sync, all in the sidebar lib:

- Gate the legacy-consumed marker on a proven v2 transfer. writeOwnOutbox
  now returns whether the fresh key's setItem succeeded; the whole-blob
  hooks write the marker only when durable, so a quota failure leaves the
  legacy record replayable instead of silently suppressing the only copy.
- Reconnect wakes the existing pending edit (retryPendingPublish) rather
  than re-queueing via publish*(), which reset the frozen baseline and
  published a stale edit over an advanced remote.
- Merge lanes clear an own outbox only after an authoritative retained-head
  fetch proves the head subsumes the attempted store; a bare EVENT OK on a
  superseded NIP-33 write no longer drops the loser's durable click.
- An unreadable/unsupported pre-publish head retains the pending edit and
  retries rather than overwriting state the client could not inspect.
- Document the accepted mixed-fleet residual (a same-second old-build click
  reads rev 0 and loses to an earlier new-build rev, healing on the next
  later-second click) and pin it with a test; no protocol change.

Regressions added for each item, mutation-verified where the fix lives in
the hook.

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>

@wesbillman wesbillman left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Carl, an automated reviewer, commenting via Wes’s GitHub account.

REQUEST CHANGES on exact head 6078943f3be3a3fea99e86d225b79246af53f599 against ef0d2025683869418e8eee22ac5b5ac16c5198b7. I reviewed GitHub metadata, diff, and exact-head source only; I did not execute PR code.

Product contract: the relay is authoritative. Sections/sort converge as whole-blob LWW; stars/mutes converge by per-entry max-merge. A client must not overwrite relay state it could not read, because that turns a transient fetch/decrypt failure into durable cross-device data loss.

P1 — Retain edits when the authoritative pre-publish read fails

All four lanes fall through to publishing local state when the pre-publish relay query throws. Sections does so at channelSectionsSync.ts:371-416 and sort has the matching branch; stars does so at channelStarsSync.ts:210-232 and mutes mirrors it. The client therefore cannot distinguish “no head exists” from timeout/auth/socket failure.

Reproduction: device B publishes a newer sections/sort blob or independent star/mute entries after device A’s last observation; A edits while its pre-publish fetch fails. A signs above its stale watermark and can replace the unseen authoritative head, erasing B’s data. Durable outboxes and retries do not recover content that A never merged.

Return a tri-state publish decision in every lane: only a successful, genuinely absent head may publish the local store directly; fetch failure must retain the outbox and retry. Add causal tests for an unseen newer head plus rejected pre-publish fetch across sections, sort, stars, and mutes.

P1 — Stars and mutes also overwrite an existing but unreadable head

channelStarsSync.ts:220-231 records an event but returns the local store when decryption, JSON parsing, or schema validation fails; doPublish then publishes it (:315-357). Mutes has the identical path. This can destroy entries in a temporarily undecryptable or future-schema head. Bootstrap already classifies this condition as failed, while sections/sort correctly return retain for it.

Give merge-lane pre-publish reads the same retain/retry behavior and cover decrypt failure, malformed JSON, and unsupported schema. A max-merge is safe only after both operands were actually read. Dungeon law is unhelpfully strict about this.

P2 — Reject numeric revisions that cannot advance

channelStarsStorage.ts:59-69 and the matching mutes parser accept any finite non-negative integer rev, including values beyond Number.MAX_SAFE_INTEGER. Local clicks mint maxRev + 1 (useChannelStars.ts:252-267, mirrored for mutes); at sufficiently large IEEE-754 values that no longer increases. With equal timestamps/revisions, true wins (channelStarsStorage.ts:199-207), so a malformed starred:true or muted:true entry can suppress later unstar/unmute attempts indefinitely.

Require safe, bounded integers for revision and timestamp inputs, define malformed-input handling, and add a regression proving a huge revision cannot wedge a later false toggle.

The prior blockers around multi-window ownership, reconnect baselines, unreadable whole-blob heads, publish retention confirmation, and timestamp clamping are materially addressed. Exact-head CI is broadly green; it does not cover the failed-read transitions above.

Extends the retain-on-unreadable pattern to the two remaining overwrite
paths and hardens the rev/timestamp parse boundary.

- All four lanes tri-state the pre-publish read: a THROWN fetch (timeout /
  auth / socket) now retains and retries instead of falling through to
  publish, so an edit during a transient outage can no longer sign above a
  stale watermark and erase an unseen newer head.
- The merge lanes (stars/mutes) retain when the existing head fails
  decryption/JSON/schema parsing rather than publishing the local store
  over it — a max-merge is only safe once both operands were read, matching
  the whole-blob lanes.
- The rev/timestamp parsers require Number.isSafeInteger: an unsafe rev is
  normalized to 0 and an unsafe timestamp entry is dropped, so a malformed
  huge-rev true entry can no longer wedge later false toggles forever.

Retain reuses the existing bounded-backoff retry and generation CAS, so a
later readable/successful head resumes normal resolution. Regressions cover
all four lanes for the failed-fetch and unreadable-head paths and the
rev-wedge P2 case in both storage suites.

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
…adroom

The rev parse bound used Number.isSafeInteger, which accepts
Number.MAX_SAFE_INTEGER itself. The click path mints rev = maxRev + 1,
so an accepted boundary rev overflows to an unsafe value that never
advances again — recreating the same-second toggle wedge the bound
exists to prevent. Reject rev >= Number.MAX_SAFE_INTEGER (exclusive,
normalize to 0) so maxRev + 1 always stays safe. updatedAt keeps its
inclusive bound: the click path takes Math.max(now, observed), never
increments, so it cannot overflow.

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
wholeBlobSyncManager: no-op guard now compares only against the freshly
fetched readable head (decision.fetchedRemote.store). Absent, failed,
unreadable, and foreign-author pre-publish paths can never no-op; the
stale lastPublishedStore comparison is removed entirely.

useChannelMutes/useChannelStars: comment-only, removing obsolete field
reference that no longer exists.

Test consolidation (22 files, 2,570 ins / 3,008 del):
- 17 changed test files: 4,458 lines; all 24 head sidebar lib test
  files: 4,786 lines; caller files 6 lines each
- Helper sidebarSyncTestHelpers.mjs: 487 lines; sibling
  sidebarAdapterContractTestHelpers.mjs (new): 587 lines; both under
  the 1,000-line repo ceiling (AGENTS.md:590)
- Whole-surface total (24 tests + 2 helpers): 5,860 lines (was 6,303,
  base 23 tests + 1 helper; -443)
- 5,518/5,518 desktop tests passing at this commit; no new biome
  warnings in changed files (pre-existing: channelMutesStorage
  .test.mjs:115-116 useLiteralKeys, SetupStep.acpForcedGate.test.mjs
  :423 useOptionalChain, wholeBlobSync.shared.test.mjs:554
  noUnusedFunctionParameters, terminal.css:276 noImportantStyles)

Adapter-contract refactor: _runAdapterContract (15-knob generic)
deleted; runSectionsAdapterContract and runSortAdapterContract are
genuinely concrete, each using typed lane APIs and lane-local constants
inline. Repeated test infrastructure (assert, test, mock, relayClient,
relay URL) hoisted into _adapterCtx() cache + ADAPTER_RELAY constant;
phases 2/3 use _runDurableResumeTest / _runUnsupportedVersionTest with
5 lane-specific params each (laneLabel, Manager, publishTo, readOutbox/
badVersionPayload, makeStore). Adapter section split into
sidebarAdapterContractTestHelpers.mjs (587 lines) to satisfy the
1,000-line ceiling; re-exported from main helper so callers are
unaffected. grep confirms zero old descriptor knob names in either file.

Test coverage added:
- wholeBlobSync: S1->S2->S1 folded lifecycle - S2 delivery observed via
  subscription callback; S1 pending and second durable outbox write
  asserted before confirmation; clear on exact retained-ACK only
- mergeLaneHook: click-before-opposite-peer-arrival with causal mutations
  (a) drop listener -> idsField hook-membership assertion fails; (b)
  suppress fetchRemote* inside tick -> fetch-count assertion fails; (c)
  drop fetched result -> stored entry stays at arrival tuple (900,12)
  not relay head (950,13)
- Adapter runner (both lanes): opposite-lane sentinel deep-equal before
  own-outbox-cleared; fetch and live filters exact-object deep-equal;
  fetched and callback stores exact deep-equal (sections includes
  assignments; sort has groups.p === 'recent'); unsupported-version
  manager.destroy() in finally

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
@wpfleger96 wpfleger96 changed the title fix(sync): converge channel sidebar state across devices on the same identity fix(sidebar): correct no-op guard; consolidate sync tests Sep 1, 2026
@wpfleger96 wpfleger96 changed the title fix(sidebar): correct no-op guard; consolidate sync tests fix(sidebar): converge channel sidebar state across devices Sep 1, 2026

@wesbillman wesbillman left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Carl, an automated reviewer, commenting via Wes’s GitHub account.

REVIEW CLEAR: prior production blocker resolved

Reviewed head e3624ed22f826a83eb1be666e7a629ec1d3a94c3 against base ef0d2025683869418e8eee22ac5b5ac16c5198b7, focusing on the correction since 0f98440bea7dea8f73c78992e1fcfb4e7b6c2870. No remaining actionable production blocker found in this corrective review. This is a comment, not an approval.

  • The stale no-op defect is fixed. lastPublishedStore is removed. The preflight carries its freshly fetched readable head, and the no-op branch compares only against that head. Therefore S1 published → S2 observed → explicit S1 publishes again rather than silently dropping the edit. Missing heads cannot prove a no-op; failed/unreadable reads retain the durable pending edit. Generation checks and retained-head confirmation remain intact.
  • Nonblocking test follow-up: the causal S1→S2→S1 witness exercises the production shared engine using SyntheticWholeBlobManager, not both concrete adapters. Its ID-only synthetic equality and outbox counters do not prove same-ID section rename/assignment restoration, sort-mode restoration, or actual persisted contents through delayed confirmation. Add that scenario through both concrete managers with their real outboxes. The current witness catches the original engine defect; the unchanged concrete equality functions compare the relevant fields correctly, so this coverage limitation is not evidence of another production P2. The merge-hook retention phase now drives actual timers; the earlier click-before-older-opposite-live recommendation remains nonblocking. Restore the deterministic reclamation read→delete-gap witness as well: the new stale/fresh-key coexistence case no longer injects an arrival during reclamation, although the unchanged implementation still deletes only the proven immutable key.
  • Scope and validation: retained the established sections/sort whole-blob LWW and stars/mutes per-entry merge contract, scoped outbox ownership, community/signer teardown, and accepted compatibility residuals. Rechecked current adapters/consumers and protocol premises against prior integrated coverage; reviewed the consolidated tests with independent bounded lanes. Read-only exact-ref source review only: no checkout, build, test execution, or live multi-device validation. Native/runtime behavior and other client implementations are not newly validated here.

@kalvinnchau kalvinnchau left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes requested

Reviewed exact head e3624ed22f826a83eb1be666e7a629ec1d3a94c3 against base ef0d2025683869418e8eee22ac5b5ac16c5198b7.

[P1] Stale whole-blob outboxes are replayed above newer relay state — desktop/src/features/sidebar/lib/useChannelSections.ts:164, desktop/src/features/sidebar/lib/useChannelSortPreference.ts:161

On bootstrap, both hooks apply the fetched relay head and then unconditionally pass the selected outbox record to publishSections / publishSortPrefs. That call freezes the just-fetched head as its baseline, and the manager stamps the replay above it.

A reachable sequence is: window A queues an edit at t=100 and closes during the debounce; another device publishes a newer head at t=200; A later opens. The t=100 outbox is promoted to t=201 and overwrites the newer state, even though the reclamation contract immediately below says records with queuedAt < head.createdAt lost whole-blob LWW. Reclamation runs after replay and never touches this window's newly copied own key, so it cannot prevent the overwrite.

Preserve queuedAt in the resume result and, when bootstrap fetched a head, replay only records that are not strictly older; clear/reclaim a stale selected record instead. Add this sequence to both whole-blob lane tests.

[P2] The whole-blob baseline is not the state the user edited — desktop/src/features/sidebar/lib/wholeBlobSyncManager.ts:421-433

publish() freezes lastRemoteHead, but that value means “head observed by the manager,” not “head incorporated into React/localStorage.” This loses data in both directions:

  • If bootstrap failed or is still unresolved, lastRemoteHead is {0, ""}. A sections/sort edit made after mount freezes that empty baseline. When the pre-publish fetch later finds an older, pre-existing relay head, remoteAdvancedSince() classifies it as post-edit, adoptRemote() clears the pending store/outbox, and the fresh click is never published.
  • A live callback does the reverse at wholeBlobSyncManager.ts:762-770: it records the head before async decrypt/application. A click in that gap freezes the unapplied head as its baseline; the subsequent pre-publish fetch sees equality and publishes a stale whole blob over remote-only changes.

Base arbitration on the exact head successfully applied to the editable store. Queue/replay mutations until bootstrap establishes that base, and do not advance the editable baseline merely because a raw event arrived. Add deterministic coverage for both deferred/failed bootstrap and the live decrypt/application gap.

[P2] The pre-publish merge can evict the click that the hook explicitly preserved — desktop/src/features/sidebar/lib/mergeLaneSyncManager.ts:293-299

Stars/mutes preserve the clicked channel while locally bounding to 500 entries (useChannelStars.ts:269-288, mirrored by mutes). The manager then calls mergeWithRemote() without that preserved key. mergeStores() re-bounds at channelStarsStorage.ts:238-253 using only updatedAt and channel ID for eviction.

With 500 remote entries in the same second plus a lexicographically smaller clicked channel, the hook keeps the click, the pre-publish merge drops it despite its higher rev, and retained-head confirmation clears the outbox. Carry the mutated key through the publish merge, or make eviction ordering preserve the newest mutation tuple. Add a 501-entry regression for both stars and mutes.

Assessment

  • Minimalness: 6/10 — shared engines help, but 8,501 additions plus four duplicated hook orchestration loops create a large change surface.
  • Elegance: 7/10 — the shared lane split is directionally sound, but the abstractions omit two load-bearing facts: which remote head the UI actually contains and which entry must survive capacity bounding.
  • Correctness: 6/10 — the reproduced paths silently discard user intent, violating the core contract of this change.

Validation

  • Desktop unit suite: 5,518/5,518 passed at the exact head.
  • Desktop typecheck: passed at the exact head.
  • GitHub Desktop, Unit Tests, and Desktop E2E checks are green.
  • The findings are uncovered ordering/capacity cases rather than failures in the existing suite.

Duncan and others added 10 commits September 2, 2026 13:31
… baseline split, P3 preservedKey through mergeWithRemote

P1: Gate bootstrap outbox replay on queuedAt > appliedHead.createdAt in
both whole-blob hooks (useChannelSections, useChannelSortPreference).
Stale records (queuedAt <= head.createdAt) are left for reclaimSuperseded*
cleanup. Legacy records (queuedAt=0) under a non-zero head are skipped.
Exposes queuedAt from resumeWholeBlobOutbox and the two outbox read helpers.

P2b: Split subscribeLive in wholeBlobSyncManager — watermark+lastRemoteCreatedAt
advance stays synchronous (preserves undecryptable-event seed-publish guard),
lastRemoteHead tuple update moved into the .then() after successful decrypt.
Prevents publishing pre-decrypt content over remote-only changes.

P3: Thread preservedKey through publish(store, preservedKey?) ->
pendingPreservedKey field -> fetchOwnBlobBeforePublish -> mergeWithRemote
in MergeLaneSyncManager. Both channelStarsSync/channelMutesSync wrappers
and both useChannelStars/useChannelMutes click handlers updated to forward
the channelId as preservedKey.

Regression tests:
- wholeBlobSync.shared.test.mjs: P1 stale-replay (manager level, both lanes
  via SyntheticWholeBlobManager) + P2b decrypt-gap sequence
- wholeBlobHook.shared.test.mjs: P1 stale outbox not replayed test added to
  runWholeBlobHookSuite; useChannelSections + useChannelSortPreference opt in
- channelStarsSync.test.mjs: P3 501-entry merge with preservedKey retains
  the clicked channel

P2a (publish-baseline contract for failed bootstrap) held pending Will's
ruling on Option A (narrow guard) vs Option B (queue-until-bootstrap).

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
The P2b decrypt-gap test assumed fetchEvents would be called twice
(once for bootstrap, once for pre-publish fetch), but the test never
calls bootstrap(). The first and only fetchEvents call goes to
fetchOwnBlobBeforePublish; returning [] there produces a 'publish'
decision and the test fails.

Remove the incorrect two-branch mock. The single fetchEvents call is
the pre-publish fetch; it returns the live event as decryptable
(good-cipher) so fetchOwnBlobBeforePublish sees remoteAdvancedSince
true → adopt, verifying the P2b fix.

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
…reservation; causal tests

P1 guard shape fix (both whole-blob hooks):
- Was: reduce all non-apply-remote paths to headCreatedAt=0, gate on queuedAt>0
- Now: branch on bootstrap result explicitly
  - hold/absent/failed (result.action !== 'apply-remote'): always replay,
    including legacy queuedAt=0 records (old 0>0 guard silently stranded them)
  - apply-remote: suppress only on strict queuedAt < appliedHead.createdAt;
    equality replays (aligns with reclamation's strict-lt, prevents stranding
    a same-second edit neither consumed nor published)
- Documents clock-skew residual: queuedAt vs relay created_at are independent
  wall clocks — accepted tradeoff of the existing LWW max-merge strategy

P3 reconnect preservation (MergeLaneSyncManager + both stars/mutes hooks):
- Add retryReconnectPublish() to MergeLaneSyncManager: re-drives the existing
  generation without calling publish(), so pendingPreservedKey is not reset
- Expose retryReconnectStarsPublish() / retryReconnectMutesPublish() on adapters
- Switch both hooks' reconnect effects to retryReconnectPublish() instead of
  publishStars/Mutes(pending), which would open a new generation and clear key
- Clear pendingPreservedKey in destroy()

P3 remount preservation (storage layer + both hooks):
- writeOwnOutbox: accept optional preservedKey, include in envelope JSON
- parseEnvelope: read preservedKey from envelope; OutboxRecord gains field
- enumerateOutbox: propagate preservedKey into OutboxRecord
- mergeLaneStorage.shared: writeOutbox forwards preservedKey; add
  readOutboxPreservedKey to read own-window record's key for bootstrap
- channelStarsStorage / channelMutesStorage: accept+forward preservedKey in
  write, expose readChannelStarsOutboxPreservedKey / readChannelMutesOutboxPreservedKey
- MergeLaneSyncManager: writeOutbox config type includes preservedKey param;
  publish() forwards it to config.writeOutbox
- useChannelStars / useChannelMutes: import and call readPreservedKey on
  bootstrap outbox replay; pass to publishStars/Mutes alongside the store

Causal test coverage:
- wholeBlobHook.shared: replace non-causal stale-replay test with four-case
  P1 regression using makeHookTimerBed.hasDelay(2000) as the synchronous
  causal signal (timer present iff publish() called): strict-lt suppresses,
  equality replays, hold+v2 replays, hold+legacy-queuedAt=0 replays
- channelStarsSync.test: add P3 reconnect (retryReconnectStarsPublish keeps
  key) and P3 remount (storage round-trip write+read preservedKey)
- channelMutesSync.test: add matching P3 501-entry, reconnect, and remount
  regressions parameterizing coverage across both lanes

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
…rvedKey from foreign records; add P3 reconnect+restart hook regressions

Thufir pass-2 findings — all four corrections applied:

1. P3 restart durability: replace two-function readOutbox+readOutboxPreservedKey
   with readOutboxWithMeta<S> that enumerates records once and returns
   { store, preservedKey }. Key selected by max queuedAt across ALL records (own
   and foreign); after quit, the prior window's nonce is gone so its record is
   foreign — the old isOwn filter returned undefined, losing the reservation.
   Both hooks forward outboxMeta.preservedKey directly to publish* on bootstrap.

2. Two-read snapshot/provenance split closed: the capacity bound now applies
   inside readOutboxWithMeta using the selected preservedKey, so the clicked
   channel is never evicted before it reaches the manager.

3. discardPending() clears pendingPreservedKey: minor correctness fix so a
   confirmed publish resets the field alongside pendingStore and clearOutbox.

4. Test consolidation: remove non-causal reconnect/remount tests from
   channelStarsSync.test.mjs (+120) and channelMutesSync.test.mjs (+192).
   Replace with two parameterized regressions in mergeLaneHook.shared.test.mjs:
   - P3 reconnect: triggers the actual subscribeToReconnects callback registered
     by the hook, asserts clicked channel survives 501-entry merge.
     Failing mutation: revert hook reconnect to publish*(pending).
   - P3 restart: seeds a foreign-nonce outbox envelope with preservedKey,
     mounts hook (bootstrap reads foreign record → recovers key → publishes),
     fires 2s debounce, asserts clicked channel survives 501-entry merge.
     Failing mutation: revert to readOutboxPreservedKey (isOwn filter) or drop
     bootstrap preservedKey forward.

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
…uring

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
… drives merge at doPublish not bootstrap

Bootstrap fetch returns [] (hold) so publishStars is called unconditionally —
the subsumed guard only runs on apply-remote, which would require extra async
ticks to verify. The 501-entry merge still exercises at the production seam:
when the 2s debounce fires and doPublish calls fetchRemoteStars (fetchCalls=2),
the 500-entry remote head triggers mergeWithRemote producing 501 entries. Without
preservedKey forwarded from the foreign record, boundStarStore evicts the clicked
channel there. The test structure now matches the existing bootstrap replay test
(hold bootstrap → timer fires → publish cycle exercises the production path).

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
…ithMeta

Without bestKey in the per-record reduce, folding two durable outbox records
totaling 501 entries evicts the clicked channel before the final bound can
protect it. Thread bestKey through merge(acc, r.store, bestKey) on every step
so the reservation is upheld at each intermediate capacity-bound merge.

Update the merge callback signature from (a, b) to (a, b, preservedKey?) —
both mergeStores implementations already accept the optional third arg.

Also fix two test gaps from Thufir pass 3:

reconnect test: both the reconnect-callback fetch and the subsequent
pre-publish fetch must return the remote 500-entry head so that reverting
the hook reconnect to publish*(pending) triggers the 501-entry merge and
fails. The previous single-return-on-call-1 mock left the pre-publish fetch
returning [] — no 501-merge occurred and the wrong wiring stayed green.

restart test: upgrade from one to two foreign-nonce outbox records. Record 1
carries the clicked channel + 499 entries with preservedKey; record 2 carries
the 501st entry with no key. This exercises the per-record fold eviction
(mutation c) directly: without bestKey in the reduce the fold evicts the click
at the record-2 merge, before the final bound runs. The single-record test
could not catch this class of defect.

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
No publish debounce completes until bootstrap resolves. Edits queued
during an unresolved bootstrap store their intent durably (outbox write +
in-memory pending state) but the debounce timer is deferred. On
resolution, releaseDeferred() re-freezes publishBaseline from the
now-established lastRemoteHead before scheduling the timer, so the
pre-publish fetch sees the bootstrap head as 'not advanced' and
publishes above it rather than adopting it away.

Failed-bootstrap corner: when bootstrap fails, lastRemoteHead stays
{0,""} and bootstrapFailed=true is recorded. The empty-baseline
exception in fetchOwnBlobBeforePublish fires only when both flags
indicate a failed bootstrap with no known head, treating the first
fetched head as the new baseline rather than a competing remote.

Three causal regressions in wholeBlobSync.shared.test.mjs:
- P2a test 1 (load-bearing): click during unresolved bootstrap on a
  fresh device publishes above the relay head; failing mutation: remove
  bootstrapStarted guard from publish() → {0,""} baseline → adopt.
- P2a test 2: genuinely newer peer head that arrives after bootstrap
  is adopted via normal LWW (publishBaseline re-frozen to bootstrap
  head, peer head at higher ts is a genuine advance).
- P2a test 3 (Kalvin's original sequence): click after failed bootstrap
  publishes above the relay head; failing mutation: remove bootstrapFailed
  exception → hard-adopt against {0,""} baseline.

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
…e defects

Thufir's focused pass (890de2d) identified three executable loss/overwrite
paths and two test-gate gaps. This commit addresses all five findings.

Production fixes:

1. Absent-bootstrap seed must not clobber a pending edit.
   When bootstrap resolves absent and local state is non-empty, runBootstrap
   calls publishFn(localStore) to seed the relay. If a click was already
   pending (pendingStore !== null), that click IS the seed intent; calling
   publish(localStore) again bumps the generation and overwrites the outbox
   with the stale mount snapshot. Fix: guard the publishFn lambda so
   publish(localStore) is a no-op when a pending edit already exists.

2. releaseDeferred must use the bootstrap-result head, not mutable lastRemoteHead.
   subscribeLive can advance lastRemoteHead with a live peer head while bootstrap
   is in-flight. A live head arriving after the click but before bootstrap
   resolves must remain a genuine remote advance — if releaseDeferred re-freezes
   publishBaseline from lastRemoteHead, the live head folds into the baseline,
   the pre-publish check sees equality, and the edit publishes over the peer.
   Fix: derive bootstrapResultHead from fetchResult directly (not lastRemoteHead),
   and pass it into releaseDeferred. fetchResult carries exactly the head the
   bootstrap operation itself fetched; subscribeLive's passive observations
   are invisible to it.

3. Failed-bootstrap exception must be disarmed once any head is independently
   observed (live or reconnect/periodic fetch).
   bootstrapFailed+empty-baseline correctly identifies the "first unknown base"
   case, but after a transient pre-publish fetch failure and a subsequent live
   peer head arrival, the exception re-fires on retry and treats a genuine peer
   advance as a base-establishment, publishing over it. Fix: add
   bootstrapFailedExternalHeadObserved flag; set it from subscribeLive decode
   completion and from fetchRemoteBlob when it finds a head while
   bootstrapFailed is true. The exception checks !bootstrapFailedExternalHeadObserved.

Test-gate fixes:

- P2a cases moved to a new wholeBlobSyncP2a.shared.test.mjs (449 lines) and
  exported as runWholeBlobP2aSuite; called from both useChannelSections.test.mjs
  and useChannelSortPreference.test.mjs for two-lane coverage. Removes the P2a
  block from wholeBlobSync.shared.test.mjs (841 lines, down from 1080).

- Five causal regressions, each named by its exact production mutation:
  T1: drop bootstrapStarted guard → baseline {0,""} at debounce → adopt fires.
  T2: releaseDeferred uses mutable lastRemoteHead → live head folds into baseline
      → pre-publish sees equality → publish over peer (load-bearing for defect 2).
  T3: drop pendingStore===null guard → seed clobbers edit → stale mount published.
  T4: drop !bootstrapFailedExternalHeadObserved → exception fires after live
      observation → publish over genuine peer advance.
  T5 (Kalvin's original): remove bootstrapFailed exception block → hard-adopt
      against {0,""} → click lost (load-bearing for defect 3).

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
…eue-time baseline

Plain replacement `publishBaseline = { ...bootstrapResultHead }` discarded the
queue-time baseline when bootstrap resolved with an older head. If a live peer
head B fully applied before the user clicked, publish() froze publishBaseline=B,
but releaseDeferred then regressed it to the bootstrap-result head A. The
pre-publish fetch saw B as an advance and adopted the click away (0 publishes,
1 adopt).

Fix: use canonicalMax(this.publishBaseline, bootstrapResultHead), preserving
whichever head is canonically greater. If the click was authored from B and
bootstrap returned older A, canonicalMax keeps B as the baseline and the edit
publishes above B correctly. For a failed bootstrap, bootstrapResultHead is
{0,""} and canonicalMax leaves the queue-time baseline unchanged.

Add T6 regression to the P2a shared suite (runs through both
ChannelSectionSyncManager and ChannelSortSyncManager): blocked bootstrap on A
→ live B applies → click from B → bootstrap resolves A → edit publishes above
B. Failing mutation: revert to plain replacement → baseline regresses B→A →
pre-publish adopts B (0 publishes, 1 adopt).

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>

@wesbillman wesbillman left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Carl, an automated reviewer, commenting via Wes’s GitHub account.

CHANGES REQUESTED

Reviewed exact head 8148ead9cf657e6e489e466644c4b64f8e405083 against base ef0d2025683869418e8eee22ac5b5ac16c5198b7, focusing on the repairs since e3624ed22f826a83eb1be666e7a629ec1d3a94c3. Three unresolved data-loss paths remain in the contracts this revision repairs.

[P1] Failed bootstrap promotes an old outbox into a fresh edit

The new failed-bootstrap exception cannot distinguish a fresh click from a restored outbox. Both hooks replay unconditionally on hold (sort: lines 161–199).

Reproduction: A queues sections/sort at t=100 and quits before debounce; another device retains B at t=200; A reopens, but its bootstrap fetch throws. hold calls publish(A), freezing {0,""} and replacing the outbox's original queuedAt with the current time. The first successful preflight reads B before any live/reconcile observation, so the exception folds B into the baseline and publishes A above it. B's newer layout/preferences are overwritten. The successful-bootstrap age guard never runs in this sequence; retained confirmation then legitimately confirms the wrong replacement.

Repair: retain restored-edit provenance/age through a failed bootstrap and resolve its freshness when a head can actually be read. Do not grant a restored outbox the fresh-click exception or remint its age before that decision. Add failed-bootstrap → recovered newer head coverage through both actual hooks, asserting no stale publication and correct durable-outbox disposition.

[P2] Hook replay and refresh still freeze unapplied heads into the editable baseline

publish() still copies lastRemoteHead, which is not necessarily the head reflected in the editable store. Two concrete paths bypass the new baseline fixes:

  1. Bootstrap is blocked with H100; a click at wall time 101 queues A; live H102 decrypts while A is pending, so applyRemote refuses to apply it. Bootstrap then resolves H100. releaseDeferred correctly retains H100 as the baseline, but the hook immediately reads A from its own outbox and calls publishSections again (sort: 175–189). This resets the baseline to H102. Preflight sees equality and publishes the pre-H102 blob over H102's remote-only changes instead of adopting H102. T2 exercises the concrete manager without this hook callback, so it does not catch the production sequence.
  2. After normal bootstrap, a periodic/reconnect fetch receives B and fetchRemoteBlob records its tuple before awaiting decrypt. A click during that await freezes B while still editing A. The eventual hook application is suppressed by hasPendingEdit; preflight B again sees equality and publishes stale content over B. Moving tuple advancement after decrypt only in subscribeLive leaves this path unchanged.

Repair: preserve the pending generation/baseline when bootstrap finds the current session's already-queued edit, and keep the editable baseline tied to successfully incorporated state across live and fetch paths. Add actual-hook ordering regressions for both sequences, including durable state and the final publish/adopt outcome.

[P2] Capacity-bounded subsumption can certify a click the relay does not contain

The preserved key reaches the publish merge, but isStarsStoreSubsumedBy and the mutes equivalent still prove retention using mergeStores(head, candidate) with no preserved key. That merge bounds to 500 entries before testing equality.

Take a full head R with 500 entries at the same updatedAt. A candidate contains 499 of those entries plus a lexicographically smaller clicked channel X (with a higher rev), and its durable preservedKey is X. R does not contain X. The proof merges 501 entries, evicts X using timestamp/ID ordering, gets R back, and returns true. On restart, the bootstrap subsumption gate skips replay and reclamation deletes the foreign outbox. In a same-second publish collision, retained-head confirmation can likewise clear the own outbox even though the winning head omits X. Both lanes lose the explicitly protected click.

Repair: do not use lossy capacity eviction as evidence that the retained head contains the mutation. Preserve the clicked-entry requirement through bootstrap suppression, reclamation and ACK confirmation, or use an appropriate non-lossy dominance proof. Add found-full-head restart and ACK-but-unretained full-head regressions for stars and mutes. The new restart test intentionally returns an absent bootstrap head (mergeLaneHook.shared.test.mjs:694–704), and the new manager tests return an empty confirmation read, so neither exercises this boundary.

Scope and validation

Read-only exact-ref source/metadata review with independent whole-blob and merge/storage lanes. No checkout, build, test execution or live multi-device validation. Reused and verified unchanged consumer/protocol premises from the prior integrated review: community/identity isolation, all four kind-30078 lane tags, canonical replacement order and ACK-versus-retention semantics. The fresh-readable-head no-op correction and write-once ownership remain intact. Existing mixed-version, clock and general capacity residuals are not reopened; the findings above directly violate this revision's stale-resume, applied-baseline and preserved-click contracts.

P1 — restored-outbox replay re-mints queuedAt and triggers the
failed-bootstrap exception designed only for fresh in-session clicks.
Hook-replayed outbox calls now pass isRestoredReplay=true through
publishSections/publishSortPrefs → publish(), which suppresses the
exception (pendingIsRestoredReplay) and does not re-freeze publishBaseline
from mutable lastRemoteHead.

P2a-1 — hook .then() replay after releaseDeferred establishes the correct
baseline called publish() unconditionally, overwriting the baseline with
the mutable lastRemoteHead (which by then could carry a suppressed live
peer head). The isRestoredReplay flag added for P1 also closes this path:
restored replays skip the publishBaseline re-freeze so the baseline
releaseDeferred established is preserved through the callback.

P2b — fetchRemoteBlob (periodic/reconnect path) called recordRemoteHead
before decryptAndParse, leaving a decrypt-gap window where a concurrent
click could freeze its publishBaseline against the pre-decrypt head.
Moved recordRemoteHead to after decryptAndParse succeeds, mirroring the
existing subscribeLive fix.

P3 — isStarsStoreSubsumedBy / isMutesStoreSubsumedBy proved subsumption
via a capacity-bounded mergeStores(head, candidate) without a preservedKey.
At the 500-entry cap the clicked channel (lexicographically smallest) could
be evicted during the proof merge, certifying retention of a click the relay
never kept. Two consumers affected: bootstrap suppression (hook files pass
outboxMeta.preservedKey to the subsumption check) and ACK confirmation
(mergeLaneSyncManager passes this.pendingPreservedKey to config.isSubsumedBy).
Both storage functions now take an optional preservedKey and use a direct
membership check when present: head must contain the preserved channel with
rev >= candidate's before the rest of the store is verified.

Regressions — hook-layer tests in new shared suites:
- wholeBlobSyncCarl.shared.test.mjs: P1 (failed-bootstrap replay adopts
  newer retained head), P2a-1 (H102 survives hook replay as genuine advance),
  P2b (fetchRemoteBlob decrypt gap). Wired from useChannelSections and
  useChannelSortPreference test files.
- mergeLaneSyncCarl.shared.test.mjs: P3-bootstrap (500-entry head does not
  subsume click via eviction) and P3-ack (confirmRetainedHeadSubsumes retains
  outbox when relay head omits clicked channel). Wired from useChannelStars
  and useChannelMutes test files.

All 47 targeted tests pass (43 pre-existing + 4 new Carl-round P3 tests
plus 6 new P1/P2a-1/P2b tests). T1-T6 manager-level contracts unchanged.

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>

@wesbillman wesbillman left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Carl, an automated reviewer, commenting via Wes’s GitHub account.

Changes requested

Source-only re-review of 31510eabb18207dce052c3fbe3afdf904c01d35c against base ef0d2025683869418e8eee22ac5b5ac16c5198b7, focused on the repairs since 8148ead9cf657e6e489e466644c4b64f8e405083. No PR code was checked out, built, tested, imported, or executed; no runtime/CI pass is claimed.

The exact prior failed-bootstrap stale-replay, bootstrap/replay baseline-refreeze, and fetch/decrypt-gap witnesses are repaired. The preserved-click proof is also repaired for bootstrap suppression and ACK confirmation. Four actionable cases remain:

1. [P1] Successful reopen discards a valid newer sections/sort outbox

wholeBlobSyncManager.ts:530-544, with releaseDeferred at 394-410 and hook replay at useChannelSections.ts:178-195 / useChannelSortPreference.ts:175-192.

Relay holds H100; the user edits A at 101 and reloads before debounce, then makes no new click. Bootstrap successfully finds unchanged H100. Because the new manager has no pendingStore, releaseDeferred() never initializes publishBaseline. The hook correctly accepts A's age, but now calls publish(A, true), preserving {0,""}. Preflight therefore treats H100 as an advance and adopts it (628-680) without publishing A. On same-window reload, the replay transfer replaces the original own outbox, and adoption deletes that replacement too. An ordinary successful resume loses the saved edit.

Establish a bootstrap-result baseline for a genuinely restored, eligible outbox; do not assume a pending generation already established one. Keep post-click live heads out of that baseline. Add actual-hook outcome tests for both sections and sort with a distinct newer outbox, unchanged readable head, fired debounce, and retained publication/outbox assertions, not just a scheduled timer.

2. [P2] A second click still freezes a head the UI never incorporated

wholeBlobSyncManager.ts:530-532, reached from the sections/sort mutation callbacks; observed-head updates are at 292-309 and 908-921, while hook application is suppressed at useChannelSections.ts:115 / useChannelSortPreference.ts:114.

Start with H100 applied, queue A1, then deliver/decrypt live or periodic H102 while A1 is pending. The manager records H102, but applyRemote deliberately keeps A1 visible. A second user edit A2 is authored from A1, not H102. Its normal publish(false) nevertheless freezes lastRemoteHead=H102. Preflight sees equality and publishes A2 over H102-only changes. Generation guards and retained-head confirmation do not prevent that incorrect replacement. Moving observation after decrypt fixes the previous gap, but successfully decrypted still does not mean incorporated.

Tie the editable baseline to the state actually incorporated into the edit (or preserve the pending baseline across edits that still exclude that head). Cover the two-click/suppressed-head sequence through both production hooks.

3. [P2] Failed-bootstrap replay misclassifies an in-session click as restored

useChannelSections.ts:164-195 / useChannelSortPreference.ts:161-192, and wholeBlobSyncManager.ts:533-536,667-680.

With no prior outbox, start bootstrap, click before it resolves, then let bootstrap fail. releaseDeferred schedules the fresh pending edit with its zero baseline. The hook immediately reads the outbox just written by that same click and republishes it with true, relabeling it as restored. If the 2s preflight recovers the pre-existing H100 before the initial reconciliation timer, the new flag disables the intended first-unknown-base exception and adopts H100, deleting the fresh click even though no competing peer wrote anything.

Do not reclassify/requeue the already-pending current-session generation as an old outbox. Preserve its provenance separately from a genuine restore. Test click-before-bootstrap-failure through both hooks and drive publication to completion; a manager-only call or clicking after failure misses this path.

4. [P2] Foreign-outbox reclamation still omits the preserved click from its proof

mergeLaneStorage.shared.ts:254-268, invoked by channelStarsStorage.ts:393-405 / channelMutesStorage.ts:383-395 after hook bootstrap.

Reclamation still calls isSubsumedBy(record.store, head) without record.preservedKey, so it selects the old lossy bounded-merge fallback. Use a full 500-entry head R lacking X and a foreign record containing 499 R entries plus reserved X. All entries have the same updatedAt, X has a higher rev but a lexicographically smaller ID than every R entry, so the unkeyed capacity bound evicts X and returns R, falsely proving subsumption. For a concrete durability failure, let bootstrap replay find this foreign record but fail to write its new own outbox (quota); writeOwnOutbox returns false, but the merge wrapper discards that result. Reclamation then deletes the foreign record even though the relay never retained X and no durable replacement exists. Quit before the pending timer and the click is gone. A foreign write between the replay snapshot and reclamation's separate enumeration is another path with no transfer at all.

Thread each enumerated record.preservedKey through the reclamation predicate as well. The existing new predicate already supplies the narrow proof needed. Add a full-capacity foreign-reclamation regression for both lanes, including failed transfer or a newly enumerated record. Bootstrap and ACK tests do not establish reclamation safety.

Bounded exit criteria

Preserve the existing version-1 payloads, whole-blob LWW policy, community/identity scoping, write-once ownership, and canonical relay ordering. Resolve the four traces above with production-hook/reclamation outcome coverage. Unchanged AppSidebar/AppShell consumers, signer/community remount boundaries, kind constants, and relay ACK/retention premises were checked against the previous reviewed source by blob identity. Native multi-device runtime and new mobile/web/CLI implementations are not validated by this review. No broader protocol migration, unbounded capacity retention, or unrelated hardening is requested. The new whole-blob suite manually invokes managers despite its “hook-layer” labels; the existing actual-hook replay check only tests timer presence. Those gaps support the defects above, not a separate testing blocker.

Duncan and others added 4 commits September 2, 2026 21:15
CRITICAL 1 (C1): restored replay must carry original queuedAt age.
  - WholeBlobSyncManager.publish() now takes restoredQueuedAt param;
    pendingRestoredQueuedAt drives the adopt-guard in
    fetchOwnBlobBeforePublish (publish when remote.createdAt <=
    restoredQueuedAt; adopt when strictly newer). writeOutbox called
    with original queuedAt so on-disk stamp is never reminted.
  - channelSectionsSync/channelSortSync publishSections/publishSortPrefs
    accept and thread restoredQueuedAt through to publish().
  - channelSectionsStorage/channelSortPreference writeOutbox signatures
    updated to accept optional nowSecs override.
  - Hook-layer tests in runWholeBlobCarlSuite use the actual React hook
    (renderHook), driving the real bootstrap .then() callback, outbox
    read, queuedAt, and shouldReplay guard.

CRITICAL 2 (C2): successful-bootstrap replay needs a baseline.
  - bootstrap() snapshots the fetched head to this.bootstrapResultHead
    before returning. publish(_, true) uses canonicalMax(current,
    bootstrapResultHead) so a no-prior-pending restored replay gets the
    correct baseline from the immutable bootstrap snapshot rather than
    the mutable lastRemoteHead (which may carry a suppressed live peer
    head H102 that must remain a genuine advance).
  - New runWholeBlobC2Suite (manager layer) drives publish(_, true,
    undefined) so C1's queuedAt guard is disabled and only C2's
    bootstrapResultHead mechanism can prevent the adopt.

CRITICAL 3 (C3): confirmRetainedHead decrypt gap.
  - For a foreign winner, lastRemoteHead is advanced only AFTER
    decryptAndParse succeeds (mirrors fetchRemoteBlob fix). A click
    arriving during the confirm decrypt gap now sees lastRemoteHead as
    {0,""} (or the pre-confirm value) and correctly adopts the foreign
    winner rather than publishing over it.
  - New runWholeBlobC3Suite (manager layer) gates the first decrypt call
    and injects a click during the gap; asserts publishCalls === 1.

CRITICAL 4 (C4 / P3-reclaim): reclaimSubsumedOutbox drops preservedKey.
  - mergeLaneStorage.shared.ts reclaimSubsumedOutbox signature updated
    to accept optional preservedKey; calls isSubsumedBy(record.store,
    head, record.preservedKey) so a capacity-bounded proof can never
    evict the reserved channel and falsely certify retention.
  - New P3-reclaim tests (already in mergeLaneSyncCarl.shared.test.mjs)
    exercise 500-entry relay head + foreign outbox with clicked channel
    absent; mutation (drop preservedKey) → record deleted → test red.

IMPORTANT: test coverage.
  - P1/C1 hook tests now drive renderHook with the real outbox read,
    queuedAt threading, shouldReplay guard, and bootstrap .then()
    callback (not manager.publishReplay directly).
  - P3-ack fixture clock set to t=1s so clicked channel is the
    lexicographically evicted entry at 501-entry capacity, making the
    pendingPreservedKey-drop mutation genuinely causal (2 red).
  - All 6 named mutations verified red on exactly their tests, green
    on fixed source: C1 (2r), C2 (2r), C3 (2r), C4/P3-reclaim (2r),
    P3-ack (2r), P2a-1 (2r), P2b (2r). 57/57 pass on fixed source.
Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
…s and MINOR adopt-clear fix

IMPORTANT 1 — add C1-stale actual-hook test (failed-bootstrap stale replay must
adopt, not publish). Causality: drop !pendingIsRestoredReplay guard → exception
fires → publishes stale edit over newer head → 2 reds (sections + sort).

IMPORTANT 2 — assert original queuedAt persists in transferred v2 outbox before
debounce fires. Causality: replace restoredQueuedAt with undefined in writeOutbox
call → stamps wall-clock value → 2 reds (sections + sort).

IMPORTANT 3 — add runWholeBlobP2a1HookSuite driving blocked-bootstrap sequence
through the real hook: bootstrap parked → click via hook API → H102 live → release
bootstrap → real .then() reads/replays outbox naturally → debounce → H102 adopted
as genuine advance. Causality: publishBaseline = lastRemoteHead in publish(_, true)
→ H102 folds into baseline → publishes over H102 → 2 reds (hook: sections + sort).

MINOR — extract clearPendingState() helper shared by discardPending() and
adoptRemote(), so adoptRemote's gen===pendingGeneration branch also clears
pendingIsRestoredReplay and pendingRestoredQueuedAt on adopt.

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
…rl hook tests

P1/C1-stale: after adopt completes, assert (a) lane storage equals the adopted
remote store, and (c) readOutbox returns null. Two new mutations now trip these
tests: removing clearOutbox from clearPendingState and removing the
onRemoteAdopted callback each leave the two stale tests red.

P2a-1 hook: after H102 adopt completes, assert lane storage equals H102's store
and the click outbox is cleared. The dropped-callback mutation turns the two
hook tests red.

runWholeBlobP2a1HookSuite signature extended with storageKey and readOutbox;
callers (sections, sort) pass readOutbox from their respective lane modules.

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Three assertion-only additions to close Thufir's pass-4 BLOCK:

1. runWholeBlobCarlSuite + runWholeBlobP2a1HookSuite each gain an
   assertHookState parameter (lane-specific hook oracle). Both callers
   (sections: .sections/.assignments; sort: .sortModeFor("remote"))
   pass assertions that target hook.result.current after the act() flush.

2. The four named test sites call assertHookState after the storage and
   outbox assertions, independently verifying React state. The erroneous
   "collapsed into storage" comment is removed.

3. Removed the applyRemote-returns-prev class of regression: if the
   callback writes storage but returns stale React state the hook tests
   now go red while the storage assertions stay green — exactly the
   mutation Thufir proved in his disposable tree.

4. wholeBlobSyncCarl.shared.test.mjs trimmed from 1009 to 995 lines by
   collapsing redundant inline comments (already covered in JSDoc).
   All touched files ≤1000 lines (wc -l).

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
…l P2)

wholeBlobSyncManager.startPendingEdit() unconditionally reseeded
publishBaseline from mutable lastRemoteHead on every non-replay call.
Sequence: A1 click pending → live H102 suppressed by hasPendingEdit →
A2 second click → lastRemoteHead (=H102) folded into baseline →
preflight saw no advance → A2 silently overwrote H102-only changes.
Contract violation per JSDoc :51-55/:163-174.

Fix: reseed publishBaseline only when (a) starting a fresh pending
sequence (wasIdle) or (b) lastRemoteHead holds our OWN in-flight
published attempt (lastIsOwnAttempt: eventId ∈ ambiguousAttemptIds).
Case (b) is required so foldSupersedingAttemptWinner can repair the
baseline on a same-second collision. A suppressed live remote head is
never in ambiguousAttemptIds → lastIsOwnAttempt=false → baseline is
preserved across all pending phases (debounce, preflight, publish,
confirmation wait).

Replay path (isRestoredReplay) semantics unchanged.

New tests: wholeBlobSyncCarlP2SecondClick.shared.test.mjs, Variant A
(debounce-window) and Variant B (in-flight), wired into both lane
test files. Required mutations: timer-phase revert turns Variant B
red; unconditional-reseed revert turns all four red. Same-second
collision test remains green under both mutations.

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>

@wesbillman wesbillman left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Carl, an automated reviewer, commenting via Wes’s GitHub account.

Changes requested: two remaining P2 data-loss paths

Reviewed head ffda3579a5e61af7073f747e257ed2a4b5509755 against base ef0d2025683869418e8eee22ac5b5ac16c5198b7, with corrective comparison to 31510eabb18207dce052c3fbe3afdf904c01d35c. This is a read-only source review; I did not check out or execute PR code, run tests, or claim runtime validation.

1. P2: Replay age must not override an observed post-click remote advance

Anchor: wholeBlobSyncManager.ts:655–667.

The new restored-replay branch executes even after a successful bootstrap and ignores whether a competing head was independently observed. A concrete synchronized-clock sequence through either production hook:

  1. Bootstrap is waiting to return H100 (created_at=100). The user edits at wall second 101; its own outbox records queuedAt=101.
  2. Later in that same second, peer H101 is retained and delivered live. It decrypts successfully, but the hook suppresses applying it while the edit is pending.
  3. Bootstrap returns H100. The hook rereads its current-session outbox and calls publishSections(store,true,101) / publishSortPrefs(store,true,101). The repaired baseline correctly remains H100.
  4. Preflight finds H101 and correctly detects an advance, but 101 <= pendingRestoredQueuedAt overrides adoption. The client signs above H101 and publishes a blob that lacks H101’s peer-only changes.

This regresses the previously repaired bootstrap/live-head contract without clock skew. The override also affects a genuinely restored record: resume an outbox queued at 101 against H100, then observe peer H101 later in the same second while that replay is pending. Preserving fresh-session provenance alone does not protect that replay. Preserve genuine observed-advance arbitration; age-based replay eligibility cannot establish that a head was incorporated. The actual-hook P2a-1 regression currently uses queuedAt=50 and a live head at 200 (wholeBlobSyncCarl.shared.test.mjs:638–659,719–721), which avoids this branch. Add the equal-second witness through both real hooks and assert the adopted UI/cache and no overwrite.

2. P2: Bootstrap completion still relabels a fresh pending edit as restored

Anchors: useChannelSections.ts:178–196, mirrored in useChannelSortPreference.ts:175–193; engine wholeBlobSyncManager.ts:529–532,642–670.

Prior finding #3 is only masked when the newly fetched head’s timestamp is at or below the click’s wall-clock stamp. Start with no old outbox and no observed baseline. While bootstrap is pending, make a fresh click at wall second 100, then fail bootstrap. Its callback rereads that click’s own outbox and republishes it as (store,true,100), disabling the fresh first-unknown-base exception. If preflight discovers already-retained H101, the age branch fails (101 > 100), so adoption deletes the fresh pending edit and outbox without publishing it.

H101 can predate this click on synchronized clocks: another window can legitimately publish clampPublishCreatedAt(100,100)=101 (sidebarSyncWatermark.ts:45–52). This is the implementation’s normal timestamp bump, not the accepted clock-skew residual. Leaving the already-pending current-session generation intact preserves its first-unknown-base behavior; turning it into a restored generation does not. Preserve that provenance rather than compensating with timestamps. Cover a fresh click before bootstrap failure through both hooks, through preflight and completion; a preseeded old outbox or a manager click after bootstrap failure does not exercise this seam.

Credited repairs and stable exit criteria

The successful restored-outbox bootstrap baseline initialization, ordinary second-click baseline preservation, and per-record preservedKey forwarding during foreign merge-outbox reclamation are repaired in the inspected source. The confirmation decrypt ordering is also improved. Keep those repairs and the existing version-1, whole-blob/per-entry conflict contracts. Resolve only the two causal sequences above with production-hook regression coverage; no general clock, mixed-fleet, legacy, or capacity redesign is requested.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants