fix(sidebar): converge channel sidebar state across devices - #6525
fix(sidebar): converge channel sidebar state across devices#6525wpfleger96 wants to merge 52 commits into
Conversation
3da6677 to
d0c23c9
Compare
Channel-section sidebar state diverged between a user's devices and sometimes never self-healed. Four client-side gaps fed the divergence: - A local edit that lost whole-blob LWW was silently republished as remote content while the UI kept showing the edit. Now the manager adopts the winning remote head (writes it through to state + storage, advances the watermark) and skips publishing, unifying with the relay's OK-false conflict path as one convergence mechanism. - Edits made inside the 2s publish debounce were dropped on quit or community switch. A durable localStorage outbox persists every edit synchronously and resumes it on next mount; adopt clears the outbox so a superseded edit can never be replayed. - A skewed remote head could push the published createdAt past the relay's future-drift window and wedge all later publishes. createdAt is now clamped inside that window. - Stale-at-open state waited for a reconnect that a healthy socket never fires. A reconciliation loop periodically refetches the head (steady 60s, backoff on failure) and refreshes on window visibility. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Three cross-layer races defeated the one-convergence-mechanism design: - An older in-flight publish unconditionally cleared pending state on completion, erasing a newer edit queued mid-flight. Each pending edit now carries a monotonic generation; a completion clears pending/outbox/retry only via compare-and-swap on the generation it published. - Hook-level remote application (bootstrap/live/periodic) cancelled the pending publish's timers without deciding supersession, stranding the durable outbox and clobbering the optimistic edit. applyRemote now defers entirely to a pending edit, whose own debounced publish converges via publish-or-adopt; the manager's adopt path clears pending before write-through so the winning remote still applies. - The equal-timestamp tie-break kept the largest event id, opposite the relay/database canonical order (created_at DESC, id ASC → lowest id wins). applyRemote now applies a strictly-lower id and ignores ids >= the last applied, so the UI converges on the event the relay actually stored. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
useChannelStars, useChannelMutes, and useChannelSortPreference carried the same inverted equal-timestamp comparator as channel sections: applyRemote kept the largest event id, opposite the relay/database canonical order (created_at DESC, id ASC -> lowest id wins). Two devices writing the same second could leave the UI showing an event the relay did not store. Apply a strictly-lower id and ignore ids >= the last applied, matching the sections fix and the relay winner across all four 30078 sidebar surfaces. Each hook gains a regression test: larger-then-lower id delivery at equal timestamp, lower-id store wins (mutation-checked - reverting >= to <= fails each). Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Two convergence holes one layer under the pass-1 fixes: Sections: the pre-publish head check compared the fetched head against the mutable lastRemoteCreatedAt, which a live event observed during the debounce window already advanced to that same head — equality fell through to publish and the local blob overwrote a remote that became head after the edit was queued. Freeze a canonical head baseline (created_at, id) at publishSections and compare the fetched head against that generation baseline instead, adopting when the head advanced. Stars/mutes: applyRemote admits the canonical lower-id winner but then mergeStores resolved equal per-entry updatedAt as local/prev-wins, so a stale larger-id value delivered first survived and undid the winner. Add mergeApplyingRemote which resolves an entry-timestamp tie toward the canonical incoming blob while keeping strictly-newer local entries. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Each prior round patched one cross-generation interleaving and opened another a layer deeper. Kill the race class structurally instead. Sections: serialize publish cycles (one in-flight at a time; a newer edit queued mid-cycle defers and the completion re-drives it). The per-edit pre-publish baseline is frozen at queue time, so a genuine remote observed during the debounce window still adopts, while our own accepted head is folded forward via canonicalMax so a stale generation's own write is never mistaken for a competing remote and adopted away. Dual generation guards in doPublish (post-fetch and pre-publish) stop a stale generation signing or publishing after a newer edit exists. Stars/mutes: scope mergeApplyingRemote (remote-wins on entry-tie) and the pending-publish cancel to fire only on a canonical supersession of an already-applied same-timestamp larger-id head. Every other application (bootstrap/live/newer-timestamp) keeps local-wins mergeStores and does not cancel the pending publish, so a later same-second local click is no longer clobbered by an older remote entry that decrypts late. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
…biguous-ACK heads Round-4 client-side convergence fixes for channel-sections/stars/mutes sync, closing two silent edit-loss variants that survived publish serialization. Ambiguous-ACK fold: a publish whose ACK is lost may still have been accepted by the relay. Retain each attempt's signed id; when a later cycle's pre-publish fetch returns a head whose id matches a prior attempt, fold it forward as our own accepted predecessor and publish above it instead of adopting it away and erasing the queued edit. A head the relay never accepted can never surface by id, so the fold is proof-gated on an exact id match. Canonical-supersession dirty overlay (stars + mutes): a lower-id canonical correction that arrives after a same-second local click must not clobber the click. Apply the correction to the prior remote layer, then overlay entries changed locally since that layer; never cancel a pending publish merely because a correction arrived. Client-only: loss discovery relies on the existing pre-publish fetch, live subscription, and reconcile loop rather than a relay conflict signal. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Replace the LWW register plus ownership/dirty-set/canonical-supersession machinery with a per-entry Lamport `rev` and a single max-merge on every path, mirroring the read-state data model. Each entry carries an additive optional `rev` (missing implies 0; payload stays `version: 1` so older builds keep parsing our blobs). One `mergeStores` orders by updatedAt then rev then the starred/muted-true leaf, and ends in the 500-entry bound. Clicks stamp `updatedAt = max(now, localEntry?.updatedAt ?? 0, maxUpdatedAtSeen(id))` and mint `rev = max(localEntry.rev, maxRevSeen(id)) + 1`, so a click strictly dominates every state its replica has observed and cannot lose to a same-second remote. The sync managers hold a per-channel two-field high-water map fed by a single `observe()` on every ingest path. Stars sync keeps the generation-CAS + single-flight lane and bounded-backoff retry plus a durable outbox so an in-flight publish can never clear a newer pending edit; mutes sync mirrors it. Remote ingestion never touches the pending lane. Deleted: mergeApplyingRemote, mergeStoresWithTie, mergeCanonicalSupersession, dirtyChannelIds, lastAppliedRemoteTs/lastAppliedEventId, and the event-clock branch. Sections and sort are untouched. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
… contract The hard-eviction-branch test asserted only the rev tuple outcome on two one-entry stores while its comment claimed the >500-entry eviction/remount setup. Build the real fixture: >500 equal-updatedAt entries so the bound evicts the target by the id tiebreak, then a remounted rev-1 click merged against the retained rev-100 remote, asserting the deterministic rev-100 outcome in both merge orders. Stars and mutes. The unobserved-future mixed-fleet residual was stated but never exercised directly: the fast-clock suites cover only the observed-future fix. Add a click with an empty high-water at t, then a genuinely unobserved opposite-value head at t+300 that wins on the primary updatedAt key. Both hook suites. Test-only; no production source changes. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
wesbillman
left a comment
There was a problem hiding this comment.
Carl, an automated reviewer, commenting via Wes’s GitHub account.
Requesting changes at exact head 0d797b550e405eea9557e053f884533ad3dd891a. The relay ordering and the per-entry max-merge are coherent, but the lifecycle still has reproducible edit-loss paths.
[P1] The durable outboxes are not safe across desktop windows
Sections, stars, and mutes share one localStorage outbox key per identity + relay, but each window owns only an in-memory generation. A write in one window therefore replaces another window's pending payload, and a completion in either window removes the shared key without proving that it still owns the persisted value.
For sections, window A can start publishing edit A, window B can replace the outbox with newer edit B, and A's ACK then clears B's outbox. If B closes before its debounce fires, the next bootstrap applies relay head A before discovering there is no outbox, so B is permanently lost. See channelSectionsStorage.ts:211-245, channelSectionsSync.ts:262-281,284-313,522, and useChannelSections.ts:138-156.
Stars and mutes can lose independent clicks even earlier: two windows starting from the same store can click different channels before receiving each other's asynchronous storage event. Their whole-store main/outbox writes race, teardown cancels both timers, and remount resumes only the last blob. The storage handler merges only into React state; it does not durably merge the main store or outbox. See channelStarsStorage.ts:206-242, useChannelStars.ts:57-72,95-113, and channelStarsSync.ts:256-260,383-393 (mutes mirror these paths).
Please give persisted attempts cross-window ownership, not only manager-local generations. A per-operation outbox whose owner deletes only its own record, or an actual cross-window serialization mechanism, would close both overwrite and stale-clear races. Add multi-window tests that interleave write, ACK, storage delivery, teardown, and remount.
[P1] Sort preferences still drop pending edits on ordinary lifecycle and failure paths
publishSortPrefs keeps intent only in memory (channelSortSync.ts:113-121). destroy() deliberately cancels and discards it (:252-262), while a failed publish only logs and never retries (:167-210). A live remote also cancels the debounce while leaving pendingStore stranded (useChannelSortPreference.ts:82-103).
Reproduction: change a sort mode and quit/switch communities within two seconds, or let one publish time out and remount. Bootstrap then whole-blob-replaces the local cache with the relay head (useChannelSortPreference.ts:108-122), visibly reverting the user's choice. Please carry the durable outbox, generation ownership, serialized retry, and pending-aware remote application used by sections over to sort, with lifecycle tests.
[P2] Future relay heads can wedge stars, mutes, and sort publishing
Sections clamps created_at inside the relay's future-drift window (channelSectionsSync.ts:463-472), but stars (channelStarsSync.ts:298-301), mutes (matching code), and sort (channelSortSync.ts:183-186) stamp lastRemoteCreatedAt + 1 without a cap. If a self-authored head was accepted near the relay's +900s boundary, a correctly clocked second device emits +901s; the relay rejects it, and retries continue deriving from the same head until wall time catches up. Apply the same bounded timestamp rule across all four sidebar sync surfaces.
CI is green at this head. I did not duplicate the CI-equivalent suite locally; these failures are source-reproduced interleavings absent from the current tests.
0d797b5 to
16aeab6
Compare
wesbillman
left a comment
There was a problem hiding this comment.
Carl, an automated reviewer, commenting via Wes’s GitHub account.
Re-reviewed exact current head 16aeab68b094baaba508a0f2fc91c06b0c430c1c against base ef0d2025683869418e8eee22ac5b5ac16c5198b7 after the force-rebase. Changes are still required.
All 19 files in this PR have the same Git blob IDs as at previously reviewed head 0d797b550e405eea9557e053f884533ad3dd891a, including all production sync files and tests. The only sidebar differences between the old and new repository trees are three UI files inherited from the new base; they do not touch the NIP-78 sync producers, consumers, or their direct dependencies. Consequently, the previous review's blockers survive unchanged:
- P1: sections/stars/mutes still lack cross-window ownership for the shared durable outboxes. Manager-local generation fencing cannot stop one desktop window from overwriting another window's persisted payload or from unconditionally clearing the newer payload after its own older ACK/no-op/adopt completion. The existing tests remain single-manager and do not cover two windows sharing the key.
- P1: sort still drops pending intent. It still has only an in-memory
pendingStore, no durable outbox or failure retry,destroy()still cancels and nulls the edit, and a live remote can still cancel the timer while leaving the intent stranded. - P2: stars, mutes, and sort still derive
created_at = max(now, lastRemoteCreatedAt + 1)without the future-drift clamp used by sections. A self-authored head accepted near the relay's +900s limit can therefore wedge subsequent writes until wall time catches up.
Please address the concrete reproductions and repair boundaries in the review on 0d797b550: #6525 (review)
Current CI is not green: Desktop Smoke E2E (2) failed at this head while several desktop jobs remain in progress. I am not using that still-unclassified failure as a separate code finding; the source blockers above independently require changes.
Carl's re-review found three edit-loss paths remaining after the stars/mutes rev-merge landed. [P2] Stars, mutes, and sort stamped createdAt = lastRemoteCreatedAt + 1 uncapped, so a self-authored head accepted near the relay's +900s drift boundary wedged every later publish until wall time caught up. Extract the sections clamp into a shared clampPublishCreatedAt in sidebarSyncWatermark.ts (all four surfaces already import that module) and wire sections/stars/mutes/sort to it. [P1] Sort preferences never got the durable lane: doPublish only logged on failure, destroy() discarded the pending edit, and a live remote cancelled the debounce with the edit stranded. Port the sections lane — durable outbox + bootstrap resume, generation/CAS ownership, single- flight + completion re-drive, 2s->30s backoff, 60s reconcile loop, and pending-aware remote application. Sort stays whole-blob LWW; a lost head is adopted at pre-publish rather than republished. Tests: a clamp test per surface, and sort lifecycle coverage — outbox resume after destroy-inside-debounce, retry without a later edit, overlapping-generation safety, live-remote-during-debounce adopt, and a hook-level pending-defer test. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Sections, stars, mutes, and sort each persist an unpublished edit under one
localStorage outbox key per identity+relay, but generation ownership is only
in-memory per window. Without cross-window ownership, one window's completing
publish could clear a peer window's still-unpublished edit, and (on the merge
lanes) a peer's write could overwrite an edit before it published.
Every outbox write now mints an ownership token and stores a {store, token}
envelope; a completing publish compare-and-clears only when the stored token
still matches its own, so a peer's newer write survives an older window's ACK.
Stars and mutes additionally read-merge-write both the durable outbox and the
main store via their per-entry mergeStores, so two windows editing different
channels both survive; sort and sections replace whole-blob with LWW resolution
matching the relay. Sort also gains the durable outbox + bounded retry it
previously lacked, and stars/mutes/sort now clamp publish created_at inside the
relay's future-drift window like sections. The envelope reader tolerates a
legacy token-less entry so an outbox written by a prior build still resumes.
Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
wesbillman
left a comment
There was a problem hiding this comment.
Carl, an automated reviewer, commenting via Wes’s GitHub account.
Requesting changes at exact head 9141fe292268e2681e65664bbeb708b732af2535.
[P1] The multi-window outbox operations are still racy
The new ownership token does not make the localStorage operations atomic. clearOutboxEntry reads and validates the stored token, then calls removeItem separately (sidebarSyncWatermark.ts:141-162). Window A can read token A, window B can write its newer {store, tokenB} envelope, and A can then remove B's value using the stale read. That still loses an unpublished edit on quit/remount.
Stars and mutes have the matching write-side race: writeChannelStarsOutbox reads the shared entry, merges in memory, then writes separately (channelStarsStorage.ts:226-237; mutes mirrors it). If two windows read before either writes, each computes a one-sided merge and the later setItem drops the other pending click. The main-store read/merge/write has the same shape (channelStarsStorage.ts:135-144).
The added multi-window tests execute whole operations sequentially, such as A write, B write, then A clear (multiWindowOutbox.test.mjs:62-167), so they cannot exercise either read/write or read/remove interleaving. localStorage provides no compare-and-delete or transactional read-modify-write primitive; the token proves what was read, not what is still stored at the destructive operation.
Please move this durability boundary to a design that does not depend on atomicity localStorage lacks, such as per-operation append-only records with owner-specific deletion, or an appropriately serialized cross-window store. Add tests that pause operations between their read and write/remove steps and verify teardown/remount preserves every unpublished intent.
The LWW comparator itself matches the relay's created_at DESC, id ASC rule; this review is blocking only on the durability race above. I reviewed read-only GitHub metadata and diff and did not check out or execute PR code.
localStorage has no atomic compare-and-delete or transactional read-modify-write, so a single outbox key shared across every window could not be mutated safely: one window's read-then-write or read-then-remove races a peer's write in the gap and drops its still-unpublished edit. A per-write ownership token narrowed that window but could not close it — the token proves what was read, not what is still stored at the destructive op. Key the outbox per window instead: <prefix>:<pubkey>:<relay>:<nonce>, where the nonce is minted once and parked in sessionStorage. Each window is the sole writer of its own key, so a hot-path write is one unconditional setItem — the write race is designed out, not guarded. Resume enumerates every window's key: merge lanes (stars/mutes) fold all records order-independently; whole-blob lanes (sort/sections) replay the max-queuedAt record with a nonce tiebreak. Redundant foreign keys are reclaimed at boot, gated on durable relay evidence (merge: head subsumes; whole-blob: head created_at supersedes) and re-read immediately before removal so a live peer's fresh write in the recheck gap survives. Reclamation runs only on a successful head fetch, never on a failed one. The token contract is deleted entirely — ownership is the key. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
…ce-free The per-window token/recheck reclamation still performed a non-atomic compare-then-delete on a mutable foreign key: a live owner could rewrite its key between the reclaim decision-read and the removeItem, and the whole-blob `queuedAt <= head.created_at` gate dropped same-second and legacy `queuedAt=0` records that had not provably lost LWW. Records are now write-once: a key is `<prefix>:<pubkey>:<relay>:<nonce>:<seq>` and is never rewritten. A new edit writes a new key (next zero-padded seq) then deletes its own older keys (write-before-delete, so a crash leaves at least one record). Foreign reclamation reads an immutable record, proves it reclaimable against durable relay evidence, and deletes it with no recheck. Whole-blob supersession is strict (`queuedAt < head.created_at`); replay runs before reclamation in every hook so a same-second record is consumed into pending first; the legacy v1 shared key is only ever replayed, never deleted. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
The legacy v1 shared outbox key is never deleted (it is mutable and a concurrently-live old build may still rewrite it), so the whole-blob resume path re-read it on every boot and republished the stale blob above the current relay head forever. A found relay head never stopped it: a fresh manager has no lastPublishedStore and queueing the replay freezes publishBaseline to the just-fetched head. Distinguish compatibility replay from permanently pending intent with a durable per-value consumption marker, whole-blob lanes only. resumeWholeBlobOutbox excludes the legacy record when its exact raw matches the stored marker; a live old build rewriting the key stores a different raw and is replayed again. The hook transfers the intent into its own v2 key (synchronous publish) BEFORE writing the marker, so a crash in that gap replays the blob once more rather than losing it. Merge lanes (stars/mutes) need no marker but gained a head-subsumed gate so a lingering legacy key does not re-drive an identical boot-time publish. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
wesbillman
left a comment
There was a problem hiding this comment.
Carl, an automated reviewer, commenting via Wes’s GitHub account.
Requesting changes at exact head 3712e6fb2c71f03587a476c36081b1ac831a60c7 against base ef0d2025683869418e8eee22ac5b5ac16c5198b7.
Product direction: the relay is authoritative, while sidebar sections/sort use whole-blob LWW and stars/mutes use per-entry max-merge. The compatibility contract is that all four lanes converge across concurrently running builds/windows without dropping an unpublished user edit. I reviewed the four surfaces across bootstrap, live delivery, storage events, reconnect, periodic/visibility reconciliation, local edits, publish failure/ACK, destroy, remount, and community switch. I found these blockers:
[P1] Preserve the original whole-blob baseline when reconnect wakes a pending edit
useChannelSections.ts:264-276 and useChannelSortPreference.ts:257-269 fetch on reconnect and then call the public publish*() method for an already-pending edit. Those methods increment pendingGeneration and reset publishBaseline to the now-current lastRemoteHead (channelSectionsSync.ts:277-295, channelSortSync.ts:273-289).
Reproduction: window A queues an edit against H0; window B publishes H1 before A reconnects. A's reconnect fetch records H1, and applyRemote correctly defers because A is pending. The reconnect handler then re-queues A's old store, replacing its frozen H0 baseline with H1. The pre-publish fetch sees equality rather than remoteAdvancedSince, so A publishes above H1 instead of adopting the remote winner. This reverses the intended LWW decision on both whole-blob lanes.
Wake/retry the existing generation without resetting its baseline, and cover remote advancement plus reconnect while an edit is pending for both sections and sort.
[P1] Do not clear a merge-lane outbox merely because its EVENT received OK
channelStarsSync.ts:279-330 and channelMutesSync.ts:279-330 clear their own durable outbox after publishEvent resolves. But the relay returns accepted OK for a superseded NIP-33 write as a no-op, and two windows can also prefetch the same head and publish different whole blobs at the same second. Only one blob is retained.
The shared cache does not close this race: writeChannelStarsStore / writeChannelMutesStore at storage lines 140-151 overwrite the shared key without reading it, while the storage handlers at useChannelStars.ts:67-74 and useChannelMutes.ts:67-74 ignore e.newValue and reread whichever snapshot currently occupies that key. Two windows can therefore create different-channel snapshots from the same base, overwrite the cache before either storage event is processed, and independently receive successful OKs. The non-retained window then deletes the only durable record of its click. The relay, cache, and all outboxes can end with that edit absent.
After publication, clear an own outbox only after an authoritative retained-head fetch proves that head subsumes the attempted store; otherwise merge and retry. Add a two-window test where distinct edits race from the same head and both OK paths complete in either NIP-33 order.
[P1] A later old-build click can lose forever to an earlier new-build rev in the same second
channelStarsStorage.ts:59-77,188-195 and the identical mutes code normalize a missing rev from an old build to 0, then rank updatedAt before rev. If a new build publishes {updatedAt:T, rev:1, starred:true}, an old build observes it and the user unstars during the same second T, that later old-build payload has no rev and parses as rev:0. New builds deterministically retain the earlier true/rev:1; the user's later action cannot win until another click occurs in a later second.
Keeping payload version: 1 allows parsing but does not make the causal model backward-compatible. Use an explicit migration/version strategy that handles old writers, or define and test a product-accepted mixed-fleet limitation rather than claiming observed later intent cannot be lost.
[P1] An unreadable current head must block whole-blob overwrite, not fall through to publish
channelSectionsSync.ts:353-359 and channelSortSync.ts:341-347 record an existing head, but if decryption or parsing fails they return publish. A pending local blob is then signed at lastRemoteCreatedAt + 1 and replaces the only current head, even though the client could not inspect it or determine whether it should be adopted. A future schema, transient keychain/decrypt fault, or malformed payload therefore becomes destructive data loss on the next local edit.
Treat an unreadable/future head as a failed precondition: retain the durable pending edit and retry/surface the incompatibility, but do not overwrite unknown authoritative state. Add pre-publish coverage for decryption failure and unsupported payload version in sections and sort.
…sidual Close the pass-3 review blocker and Carl's four P1s on the channel sections/sort/stars/mutes relay sync, all in the sidebar lib: - Gate the legacy-consumed marker on a proven v2 transfer. writeOwnOutbox now returns whether the fresh key's setItem succeeded; the whole-blob hooks write the marker only when durable, so a quota failure leaves the legacy record replayable instead of silently suppressing the only copy. - Reconnect wakes the existing pending edit (retryPendingPublish) rather than re-queueing via publish*(), which reset the frozen baseline and published a stale edit over an advanced remote. - Merge lanes clear an own outbox only after an authoritative retained-head fetch proves the head subsumes the attempted store; a bare EVENT OK on a superseded NIP-33 write no longer drops the loser's durable click. - An unreadable/unsupported pre-publish head retains the pending edit and retries rather than overwriting state the client could not inspect. - Document the accepted mixed-fleet residual (a same-second old-build click reads rev 0 and loses to an earlier new-build rev, healing on the next later-second click) and pin it with a test; no protocol change. Regressions added for each item, mutation-verified where the fix lives in the hook. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
wesbillman
left a comment
There was a problem hiding this comment.
Carl, an automated reviewer, commenting via Wes’s GitHub account.
REQUEST CHANGES on exact head 6078943f3be3a3fea99e86d225b79246af53f599 against ef0d2025683869418e8eee22ac5b5ac16c5198b7. I reviewed GitHub metadata, diff, and exact-head source only; I did not execute PR code.
Product contract: the relay is authoritative. Sections/sort converge as whole-blob LWW; stars/mutes converge by per-entry max-merge. A client must not overwrite relay state it could not read, because that turns a transient fetch/decrypt failure into durable cross-device data loss.
P1 — Retain edits when the authoritative pre-publish read fails
All four lanes fall through to publishing local state when the pre-publish relay query throws. Sections does so at channelSectionsSync.ts:371-416 and sort has the matching branch; stars does so at channelStarsSync.ts:210-232 and mutes mirrors it. The client therefore cannot distinguish “no head exists” from timeout/auth/socket failure.
Reproduction: device B publishes a newer sections/sort blob or independent star/mute entries after device A’s last observation; A edits while its pre-publish fetch fails. A signs above its stale watermark and can replace the unseen authoritative head, erasing B’s data. Durable outboxes and retries do not recover content that A never merged.
Return a tri-state publish decision in every lane: only a successful, genuinely absent head may publish the local store directly; fetch failure must retain the outbox and retry. Add causal tests for an unseen newer head plus rejected pre-publish fetch across sections, sort, stars, and mutes.
P1 — Stars and mutes also overwrite an existing but unreadable head
channelStarsSync.ts:220-231 records an event but returns the local store when decryption, JSON parsing, or schema validation fails; doPublish then publishes it (:315-357). Mutes has the identical path. This can destroy entries in a temporarily undecryptable or future-schema head. Bootstrap already classifies this condition as failed, while sections/sort correctly return retain for it.
Give merge-lane pre-publish reads the same retain/retry behavior and cover decrypt failure, malformed JSON, and unsupported schema. A max-merge is safe only after both operands were actually read. Dungeon law is unhelpfully strict about this.
P2 — Reject numeric revisions that cannot advance
channelStarsStorage.ts:59-69 and the matching mutes parser accept any finite non-negative integer rev, including values beyond Number.MAX_SAFE_INTEGER. Local clicks mint maxRev + 1 (useChannelStars.ts:252-267, mirrored for mutes); at sufficiently large IEEE-754 values that no longer increases. With equal timestamps/revisions, true wins (channelStarsStorage.ts:199-207), so a malformed starred:true or muted:true entry can suppress later unstar/unmute attempts indefinitely.
Require safe, bounded integers for revision and timestamp inputs, define malformed-input handling, and add a regression proving a huge revision cannot wedge a later false toggle.
The prior blockers around multi-window ownership, reconnect baselines, unreadable whole-blob heads, publish retention confirmation, and timestamp clamping are materially addressed. Exact-head CI is broadly green; it does not cover the failed-read transitions above.
Extends the retain-on-unreadable pattern to the two remaining overwrite paths and hardens the rev/timestamp parse boundary. - All four lanes tri-state the pre-publish read: a THROWN fetch (timeout / auth / socket) now retains and retries instead of falling through to publish, so an edit during a transient outage can no longer sign above a stale watermark and erase an unseen newer head. - The merge lanes (stars/mutes) retain when the existing head fails decryption/JSON/schema parsing rather than publishing the local store over it — a max-merge is only safe once both operands were read, matching the whole-blob lanes. - The rev/timestamp parsers require Number.isSafeInteger: an unsafe rev is normalized to 0 and an unsafe timestamp entry is dropped, so a malformed huge-rev true entry can no longer wedge later false toggles forever. Retain reuses the existing bounded-backoff retry and generation CAS, so a later readable/successful head resumes normal resolution. Regressions cover all four lanes for the failed-fetch and unreadable-head paths and the rev-wedge P2 case in both storage suites. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
…adroom The rev parse bound used Number.isSafeInteger, which accepts Number.MAX_SAFE_INTEGER itself. The click path mints rev = maxRev + 1, so an accepted boundary rev overflows to an unsafe value that never advances again — recreating the same-second toggle wedge the bound exists to prevent. Reject rev >= Number.MAX_SAFE_INTEGER (exclusive, normalize to 0) so maxRev + 1 always stays safe. updatedAt keeps its inclusive bound: the click path takes Math.max(now, observed), never increments, so it cannot overflow. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
wholeBlobSyncManager: no-op guard now compares only against the freshly fetched readable head (decision.fetchedRemote.store). Absent, failed, unreadable, and foreign-author pre-publish paths can never no-op; the stale lastPublishedStore comparison is removed entirely. useChannelMutes/useChannelStars: comment-only, removing obsolete field reference that no longer exists. Test consolidation (22 files, 2,570 ins / 3,008 del): - 17 changed test files: 4,458 lines; all 24 head sidebar lib test files: 4,786 lines; caller files 6 lines each - Helper sidebarSyncTestHelpers.mjs: 487 lines; sibling sidebarAdapterContractTestHelpers.mjs (new): 587 lines; both under the 1,000-line repo ceiling (AGENTS.md:590) - Whole-surface total (24 tests + 2 helpers): 5,860 lines (was 6,303, base 23 tests + 1 helper; -443) - 5,518/5,518 desktop tests passing at this commit; no new biome warnings in changed files (pre-existing: channelMutesStorage .test.mjs:115-116 useLiteralKeys, SetupStep.acpForcedGate.test.mjs :423 useOptionalChain, wholeBlobSync.shared.test.mjs:554 noUnusedFunctionParameters, terminal.css:276 noImportantStyles) Adapter-contract refactor: _runAdapterContract (15-knob generic) deleted; runSectionsAdapterContract and runSortAdapterContract are genuinely concrete, each using typed lane APIs and lane-local constants inline. Repeated test infrastructure (assert, test, mock, relayClient, relay URL) hoisted into _adapterCtx() cache + ADAPTER_RELAY constant; phases 2/3 use _runDurableResumeTest / _runUnsupportedVersionTest with 5 lane-specific params each (laneLabel, Manager, publishTo, readOutbox/ badVersionPayload, makeStore). Adapter section split into sidebarAdapterContractTestHelpers.mjs (587 lines) to satisfy the 1,000-line ceiling; re-exported from main helper so callers are unaffected. grep confirms zero old descriptor knob names in either file. Test coverage added: - wholeBlobSync: S1->S2->S1 folded lifecycle - S2 delivery observed via subscription callback; S1 pending and second durable outbox write asserted before confirmation; clear on exact retained-ACK only - mergeLaneHook: click-before-opposite-peer-arrival with causal mutations (a) drop listener -> idsField hook-membership assertion fails; (b) suppress fetchRemote* inside tick -> fetch-count assertion fails; (c) drop fetched result -> stored entry stays at arrival tuple (900,12) not relay head (950,13) - Adapter runner (both lanes): opposite-lane sentinel deep-equal before own-outbox-cleared; fetch and live filters exact-object deep-equal; fetched and callback stores exact deep-equal (sections includes assignments; sort has groups.p === 'recent'); unsupported-version manager.destroy() in finally Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
wesbillman
left a comment
There was a problem hiding this comment.
Carl, an automated reviewer, commenting via Wes’s GitHub account.
REVIEW CLEAR: prior production blocker resolved
Reviewed head e3624ed22f826a83eb1be666e7a629ec1d3a94c3 against base ef0d2025683869418e8eee22ac5b5ac16c5198b7, focusing on the correction since 0f98440bea7dea8f73c78992e1fcfb4e7b6c2870. No remaining actionable production blocker found in this corrective review. This is a comment, not an approval.
- The stale no-op defect is fixed.
lastPublishedStoreis removed. The preflight carries its freshly fetched readable head, and the no-op branch compares only against that head. Therefore S1 published → S2 observed → explicit S1 publishes again rather than silently dropping the edit. Missing heads cannot prove a no-op; failed/unreadable reads retain the durable pending edit. Generation checks and retained-head confirmation remain intact. - Nonblocking test follow-up: the causal S1→S2→S1 witness exercises the production shared engine using
SyntheticWholeBlobManager, not both concrete adapters. Its ID-only synthetic equality and outbox counters do not prove same-ID section rename/assignment restoration, sort-mode restoration, or actual persisted contents through delayed confirmation. Add that scenario through both concrete managers with their real outboxes. The current witness catches the original engine defect; the unchanged concrete equality functions compare the relevant fields correctly, so this coverage limitation is not evidence of another production P2. The merge-hook retention phase now drives actual timers; the earlier click-before-older-opposite-live recommendation remains nonblocking. Restore the deterministic reclamation read→delete-gap witness as well: the new stale/fresh-key coexistence case no longer injects an arrival during reclamation, although the unchanged implementation still deletes only the proven immutable key. - Scope and validation: retained the established sections/sort whole-blob LWW and stars/mutes per-entry merge contract, scoped outbox ownership, community/signer teardown, and accepted compatibility residuals. Rechecked current adapters/consumers and protocol premises against prior integrated coverage; reviewed the consolidated tests with independent bounded lanes. Read-only exact-ref source review only: no checkout, build, test execution, or live multi-device validation. Native/runtime behavior and other client implementations are not newly validated here.
There was a problem hiding this comment.
Changes requested
Reviewed exact head e3624ed22f826a83eb1be666e7a629ec1d3a94c3 against base ef0d2025683869418e8eee22ac5b5ac16c5198b7.
[P1] Stale whole-blob outboxes are replayed above newer relay state — desktop/src/features/sidebar/lib/useChannelSections.ts:164, desktop/src/features/sidebar/lib/useChannelSortPreference.ts:161
On bootstrap, both hooks apply the fetched relay head and then unconditionally pass the selected outbox record to publishSections / publishSortPrefs. That call freezes the just-fetched head as its baseline, and the manager stamps the replay above it.
A reachable sequence is: window A queues an edit at t=100 and closes during the debounce; another device publishes a newer head at t=200; A later opens. The t=100 outbox is promoted to t=201 and overwrites the newer state, even though the reclamation contract immediately below says records with queuedAt < head.createdAt lost whole-blob LWW. Reclamation runs after replay and never touches this window's newly copied own key, so it cannot prevent the overwrite.
Preserve queuedAt in the resume result and, when bootstrap fetched a head, replay only records that are not strictly older; clear/reclaim a stale selected record instead. Add this sequence to both whole-blob lane tests.
[P2] The whole-blob baseline is not the state the user edited — desktop/src/features/sidebar/lib/wholeBlobSyncManager.ts:421-433
publish() freezes lastRemoteHead, but that value means “head observed by the manager,” not “head incorporated into React/localStorage.” This loses data in both directions:
- If bootstrap failed or is still unresolved,
lastRemoteHeadis{0, ""}. A sections/sort edit made after mount freezes that empty baseline. When the pre-publish fetch later finds an older, pre-existing relay head,remoteAdvancedSince()classifies it as post-edit,adoptRemote()clears the pending store/outbox, and the fresh click is never published. - A live callback does the reverse at
wholeBlobSyncManager.ts:762-770: it records the head before async decrypt/application. A click in that gap freezes the unapplied head as its baseline; the subsequent pre-publish fetch sees equality and publishes a stale whole blob over remote-only changes.
Base arbitration on the exact head successfully applied to the editable store. Queue/replay mutations until bootstrap establishes that base, and do not advance the editable baseline merely because a raw event arrived. Add deterministic coverage for both deferred/failed bootstrap and the live decrypt/application gap.
[P2] The pre-publish merge can evict the click that the hook explicitly preserved — desktop/src/features/sidebar/lib/mergeLaneSyncManager.ts:293-299
Stars/mutes preserve the clicked channel while locally bounding to 500 entries (useChannelStars.ts:269-288, mirrored by mutes). The manager then calls mergeWithRemote() without that preserved key. mergeStores() re-bounds at channelStarsStorage.ts:238-253 using only updatedAt and channel ID for eviction.
With 500 remote entries in the same second plus a lexicographically smaller clicked channel, the hook keeps the click, the pre-publish merge drops it despite its higher rev, and retained-head confirmation clears the outbox. Carry the mutated key through the publish merge, or make eviction ordering preserve the newest mutation tuple. Add a 501-entry regression for both stars and mutes.
Assessment
- Minimalness: 6/10 — shared engines help, but 8,501 additions plus four duplicated hook orchestration loops create a large change surface.
- Elegance: 7/10 — the shared lane split is directionally sound, but the abstractions omit two load-bearing facts: which remote head the UI actually contains and which entry must survive capacity bounding.
- Correctness: 6/10 — the reproduced paths silently discard user intent, violating the core contract of this change.
Validation
- Desktop unit suite: 5,518/5,518 passed at the exact head.
- Desktop typecheck: passed at the exact head.
- GitHub Desktop, Unit Tests, and Desktop E2E checks are green.
- The findings are uncovered ordering/capacity cases rather than failures in the existing suite.
… baseline split, P3 preservedKey through mergeWithRemote P1: Gate bootstrap outbox replay on queuedAt > appliedHead.createdAt in both whole-blob hooks (useChannelSections, useChannelSortPreference). Stale records (queuedAt <= head.createdAt) are left for reclaimSuperseded* cleanup. Legacy records (queuedAt=0) under a non-zero head are skipped. Exposes queuedAt from resumeWholeBlobOutbox and the two outbox read helpers. P2b: Split subscribeLive in wholeBlobSyncManager — watermark+lastRemoteCreatedAt advance stays synchronous (preserves undecryptable-event seed-publish guard), lastRemoteHead tuple update moved into the .then() after successful decrypt. Prevents publishing pre-decrypt content over remote-only changes. P3: Thread preservedKey through publish(store, preservedKey?) -> pendingPreservedKey field -> fetchOwnBlobBeforePublish -> mergeWithRemote in MergeLaneSyncManager. Both channelStarsSync/channelMutesSync wrappers and both useChannelStars/useChannelMutes click handlers updated to forward the channelId as preservedKey. Regression tests: - wholeBlobSync.shared.test.mjs: P1 stale-replay (manager level, both lanes via SyntheticWholeBlobManager) + P2b decrypt-gap sequence - wholeBlobHook.shared.test.mjs: P1 stale outbox not replayed test added to runWholeBlobHookSuite; useChannelSections + useChannelSortPreference opt in - channelStarsSync.test.mjs: P3 501-entry merge with preservedKey retains the clicked channel P2a (publish-baseline contract for failed bootstrap) held pending Will's ruling on Option A (narrow guard) vs Option B (queue-until-bootstrap). Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
The P2b decrypt-gap test assumed fetchEvents would be called twice (once for bootstrap, once for pre-publish fetch), but the test never calls bootstrap(). The first and only fetchEvents call goes to fetchOwnBlobBeforePublish; returning [] there produces a 'publish' decision and the test fails. Remove the incorrect two-branch mock. The single fetchEvents call is the pre-publish fetch; it returns the live event as decryptable (good-cipher) so fetchOwnBlobBeforePublish sees remoteAdvancedSince true → adopt, verifying the P2b fix. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
…reservation; causal tests
P1 guard shape fix (both whole-blob hooks):
- Was: reduce all non-apply-remote paths to headCreatedAt=0, gate on queuedAt>0
- Now: branch on bootstrap result explicitly
- hold/absent/failed (result.action !== 'apply-remote'): always replay,
including legacy queuedAt=0 records (old 0>0 guard silently stranded them)
- apply-remote: suppress only on strict queuedAt < appliedHead.createdAt;
equality replays (aligns with reclamation's strict-lt, prevents stranding
a same-second edit neither consumed nor published)
- Documents clock-skew residual: queuedAt vs relay created_at are independent
wall clocks — accepted tradeoff of the existing LWW max-merge strategy
P3 reconnect preservation (MergeLaneSyncManager + both stars/mutes hooks):
- Add retryReconnectPublish() to MergeLaneSyncManager: re-drives the existing
generation without calling publish(), so pendingPreservedKey is not reset
- Expose retryReconnectStarsPublish() / retryReconnectMutesPublish() on adapters
- Switch both hooks' reconnect effects to retryReconnectPublish() instead of
publishStars/Mutes(pending), which would open a new generation and clear key
- Clear pendingPreservedKey in destroy()
P3 remount preservation (storage layer + both hooks):
- writeOwnOutbox: accept optional preservedKey, include in envelope JSON
- parseEnvelope: read preservedKey from envelope; OutboxRecord gains field
- enumerateOutbox: propagate preservedKey into OutboxRecord
- mergeLaneStorage.shared: writeOutbox forwards preservedKey; add
readOutboxPreservedKey to read own-window record's key for bootstrap
- channelStarsStorage / channelMutesStorage: accept+forward preservedKey in
write, expose readChannelStarsOutboxPreservedKey / readChannelMutesOutboxPreservedKey
- MergeLaneSyncManager: writeOutbox config type includes preservedKey param;
publish() forwards it to config.writeOutbox
- useChannelStars / useChannelMutes: import and call readPreservedKey on
bootstrap outbox replay; pass to publishStars/Mutes alongside the store
Causal test coverage:
- wholeBlobHook.shared: replace non-causal stale-replay test with four-case
P1 regression using makeHookTimerBed.hasDelay(2000) as the synchronous
causal signal (timer present iff publish() called): strict-lt suppresses,
equality replays, hold+v2 replays, hold+legacy-queuedAt=0 replays
- channelStarsSync.test: add P3 reconnect (retryReconnectStarsPublish keeps
key) and P3 remount (storage round-trip write+read preservedKey)
- channelMutesSync.test: add matching P3 501-entry, reconnect, and remount
regressions parameterizing coverage across both lanes
Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
…rvedKey from foreign records; add P3 reconnect+restart hook regressions
Thufir pass-2 findings — all four corrections applied:
1. P3 restart durability: replace two-function readOutbox+readOutboxPreservedKey
with readOutboxWithMeta<S> that enumerates records once and returns
{ store, preservedKey }. Key selected by max queuedAt across ALL records (own
and foreign); after quit, the prior window's nonce is gone so its record is
foreign — the old isOwn filter returned undefined, losing the reservation.
Both hooks forward outboxMeta.preservedKey directly to publish* on bootstrap.
2. Two-read snapshot/provenance split closed: the capacity bound now applies
inside readOutboxWithMeta using the selected preservedKey, so the clicked
channel is never evicted before it reaches the manager.
3. discardPending() clears pendingPreservedKey: minor correctness fix so a
confirmed publish resets the field alongside pendingStore and clearOutbox.
4. Test consolidation: remove non-causal reconnect/remount tests from
channelStarsSync.test.mjs (+120) and channelMutesSync.test.mjs (+192).
Replace with two parameterized regressions in mergeLaneHook.shared.test.mjs:
- P3 reconnect: triggers the actual subscribeToReconnects callback registered
by the hook, asserts clicked channel survives 501-entry merge.
Failing mutation: revert hook reconnect to publish*(pending).
- P3 restart: seeds a foreign-nonce outbox envelope with preservedKey,
mounts hook (bootstrap reads foreign record → recovers key → publishes),
fires 2s debounce, asserts clicked channel survives 501-entry merge.
Failing mutation: revert to readOutboxPreservedKey (isOwn filter) or drop
bootstrap preservedKey forward.
Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
…uring Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
… drives merge at doPublish not bootstrap Bootstrap fetch returns [] (hold) so publishStars is called unconditionally — the subsumed guard only runs on apply-remote, which would require extra async ticks to verify. The 501-entry merge still exercises at the production seam: when the 2s debounce fires and doPublish calls fetchRemoteStars (fetchCalls=2), the 500-entry remote head triggers mergeWithRemote producing 501 entries. Without preservedKey forwarded from the foreign record, boundStarStore evicts the clicked channel there. The test structure now matches the existing bootstrap replay test (hold bootstrap → timer fires → publish cycle exercises the production path). Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
…ithMeta Without bestKey in the per-record reduce, folding two durable outbox records totaling 501 entries evicts the clicked channel before the final bound can protect it. Thread bestKey through merge(acc, r.store, bestKey) on every step so the reservation is upheld at each intermediate capacity-bound merge. Update the merge callback signature from (a, b) to (a, b, preservedKey?) — both mergeStores implementations already accept the optional third arg. Also fix two test gaps from Thufir pass 3: reconnect test: both the reconnect-callback fetch and the subsequent pre-publish fetch must return the remote 500-entry head so that reverting the hook reconnect to publish*(pending) triggers the 501-entry merge and fails. The previous single-return-on-call-1 mock left the pre-publish fetch returning [] — no 501-merge occurred and the wrong wiring stayed green. restart test: upgrade from one to two foreign-nonce outbox records. Record 1 carries the clicked channel + 499 entries with preservedKey; record 2 carries the 501st entry with no key. This exercises the per-record fold eviction (mutation c) directly: without bestKey in the reduce the fold evicts the click at the record-2 merge, before the final bound runs. The single-record test could not catch this class of defect. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
No publish debounce completes until bootstrap resolves. Edits queued
during an unresolved bootstrap store their intent durably (outbox write +
in-memory pending state) but the debounce timer is deferred. On
resolution, releaseDeferred() re-freezes publishBaseline from the
now-established lastRemoteHead before scheduling the timer, so the
pre-publish fetch sees the bootstrap head as 'not advanced' and
publishes above it rather than adopting it away.
Failed-bootstrap corner: when bootstrap fails, lastRemoteHead stays
{0,""} and bootstrapFailed=true is recorded. The empty-baseline
exception in fetchOwnBlobBeforePublish fires only when both flags
indicate a failed bootstrap with no known head, treating the first
fetched head as the new baseline rather than a competing remote.
Three causal regressions in wholeBlobSync.shared.test.mjs:
- P2a test 1 (load-bearing): click during unresolved bootstrap on a
fresh device publishes above the relay head; failing mutation: remove
bootstrapStarted guard from publish() → {0,""} baseline → adopt.
- P2a test 2: genuinely newer peer head that arrives after bootstrap
is adopted via normal LWW (publishBaseline re-frozen to bootstrap
head, peer head at higher ts is a genuine advance).
- P2a test 3 (Kalvin's original sequence): click after failed bootstrap
publishes above the relay head; failing mutation: remove bootstrapFailed
exception → hard-adopt against {0,""} baseline.
Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
…e defects Thufir's focused pass (890de2d) identified three executable loss/overwrite paths and two test-gate gaps. This commit addresses all five findings. Production fixes: 1. Absent-bootstrap seed must not clobber a pending edit. When bootstrap resolves absent and local state is non-empty, runBootstrap calls publishFn(localStore) to seed the relay. If a click was already pending (pendingStore !== null), that click IS the seed intent; calling publish(localStore) again bumps the generation and overwrites the outbox with the stale mount snapshot. Fix: guard the publishFn lambda so publish(localStore) is a no-op when a pending edit already exists. 2. releaseDeferred must use the bootstrap-result head, not mutable lastRemoteHead. subscribeLive can advance lastRemoteHead with a live peer head while bootstrap is in-flight. A live head arriving after the click but before bootstrap resolves must remain a genuine remote advance — if releaseDeferred re-freezes publishBaseline from lastRemoteHead, the live head folds into the baseline, the pre-publish check sees equality, and the edit publishes over the peer. Fix: derive bootstrapResultHead from fetchResult directly (not lastRemoteHead), and pass it into releaseDeferred. fetchResult carries exactly the head the bootstrap operation itself fetched; subscribeLive's passive observations are invisible to it. 3. Failed-bootstrap exception must be disarmed once any head is independently observed (live or reconnect/periodic fetch). bootstrapFailed+empty-baseline correctly identifies the "first unknown base" case, but after a transient pre-publish fetch failure and a subsequent live peer head arrival, the exception re-fires on retry and treats a genuine peer advance as a base-establishment, publishing over it. Fix: add bootstrapFailedExternalHeadObserved flag; set it from subscribeLive decode completion and from fetchRemoteBlob when it finds a head while bootstrapFailed is true. The exception checks !bootstrapFailedExternalHeadObserved. Test-gate fixes: - P2a cases moved to a new wholeBlobSyncP2a.shared.test.mjs (449 lines) and exported as runWholeBlobP2aSuite; called from both useChannelSections.test.mjs and useChannelSortPreference.test.mjs for two-lane coverage. Removes the P2a block from wholeBlobSync.shared.test.mjs (841 lines, down from 1080). - Five causal regressions, each named by its exact production mutation: T1: drop bootstrapStarted guard → baseline {0,""} at debounce → adopt fires. T2: releaseDeferred uses mutable lastRemoteHead → live head folds into baseline → pre-publish sees equality → publish over peer (load-bearing for defect 2). T3: drop pendingStore===null guard → seed clobbers edit → stale mount published. T4: drop !bootstrapFailedExternalHeadObserved → exception fires after live observation → publish over genuine peer advance. T5 (Kalvin's original): remove bootstrapFailed exception block → hard-adopt against {0,""} → click lost (load-bearing for defect 3). Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
…eue-time baseline
Plain replacement `publishBaseline = { ...bootstrapResultHead }` discarded the
queue-time baseline when bootstrap resolved with an older head. If a live peer
head B fully applied before the user clicked, publish() froze publishBaseline=B,
but releaseDeferred then regressed it to the bootstrap-result head A. The
pre-publish fetch saw B as an advance and adopted the click away (0 publishes,
1 adopt).
Fix: use canonicalMax(this.publishBaseline, bootstrapResultHead), preserving
whichever head is canonically greater. If the click was authored from B and
bootstrap returned older A, canonicalMax keeps B as the baseline and the edit
publishes above B correctly. For a failed bootstrap, bootstrapResultHead is
{0,""} and canonicalMax leaves the queue-time baseline unchanged.
Add T6 regression to the P2a shared suite (runs through both
ChannelSectionSyncManager and ChannelSortSyncManager): blocked bootstrap on A
→ live B applies → click from B → bootstrap resolves A → edit publishes above
B. Failing mutation: revert to plain replacement → baseline regresses B→A →
pre-publish adopts B (0 publishes, 1 adopt).
Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
wesbillman
left a comment
There was a problem hiding this comment.
Carl, an automated reviewer, commenting via Wes’s GitHub account.
CHANGES REQUESTED
Reviewed exact head 8148ead9cf657e6e489e466644c4b64f8e405083 against base ef0d2025683869418e8eee22ac5b5ac16c5198b7, focusing on the repairs since e3624ed22f826a83eb1be666e7a629ec1d3a94c3. Three unresolved data-loss paths remain in the contracts this revision repairs.
[P1] Failed bootstrap promotes an old outbox into a fresh edit
The new failed-bootstrap exception cannot distinguish a fresh click from a restored outbox. Both hooks replay unconditionally on hold (sort: lines 161–199).
Reproduction: A queues sections/sort at t=100 and quits before debounce; another device retains B at t=200; A reopens, but its bootstrap fetch throws. hold calls publish(A), freezing {0,""} and replacing the outbox's original queuedAt with the current time. The first successful preflight reads B before any live/reconcile observation, so the exception folds B into the baseline and publishes A above it. B's newer layout/preferences are overwritten. The successful-bootstrap age guard never runs in this sequence; retained confirmation then legitimately confirms the wrong replacement.
Repair: retain restored-edit provenance/age through a failed bootstrap and resolve its freshness when a head can actually be read. Do not grant a restored outbox the fresh-click exception or remint its age before that decision. Add failed-bootstrap → recovered newer head coverage through both actual hooks, asserting no stale publication and correct durable-outbox disposition.
[P2] Hook replay and refresh still freeze unapplied heads into the editable baseline
publish() still copies lastRemoteHead, which is not necessarily the head reflected in the editable store. Two concrete paths bypass the new baseline fixes:
- Bootstrap is blocked with H100; a click at wall time 101 queues A; live H102 decrypts while A is pending, so
applyRemoterefuses to apply it. Bootstrap then resolves H100.releaseDeferredcorrectly retains H100 as the baseline, but the hook immediately reads A from its own outbox and callspublishSectionsagain (sort: 175–189). This resets the baseline to H102. Preflight sees equality and publishes the pre-H102 blob over H102's remote-only changes instead of adopting H102. T2 exercises the concrete manager without this hook callback, so it does not catch the production sequence. - After normal bootstrap, a periodic/reconnect fetch receives B and
fetchRemoteBlobrecords its tuple before awaiting decrypt. A click during that await freezes B while still editing A. The eventual hook application is suppressed byhasPendingEdit; preflight B again sees equality and publishes stale content over B. Moving tuple advancement after decrypt only insubscribeLiveleaves this path unchanged.
Repair: preserve the pending generation/baseline when bootstrap finds the current session's already-queued edit, and keep the editable baseline tied to successfully incorporated state across live and fetch paths. Add actual-hook ordering regressions for both sequences, including durable state and the final publish/adopt outcome.
[P2] Capacity-bounded subsumption can certify a click the relay does not contain
The preserved key reaches the publish merge, but isStarsStoreSubsumedBy and the mutes equivalent still prove retention using mergeStores(head, candidate) with no preserved key. That merge bounds to 500 entries before testing equality.
Take a full head R with 500 entries at the same updatedAt. A candidate contains 499 of those entries plus a lexicographically smaller clicked channel X (with a higher rev), and its durable preservedKey is X. R does not contain X. The proof merges 501 entries, evicts X using timestamp/ID ordering, gets R back, and returns true. On restart, the bootstrap subsumption gate skips replay and reclamation deletes the foreign outbox. In a same-second publish collision, retained-head confirmation can likewise clear the own outbox even though the winning head omits X. Both lanes lose the explicitly protected click.
Repair: do not use lossy capacity eviction as evidence that the retained head contains the mutation. Preserve the clicked-entry requirement through bootstrap suppression, reclamation and ACK confirmation, or use an appropriate non-lossy dominance proof. Add found-full-head restart and ACK-but-unretained full-head regressions for stars and mutes. The new restart test intentionally returns an absent bootstrap head (mergeLaneHook.shared.test.mjs:694–704), and the new manager tests return an empty confirmation read, so neither exercises this boundary.
Scope and validation
Read-only exact-ref source/metadata review with independent whole-blob and merge/storage lanes. No checkout, build, test execution or live multi-device validation. Reused and verified unchanged consumer/protocol premises from the prior integrated review: community/identity isolation, all four kind-30078 lane tags, canonical replacement order and ACK-versus-retention semantics. The fresh-readable-head no-op correction and write-once ownership remain intact. Existing mixed-version, clock and general capacity residuals are not reopened; the findings above directly violate this revision's stale-resume, applied-baseline and preserved-click contracts.
P1 — restored-outbox replay re-mints queuedAt and triggers the failed-bootstrap exception designed only for fresh in-session clicks. Hook-replayed outbox calls now pass isRestoredReplay=true through publishSections/publishSortPrefs → publish(), which suppresses the exception (pendingIsRestoredReplay) and does not re-freeze publishBaseline from mutable lastRemoteHead. P2a-1 — hook .then() replay after releaseDeferred establishes the correct baseline called publish() unconditionally, overwriting the baseline with the mutable lastRemoteHead (which by then could carry a suppressed live peer head). The isRestoredReplay flag added for P1 also closes this path: restored replays skip the publishBaseline re-freeze so the baseline releaseDeferred established is preserved through the callback. P2b — fetchRemoteBlob (periodic/reconnect path) called recordRemoteHead before decryptAndParse, leaving a decrypt-gap window where a concurrent click could freeze its publishBaseline against the pre-decrypt head. Moved recordRemoteHead to after decryptAndParse succeeds, mirroring the existing subscribeLive fix. P3 — isStarsStoreSubsumedBy / isMutesStoreSubsumedBy proved subsumption via a capacity-bounded mergeStores(head, candidate) without a preservedKey. At the 500-entry cap the clicked channel (lexicographically smallest) could be evicted during the proof merge, certifying retention of a click the relay never kept. Two consumers affected: bootstrap suppression (hook files pass outboxMeta.preservedKey to the subsumption check) and ACK confirmation (mergeLaneSyncManager passes this.pendingPreservedKey to config.isSubsumedBy). Both storage functions now take an optional preservedKey and use a direct membership check when present: head must contain the preserved channel with rev >= candidate's before the rest of the store is verified. Regressions — hook-layer tests in new shared suites: - wholeBlobSyncCarl.shared.test.mjs: P1 (failed-bootstrap replay adopts newer retained head), P2a-1 (H102 survives hook replay as genuine advance), P2b (fetchRemoteBlob decrypt gap). Wired from useChannelSections and useChannelSortPreference test files. - mergeLaneSyncCarl.shared.test.mjs: P3-bootstrap (500-entry head does not subsume click via eviction) and P3-ack (confirmRetainedHeadSubsumes retains outbox when relay head omits clicked channel). Wired from useChannelStars and useChannelMutes test files. All 47 targeted tests pass (43 pre-existing + 4 new Carl-round P3 tests plus 6 new P1/P2a-1/P2b tests). T1-T6 manager-level contracts unchanged. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
wesbillman
left a comment
There was a problem hiding this comment.
Carl, an automated reviewer, commenting via Wes’s GitHub account.
Changes requested
Source-only re-review of 31510eabb18207dce052c3fbe3afdf904c01d35c against base ef0d2025683869418e8eee22ac5b5ac16c5198b7, focused on the repairs since 8148ead9cf657e6e489e466644c4b64f8e405083. No PR code was checked out, built, tested, imported, or executed; no runtime/CI pass is claimed.
The exact prior failed-bootstrap stale-replay, bootstrap/replay baseline-refreeze, and fetch/decrypt-gap witnesses are repaired. The preserved-click proof is also repaired for bootstrap suppression and ACK confirmation. Four actionable cases remain:
1. [P1] Successful reopen discards a valid newer sections/sort outbox
wholeBlobSyncManager.ts:530-544, with releaseDeferred at 394-410 and hook replay at useChannelSections.ts:178-195 / useChannelSortPreference.ts:175-192.
Relay holds H100; the user edits A at 101 and reloads before debounce, then makes no new click. Bootstrap successfully finds unchanged H100. Because the new manager has no pendingStore, releaseDeferred() never initializes publishBaseline. The hook correctly accepts A's age, but now calls publish(A, true), preserving {0,""}. Preflight therefore treats H100 as an advance and adopts it (628-680) without publishing A. On same-window reload, the replay transfer replaces the original own outbox, and adoption deletes that replacement too. An ordinary successful resume loses the saved edit.
Establish a bootstrap-result baseline for a genuinely restored, eligible outbox; do not assume a pending generation already established one. Keep post-click live heads out of that baseline. Add actual-hook outcome tests for both sections and sort with a distinct newer outbox, unchanged readable head, fired debounce, and retained publication/outbox assertions, not just a scheduled timer.
2. [P2] A second click still freezes a head the UI never incorporated
wholeBlobSyncManager.ts:530-532, reached from the sections/sort mutation callbacks; observed-head updates are at 292-309 and 908-921, while hook application is suppressed at useChannelSections.ts:115 / useChannelSortPreference.ts:114.
Start with H100 applied, queue A1, then deliver/decrypt live or periodic H102 while A1 is pending. The manager records H102, but applyRemote deliberately keeps A1 visible. A second user edit A2 is authored from A1, not H102. Its normal publish(false) nevertheless freezes lastRemoteHead=H102. Preflight sees equality and publishes A2 over H102-only changes. Generation guards and retained-head confirmation do not prevent that incorrect replacement. Moving observation after decrypt fixes the previous gap, but successfully decrypted still does not mean incorporated.
Tie the editable baseline to the state actually incorporated into the edit (or preserve the pending baseline across edits that still exclude that head). Cover the two-click/suppressed-head sequence through both production hooks.
3. [P2] Failed-bootstrap replay misclassifies an in-session click as restored
useChannelSections.ts:164-195 / useChannelSortPreference.ts:161-192, and wholeBlobSyncManager.ts:533-536,667-680.
With no prior outbox, start bootstrap, click before it resolves, then let bootstrap fail. releaseDeferred schedules the fresh pending edit with its zero baseline. The hook immediately reads the outbox just written by that same click and republishes it with true, relabeling it as restored. If the 2s preflight recovers the pre-existing H100 before the initial reconciliation timer, the new flag disables the intended first-unknown-base exception and adopts H100, deleting the fresh click even though no competing peer wrote anything.
Do not reclassify/requeue the already-pending current-session generation as an old outbox. Preserve its provenance separately from a genuine restore. Test click-before-bootstrap-failure through both hooks and drive publication to completion; a manager-only call or clicking after failure misses this path.
4. [P2] Foreign-outbox reclamation still omits the preserved click from its proof
mergeLaneStorage.shared.ts:254-268, invoked by channelStarsStorage.ts:393-405 / channelMutesStorage.ts:383-395 after hook bootstrap.
Reclamation still calls isSubsumedBy(record.store, head) without record.preservedKey, so it selects the old lossy bounded-merge fallback. Use a full 500-entry head R lacking X and a foreign record containing 499 R entries plus reserved X. All entries have the same updatedAt, X has a higher rev but a lexicographically smaller ID than every R entry, so the unkeyed capacity bound evicts X and returns R, falsely proving subsumption. For a concrete durability failure, let bootstrap replay find this foreign record but fail to write its new own outbox (quota); writeOwnOutbox returns false, but the merge wrapper discards that result. Reclamation then deletes the foreign record even though the relay never retained X and no durable replacement exists. Quit before the pending timer and the click is gone. A foreign write between the replay snapshot and reclamation's separate enumeration is another path with no transfer at all.
Thread each enumerated record.preservedKey through the reclamation predicate as well. The existing new predicate already supplies the narrow proof needed. Add a full-capacity foreign-reclamation regression for both lanes, including failed transfer or a newly enumerated record. Bootstrap and ACK tests do not establish reclamation safety.
Bounded exit criteria
Preserve the existing version-1 payloads, whole-blob LWW policy, community/identity scoping, write-once ownership, and canonical relay ordering. Resolve the four traces above with production-hook/reclamation outcome coverage. Unchanged AppSidebar/AppShell consumers, signer/community remount boundaries, kind constants, and relay ACK/retention premises were checked against the previous reviewed source by blob identity. Native multi-device runtime and new mobile/web/CLI implementations are not validated by this review. No broader protocol migration, unbounded capacity retention, or unrelated hardening is requested. The new whole-blob suite manually invokes managers despite its “hook-layer” labels; the existing actual-hook replay check only tests timer presence. Those gaps support the defects above, not a separate testing blocker.
CRITICAL 1 (C1): restored replay must carry original queuedAt age.
- WholeBlobSyncManager.publish() now takes restoredQueuedAt param;
pendingRestoredQueuedAt drives the adopt-guard in
fetchOwnBlobBeforePublish (publish when remote.createdAt <=
restoredQueuedAt; adopt when strictly newer). writeOutbox called
with original queuedAt so on-disk stamp is never reminted.
- channelSectionsSync/channelSortSync publishSections/publishSortPrefs
accept and thread restoredQueuedAt through to publish().
- channelSectionsStorage/channelSortPreference writeOutbox signatures
updated to accept optional nowSecs override.
- Hook-layer tests in runWholeBlobCarlSuite use the actual React hook
(renderHook), driving the real bootstrap .then() callback, outbox
read, queuedAt, and shouldReplay guard.
CRITICAL 2 (C2): successful-bootstrap replay needs a baseline.
- bootstrap() snapshots the fetched head to this.bootstrapResultHead
before returning. publish(_, true) uses canonicalMax(current,
bootstrapResultHead) so a no-prior-pending restored replay gets the
correct baseline from the immutable bootstrap snapshot rather than
the mutable lastRemoteHead (which may carry a suppressed live peer
head H102 that must remain a genuine advance).
- New runWholeBlobC2Suite (manager layer) drives publish(_, true,
undefined) so C1's queuedAt guard is disabled and only C2's
bootstrapResultHead mechanism can prevent the adopt.
CRITICAL 3 (C3): confirmRetainedHead decrypt gap.
- For a foreign winner, lastRemoteHead is advanced only AFTER
decryptAndParse succeeds (mirrors fetchRemoteBlob fix). A click
arriving during the confirm decrypt gap now sees lastRemoteHead as
{0,""} (or the pre-confirm value) and correctly adopts the foreign
winner rather than publishing over it.
- New runWholeBlobC3Suite (manager layer) gates the first decrypt call
and injects a click during the gap; asserts publishCalls === 1.
CRITICAL 4 (C4 / P3-reclaim): reclaimSubsumedOutbox drops preservedKey.
- mergeLaneStorage.shared.ts reclaimSubsumedOutbox signature updated
to accept optional preservedKey; calls isSubsumedBy(record.store,
head, record.preservedKey) so a capacity-bounded proof can never
evict the reserved channel and falsely certify retention.
- New P3-reclaim tests (already in mergeLaneSyncCarl.shared.test.mjs)
exercise 500-entry relay head + foreign outbox with clicked channel
absent; mutation (drop preservedKey) → record deleted → test red.
IMPORTANT: test coverage.
- P1/C1 hook tests now drive renderHook with the real outbox read,
queuedAt threading, shouldReplay guard, and bootstrap .then()
callback (not manager.publishReplay directly).
- P3-ack fixture clock set to t=1s so clicked channel is the
lexicographically evicted entry at 501-entry capacity, making the
pendingPreservedKey-drop mutation genuinely causal (2 red).
- All 6 named mutations verified red on exactly their tests, green
on fixed source: C1 (2r), C2 (2r), C3 (2r), C4/P3-reclaim (2r),
P3-ack (2r), P2a-1 (2r), P2b (2r). 57/57 pass on fixed source.
Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
…s and MINOR adopt-clear fix IMPORTANT 1 — add C1-stale actual-hook test (failed-bootstrap stale replay must adopt, not publish). Causality: drop !pendingIsRestoredReplay guard → exception fires → publishes stale edit over newer head → 2 reds (sections + sort). IMPORTANT 2 — assert original queuedAt persists in transferred v2 outbox before debounce fires. Causality: replace restoredQueuedAt with undefined in writeOutbox call → stamps wall-clock value → 2 reds (sections + sort). IMPORTANT 3 — add runWholeBlobP2a1HookSuite driving blocked-bootstrap sequence through the real hook: bootstrap parked → click via hook API → H102 live → release bootstrap → real .then() reads/replays outbox naturally → debounce → H102 adopted as genuine advance. Causality: publishBaseline = lastRemoteHead in publish(_, true) → H102 folds into baseline → publishes over H102 → 2 reds (hook: sections + sort). MINOR — extract clearPendingState() helper shared by discardPending() and adoptRemote(), so adoptRemote's gen===pendingGeneration branch also clears pendingIsRestoredReplay and pendingRestoredQueuedAt on adopt. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
…rl hook tests P1/C1-stale: after adopt completes, assert (a) lane storage equals the adopted remote store, and (c) readOutbox returns null. Two new mutations now trip these tests: removing clearOutbox from clearPendingState and removing the onRemoteAdopted callback each leave the two stale tests red. P2a-1 hook: after H102 adopt completes, assert lane storage equals H102's store and the click outbox is cleared. The dropped-callback mutation turns the two hook tests red. runWholeBlobP2a1HookSuite signature extended with storageKey and readOutbox; callers (sections, sort) pass readOutbox from their respective lane modules. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Three assertion-only additions to close Thufir's pass-4 BLOCK:
1. runWholeBlobCarlSuite + runWholeBlobP2a1HookSuite each gain an
assertHookState parameter (lane-specific hook oracle). Both callers
(sections: .sections/.assignments; sort: .sortModeFor("remote"))
pass assertions that target hook.result.current after the act() flush.
2. The four named test sites call assertHookState after the storage and
outbox assertions, independently verifying React state. The erroneous
"collapsed into storage" comment is removed.
3. Removed the applyRemote-returns-prev class of regression: if the
callback writes storage but returns stale React state the hook tests
now go red while the storage assertions stay green — exactly the
mutation Thufir proved in his disposable tree.
4. wholeBlobSyncCarl.shared.test.mjs trimmed from 1009 to 995 lines by
collapsing redundant inline comments (already covered in JSDoc).
All touched files ≤1000 lines (wc -l).
Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
…l P2) wholeBlobSyncManager.startPendingEdit() unconditionally reseeded publishBaseline from mutable lastRemoteHead on every non-replay call. Sequence: A1 click pending → live H102 suppressed by hasPendingEdit → A2 second click → lastRemoteHead (=H102) folded into baseline → preflight saw no advance → A2 silently overwrote H102-only changes. Contract violation per JSDoc :51-55/:163-174. Fix: reseed publishBaseline only when (a) starting a fresh pending sequence (wasIdle) or (b) lastRemoteHead holds our OWN in-flight published attempt (lastIsOwnAttempt: eventId ∈ ambiguousAttemptIds). Case (b) is required so foldSupersedingAttemptWinner can repair the baseline on a same-second collision. A suppressed live remote head is never in ambiguousAttemptIds → lastIsOwnAttempt=false → baseline is preserved across all pending phases (debounce, preflight, publish, confirmation wait). Replay path (isRestoredReplay) semantics unchanged. New tests: wholeBlobSyncCarlP2SecondClick.shared.test.mjs, Variant A (debounce-window) and Variant B (in-flight), wired into both lane test files. Required mutations: timer-phase revert turns Variant B red; unconditional-reseed revert turns all four red. Same-second collision test remains green under both mutations. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
wesbillman
left a comment
There was a problem hiding this comment.
Carl, an automated reviewer, commenting via Wes’s GitHub account.
Changes requested: two remaining P2 data-loss paths
Reviewed head ffda3579a5e61af7073f747e257ed2a4b5509755 against base ef0d2025683869418e8eee22ac5b5ac16c5198b7, with corrective comparison to 31510eabb18207dce052c3fbe3afdf904c01d35c. This is a read-only source review; I did not check out or execute PR code, run tests, or claim runtime validation.
1. P2: Replay age must not override an observed post-click remote advance
Anchor: wholeBlobSyncManager.ts:655–667.
The new restored-replay branch executes even after a successful bootstrap and ignores whether a competing head was independently observed. A concrete synchronized-clock sequence through either production hook:
- Bootstrap is waiting to return H100 (
created_at=100). The user edits at wall second 101; its own outbox recordsqueuedAt=101. - Later in that same second, peer H101 is retained and delivered live. It decrypts successfully, but the hook suppresses applying it while the edit is pending.
- Bootstrap returns H100. The hook rereads its current-session outbox and calls
publishSections(store,true,101)/publishSortPrefs(store,true,101). The repaired baseline correctly remains H100. - Preflight finds H101 and correctly detects an advance, but
101 <= pendingRestoredQueuedAtoverrides adoption. The client signs above H101 and publishes a blob that lacks H101’s peer-only changes.
This regresses the previously repaired bootstrap/live-head contract without clock skew. The override also affects a genuinely restored record: resume an outbox queued at 101 against H100, then observe peer H101 later in the same second while that replay is pending. Preserving fresh-session provenance alone does not protect that replay. Preserve genuine observed-advance arbitration; age-based replay eligibility cannot establish that a head was incorporated. The actual-hook P2a-1 regression currently uses queuedAt=50 and a live head at 200 (wholeBlobSyncCarl.shared.test.mjs:638–659,719–721), which avoids this branch. Add the equal-second witness through both real hooks and assert the adopted UI/cache and no overwrite.
2. P2: Bootstrap completion still relabels a fresh pending edit as restored
Anchors: useChannelSections.ts:178–196, mirrored in useChannelSortPreference.ts:175–193; engine wholeBlobSyncManager.ts:529–532,642–670.
Prior finding #3 is only masked when the newly fetched head’s timestamp is at or below the click’s wall-clock stamp. Start with no old outbox and no observed baseline. While bootstrap is pending, make a fresh click at wall second 100, then fail bootstrap. Its callback rereads that click’s own outbox and republishes it as (store,true,100), disabling the fresh first-unknown-base exception. If preflight discovers already-retained H101, the age branch fails (101 > 100), so adoption deletes the fresh pending edit and outbox without publishing it.
H101 can predate this click on synchronized clocks: another window can legitimately publish clampPublishCreatedAt(100,100)=101 (sidebarSyncWatermark.ts:45–52). This is the implementation’s normal timestamp bump, not the accepted clock-skew residual. Leaving the already-pending current-session generation intact preserves its first-unknown-base behavior; turning it into a restored generation does not. Preserve that provenance rather than compensating with timestamps. Cover a fresh click before bootstrap failure through both hooks, through preflight and completion; a preseeded old outbox or a manager click after bootstrap failure does not exercise this seam.
Credited repairs and stable exit criteria
The successful restored-outbox bootstrap baseline initialization, ordinary second-click baseline preservation, and per-record preservedKey forwarding during foreign merge-outbox reclamation are repaired in the inspected source. The confirmation decrypt ordering is also improved. Keep those repairs and the existing version-1, whole-blob/per-entry conflict contracts. Resolve only the two causal sequences above with production-hook regression coverage; no general clock, mixed-fleet, legacy, or capacity redesign is requested.
Problem
Channel sidebar state — sections, sort preference, stars, and mutes — syncs per identity as kind
30078relay events. That state diverged between a user's devices and sometimes never self-healed. The main gaps: a local edit that lost a conflict was silently republished as remote content while the UI kept showing the edit; edits made inside the publish debounce were dropped on quit or community switch; a skewed remote head could push publishedcreatedAtpast the relay's future-drift window and wedge all later publishes; and stale-at-open state waited for a reconnect a healthy socket never fires.What this changes
Convergence. Sections and sort resolve whole-blob conflicts via LWW: when a local edit loses, the manager adopts the winning remote head (writes it through to state, storage, and the watermark) instead of republishing it, unifying with the relay's OK-false conflict path as one mechanism. Stars and mutes move to a per-entry Lamport
revwith a single max-merge on every path (ordered byupdatedAt, thenrev, then the starred/muted-true leaf); a click strictly dominates every state its replica has observed, so it cannot lose to a same-second remote write. Payloads stayversion: 1so older builds keep parsing.Durability. Every edit is persisted synchronously to a durable per-identity localStorage outbox and resumed on next mount, so edits survive quit, community switch, and the debounce window. Outbox records carry write-once ownership tokens, so one window's completing publish cannot clear another window's still-unpublished edit.
Publish correctness. The pre-publish no-op guard compares the pending store only against the freshly fetched readable relay head, so matching a stale previously-published value never suppresses a publish. Published
createdAtis clamped inside the relay's drift window. Publishes use generation-CAS ownership with single-flight and completion re-drive, retained-head confirmation on ACK, bounded 2s→30s retry backoff, and a reconciliation loop (steady 60s, backoff on failure, refresh on window visibility).Stale-outbox replay guard (P1). A durable outbox entry is only replayed if the relay head it was authored against is still the relay head at resume time. When the relay has advanced, the stale entry is discarded and the current remote state is adopted. This closes the path where a previously-lost edit was re-published above a newer relay head on next mount.
Bootstrap/live-head ordering (P2b). The pre-publish fetch baseline is derived from the bootstrap fetch result directly, not from the mutable
lastRemoteHeadfield thatsubscribeLivecan update during an in-flight bootstrap. A live peer head arriving after the click is held as a genuine remote advance rather than folding into the publish baseline.Queue-until-bootstrap (P2a). When a publish is debounced while bootstrap is still in flight, the edit is queued rather than fired immediately. On bootstrap completion,
releaseDeferredpublishes the queued edit usingcanonicalMax(queueTimeBaseline, bootstrapResultHead)as the baseline — preserving the queue-time baseline when bootstrap returns an older head, and never regressing a click that was authored from a live peer head that fully applied before the click. Three additional invariants: the absent-bootstrap seed call is a no-op when a pending edit is already queued (pendingStore === nullguard); failed-bootstrap base establishment usescanonicalMaxagainst{0,""}so the first observed head always wins; and abootstrapFailedExternalHeadObservedflag disarms the pre-publish exception once any peer head has been independently observed, allowing a genuine post-click peer advance to be adopted rather than published over.Merge-lane key threading (P3). Stars and mutes pre-publish fetch, reconnect fetch, and restart fetch all thread the retained event ID through the merge, so the key used for the incoming-payload merge is always consistent with the key retained in the watermark after publish.
Structure. All four lanes share two engines:
wholeBlobSyncManager.ts(sections, sort) andmergeLaneSyncManager.ts(stars, mutes); each lane manager is a thin config wrapper. Legacy preference migration proves ownership of the legacy key at read time, and retained legacy whole-blob replay is one-shot per value.Tests. Lane test suites are consolidated into shared parameterized suites (
wholeBlobSync.shared,wholeBlobSyncP2a.shared,mergeLane*.shared) plus helper modules, covering the new invariants (durable resume, multi-window outbox, cross-lane isolation, tie-breaks, migration ownership, bootstrap/live-head ordering, queue-until-bootstrap state machine, failed-bootstrap recovery) while shrinking the sidebar test surface from 6,303 to 5,860 lines. Each production defect has a named causal regression (T1–T6) executed through both concrete lane managers (ChannelSectionSyncManager,ChannelSortSyncManager).