Design: "following" feed subscription strategy (fanout-on-read vs. fanout-on-write)

[iduartgomez]

Open design question split out from tracking #8.

Question

How does a client assemble the "following" timeline?

Options

• Fanout-on-read (subscribe to N contracts): client subscribes to each followed user's posts contract and merges locally. Scales linearly with follow count; cold-start is expensive for users following many accounts.
• Fanout-on-write (follower contract): writer pushes a notification to each follower's inbox contract on publish. Cheap read, expensive write — bad for users with many followers.
• Hybrid: fanout-on-read for normal accounts, fanout-on-write for high-follower accounts, or vice versa.

Considerations

• Depends on the outcome of the per-user-vs-global design issue — if posts stay global, this question collapses.
• Affects notification design (the Webapp notifications bullet uses the same machinery).

Outcome

Decision recorded here, then drives the Discovery workstream subscription model and the Webapp notifications design.

Parent: #8

[iduartgomez]

Decision (#19): Fanout-on-read for the following timeline

The client assembles its "following" timeline by subscribing to each followed user's user-shard and merging posts locally. No copies of posts are pushed into follower-owned contracts. Above a configurable subscription cap the overflow tail degrades to bounded periodic re-GET without a live subscription, never to fanout-on-write.

Cap:

• Live-subscribe to up to an initial configurable cap N (starting value 500) followed user-shards. N is a product hypothesis, not a proven ceiling — see "empirical gates" below.
• Above N: subscribe to the most-recently-active N, and cover the tail with a bounded poll scheduler (spec below), not a blind tail_size-GETs-per-cadence loop. The tail is eventually-consistent, not live.

N and the live-vs-poll behaviour are empirical gates, revisited once the live-UPDATE seam (below) is fixed and real subscription-count behaviour is measured on a live multi-node network.

---

Rationale

Why fanout-on-read, and why not fanout-on-write as the default

Fanout-on-write (on publish, the author pushes a post copy/notification into each follower's inbox) is architecturally viable in Raven — ADR-0001 explicitly says the inbox-shard plus a delegate outbox make it viable, including delivered post copies. It is not prohibited by the ownership model; it is rejected here as the default on cost, reverse-index, abuse, and operational-complexity grounds:

• Write amplification / celebrity-hostile. M followers = M inbox-shard UPDATEs per post, M = follower count. At M = 10k that is 10k recipient-contract writes fanned out from one publish. Fanout-on-read inverts this: a 10k-follower account does zero extra work on publish; each follower pays one subscription they already wanted. The expensive axis scales with follow count (a number the reader chose), not follower count (a number the author cannot bound).
• Requires a reverse index Raven does not hold. Fanout-on-write needs the writer to enumerate every follower's VK — a follower→author reverse registry Raven has no contract for and would itself need a convergent surface to maintain. Fanout-on-read consumes the reader's own user-shard.follows CRDT, which the reader already holds and can derive keys from unilaterally.
• Turns inbox capacity/abuse into timeline capacity/abuse. The inbox-shard is anyone-writes/owner-prunes and flood-exposed (the ADR's GhostKey abuse question is unresolved). Routing the feed through it couples timeline delivery to that abuse surface and to inbox windowing.
• Schema gap (engineering, not authority). The current NotifKind is {Reply,Mention,Follow,Quote} with a single ref_id — it cannot represent a generic "new post from someone you follow." Fanout-on-write would need a new notification/copy schema and a delivery protocol. That is a real cost, but a schema change, not an impossibility.

Fanout-on-write is deferred, not foreclosed — if celebrity-scale read cost or subscription counts prove unworkable, the inbox+delegate-outbox path remains open for a future hybrid.

Trade-off table (Raven's actual numbers)

Dimension	Fanout-on-read (chosen)	Fanout-on-write (deferred)
Read cost	N subscriptions + N cold-start GETs, N = follow count. GETs serialized through getChain (one in flight).	1 subscription (own inbox-shard); recipient hydrates refs on demand.
Write cost	Zero fanout on publish. Author writes only own user-shard (already does).	M inbox UPDATEs per post, M = follower count. 10k = 10k writes from the publisher per post.
Cold-start (500 follows)	500 GETs through the serialized getChain + 500 subscribes. The real cost; mitigated by incremental render, bounded scheduling, and global-index seeding.	Instant (one inbox read) — but every author you follow paid the M× write.
Privacy / render	Renders the authoritative signed Post from the author's shard. No content copied.	Copies content (or a ref needing a follow-up GET) into each follower's public-write inbox.
Liveness dependency	"Live" timeline needs live UPDATE-to-remote-subscriber, currently broken (poll-masked). Degrades gracefully to poll.	Own inbox is local, but a follow/post made on another node is still a remote update that must propagate to the subscriber — not immune to the same seam, just cheaper (1 vs N).
New contract state	None. Reuses existing user-shard reads.	New notification/copy schema + delivery discipline; risks a second feed store to keep convergent.
Key model fit	Reader derives each followed user-shard key from VK client-side, free (blake3(user_shard_wasm ‖ vk_bytes)).	Writer must enumerate follower VKs — no such index.

Decisive asymmetry: fanout-on-read moves cost to the party that wants the data, scales the expensive axis with follow count not follower count, and requires no new contract state. That is why it wins as the default even though fanout-on-write is expressible.

---

Contract impact

None. No shard changes — the storage decisions are already done.

• user-shard already holds posts (owner-writes, self-verifying, windowed to ~200) and follows (the per-key FollowState{seq, following} CRDT). Fanout-on-read reads both and needs no new field or merge rule.
• inbox-shard is not the timeline; it stays the notification surface. A Follow notification is still delivered on follow (so the followed user learns of it), but it is not how the follower's feed is built.
• global-index-shard is not the following feed. It is the public firehose (singleton, anyone-writes, cap 2000). Its "share to public timeline" is a client-side opt-in only — the contract enforces no author opt-in and accepts any self-verifying post; "top-level only" is likewise client convention, not enforced. It doubles as a best-effort cold-start seed for the following feed, never as a correctness mechanism.

No new CRDT is proposed, so there is no new join-semilattice to prove.

---

Client impact (freenet-api.ts + store layer)

Today the client is strictly single-contract-per-role: it subscribes to exactly one user-shard (the owner's own), the global-index singleton, and lazily to thread shards. followedPubkeys is an inert empty Set (App.svelte:53); the API type merely permits an opaque follows field — it does not parse FollowState today (freenet-api.ts:119). Feed.svelte's "following" tab currently just shows the owner's own posts. Wiring required:

1. Parse the follow set. Add real deserialization of the owner's user-shard.follows into a set of active follows — entries with following == true (tombstones excluded). This does not exist yet.
2. Derive keys. For each active-followed VK, compute deriveShardContractKey(__USER_SHARD_CODE_HASH__, hexToBytes(vk)) — the same pure derivation already used for the owner's own shard. No discovery step.
3. New routing branch (not free). The current GET/UPDATE handlers (freenet-api.ts:549, :603) recognize thread, then global-index, then only the owner's user-shard; a followed shard's response would currently be discarded. Add a followedShards: Map<instanceId, vk> and a provenance-aware branch so followed-shard GET/State updates route with their author identity attached. This mirrors the threadInstanceToRoot pattern but is a genuine new code path, not automatic compatibility.
4. Bounded fan-out subscribe with readiness/retry. Subscribing before a contract is cached silently fails — the global-index code already needs "successful GET, then subscribe" (freenet-api.ts:901). Each followed shard needs the same per-shard readiness state: GET → on success subscribe → track subscribed/failed, with retry. Do not fire N blind subscribes at boot.
5. Bounded poll scheduler (replaces the naive backstop). The 60s single-singleton global-index re-GET is fine for one contract; it is not a design for hundreds of shards through a FIFO that permits one outstanding GET (getChain, :365; a request can hold the queue ~30s). The scheduler is a background token-bucket over the serialized getChain, with these concrete starting budgets (all configurable, tuned against the empirical gates):
   • One poll round at a time, never overlapping. A round refreshes at most POLL_BATCH = 25 shards, chosen as the least-recently-refreshed active-followed shards.
   • Round cadence POLL_ROUND_MS = 15s → a working set of 25 subscribed-but-poll-covered shards refreshes ≈ every 15s; a 500-shard cold/tail set cycles fully in ≈ (500/25)×15s ≈ 5 min worst case. Live subscriptions, when the seam works, make most of this moot — the scheduler is the fallback, so a multi-minute tail refresh is acceptable.
   • GET concurrency = 1 (the getChain hard constraint); the scheduler paces enqueues so background polls never occupy more than POLL_QUEUE_SHARE ≈ 50% of a round's wall-time, leaving headroom for interactive reads.
   • Priority lanes: interactive reads (own shard, open thread, likes, an explicit feed refresh) preempt background follow polling; a background poll mid-flight is not cancelled but no new background GET is enqueued while an interactive one is pending.
   • Dedupe: a shard already mid-GET or enqueued this round is not re-enqueued.
   • Retry with exponential backoff + jitter on failure (base 2s, cap 5 min), so a dead shard doesn't burn budget.
   • Overrun = drop-and-reschedule: if a round can't finish within its cadence, the unfinished shards roll into the next round's LRU selection — rounds never pile up.
     These numbers are a starting policy, not load-tested — they move with N under the same empirical gate. The live subscription path is the primary freshness mechanism; this scheduler is the bounded fallback while the live seam is broken and for the >N tail.
6. Store model — per-author snapshots, not an id grow-set. The following feed is not a monotone grow-set union. Each author shard is a ~200-post window that can evict old posts on a later full-state GET, and the follow CRDT can remove an author (unfollow). So:
   • keep postsByAuthor: Map<vk, Post[]> snapshots;
   • replace an author's snapshot on each GET/State update (do not union-accumulate — that would resurrect evicted posts);
   • remove an author's snapshot immediately on unfollow;
   • derive the visible feed as the union over active authors only, deduped by content-address id, sorted (timestamp, id) desc.
     Note: the current client sorts by timestamp only (freenet-api.ts:593), so ties are non-deterministic — the merge layer must add the id tie-break to match the contract comparator (user-shard/src/lib.rs:147).
     Open sub-decision for Discovery: follows wiring, who-to-follow, search, trending, per-user feed sharding #13: whether the owner's own posts remain visible in "Following."
7. followedPubkeys feeds all of it. Populated from step 1, mutated by the follow/unfollow write path (Discovery: follows wiring, who-to-follow, search, trending, per-user feed sharding #13). Subscription set = active follow set; unfollow removes the author's snapshot and should drop its subscription — but note this repo has no confirmed unsubscribe API path today, so "drop on unfollow" may reduce to "stop polling + let the subscription lapse" until an unsubscribe capability is confirmed.

---

Dependencies / sequencing

1. The live-UPDATE-to-remote-subscriber seam gates the live property, not the decision. Cross-node GET works and is tested (multi-node journeys 2 & 4). Live UPDATE to a remote subscriber is currently not delivering: freenet-core drops the update broadcast when the subscription tree hasn't formed, retries BROADCAST_NO_TARGETS 3× then gives up with no anti-entropy re-send. In the multi-node tier, journey 3 converges at ~59s via the 60s poll re-GET, not the live notification (commit 6dbd602: "the poll, not the notification, is delivering cross-node updates; ... notification-key routing suspect"). Client-side halves are fixed (re-subscribe after GET db74aa3; parse State-shaped notifications 52078c8); the network push is unverified. There is no CI gate on it — the multi-node tier is testIgnored in the offline sweep (playwright.config.ts:5) and runs only under cargo make test-ui-multi-node. The inbox/notifications path is subject to the same seam — a follow/reply made on another node is a remote update to the recipient's subscription — so this is not a fanout-on-read-only dependency.

   Implication: ship fanout-on-read on poll-backed liveness (GET + the bounded scheduler above); it becomes live transparently once the freenet-core broadcast/notification-key-routing bug is fixed. Do not block follow-wiring (Discovery: follows wiring, who-to-follow, search, trending, per-user feed sharding #13) on that fix.

2. Follow write path first (Discovery: follows wiring, who-to-follow, search, trending, per-user feed sharding #13). followedPubkeys must actually be populated — follow button → user-shard Follow op, and reading the follows CRDT into the client (step 1 above). Until that lands the feed has nothing to fan out over. This is the real blocker, and it is UI/wiring + the follow-parse, not architecture.

3. Notifications coherence — one decision, not two. The Webapp notifications design uses the same subscribe/route plumbing: subscribe to the owner's own inbox-shard, drop the mock records in Notifications.svelte. That is one inbox subscription (cheap, but still remote-update-dependent per point 1). Keep them coherent: the follow-feed fan-out and the inbox subscription share the routing/readiness machinery added in Discovery: follows wiring, who-to-follow, search, trending, per-user feed sharding #13. A Follow notification flows through the inbox (being-followed is surfaced); the following feed is built by fanout-on-read. This decision fixes both halves of "follows" to the read-side model.

---

Open questions kicked to #13 / later

• Cap N and overflow policy. 500 is an initial configurable value, not a proven ceiling. The real limit is whatever a Freenet node sustains as concurrent per-contract subscriptions on a live network — unmeasured (ADR-0001 leaves high subscription counts open). Requires a live measurement gate.
• Multi-node test coverage (empirical gates). Add multi-node coverage for: follow → read a followed author's posts; remote update → live-or-poll convergence; unfollow → author removed from feed; overflow-tail fairness; and behaviour at the N-subscription load point. Today's journey suite explicitly accepts "live update / poll" and is excluded from normal CI — so none of these is currently gated.
• Cold-start UX for large follow sets. Serialized getChain makes N GETs sequential; incremental render + global-index seeding are the mitigations, but the perceived-latency budget and whether to add GET parallelism/batching to the api layer are open.
• Global-index seed is cosmetic acceleration only. Capped at 2000, no protocol-level author-opt-in guarantee (the contract accepts any self-verifying post; the "top-level only" property is client convention, not enforced), so it can pre-fill the feed for perceived speed but is not the tail's correctness mechanism.
• Unsubscribe capability. "Drop subscription on unfollow" assumes an unsubscribe path this repo does not yet demonstrate. Confirm or design it.
• Merged-feed windowing. Each author is ~200-windowed on their own shard; the reader's merged feed is unbounded across authors. Whether/how the client caps the merged following feed is a store decision for Discovery: follows wiring, who-to-follow, search, trending, per-user feed sharding #13.
• Writer-credential (GhostKey) gate interaction. Not on the read path, but if inbox Follow notifications become gated, the "you were followed" surface inherits that policy. Flagged for abuse-model coherence, not decided here.

---

Decision reviewed pre-recording via the design-review gate; revised to remove the "fanout-on-write is impossible" overstatement, specify a bounded poll scheduler and per-author store model, correct the global-index/privacy language, and mark N and live-behaviour as empirical gates.