Cache flattenNodes() in View to eliminate redundant tree walks

During token streaming, every tree update caused multiple O(n) tree
traversals: once inside the View's _onTreeUpdate(), then again for
each React hook subscriber calling view.flattenNodes(). With rapid
streaming tokens this multiplied into a significant performance cost.

The View now caches the flattenNodes() result in a _cachedNodes field.
The public flattenNodes() returns this cache in O(1). A new private
_computeFlatNodes() method performs the actual tree walk and is called
only when the visible output may have changed: tree structural changes,
branch selection changes, fork auto-selection, and history page reveal.

The original design avoided caching ("no cache invalidation complexity,
at the cost of repeated traversals") because the result depends on
branch selection state. Caching is safe here because every mutation
path that can change the visible output refreshes the cache
synchronously before emitting events to consumers - JS single-threading
eliminates async staleness risks.

Author: VeskeR

docs/internals/glossary.md

@@ -118,8 +118,8 @@ A codec-provided component that assembles [decoder outputs](decoder.md#decoder-o
 
 ### Message materialization
 
-The act of producing a flat message list from the [conversation tree](conversation-tree.md) via [`flattenNodes()`](#flatten). `flattenNodes()` returns `MessageNode<TMessage>[]`. Every call rebuilds from scratch - there is no cached list - because the result depends on branch selection state. All consumers go through the view's `flattenNodes()`: React hooks, `send()` (for the HTTP POST body), `view.loadOlder()` (for pagination snapshots). See [Message lifecycle](message-lifecycle.md#why-no-cached-message-list).
+The act of producing a flat message list from the [conversation tree](conversation-tree.md) via [`flattenNodes()`](#flatten). `flattenNodes()` returns `MessageNode<TMessage>[]`. The View caches the result and returns it in O(1) on subsequent calls. The cache is refreshed when the tree structure changes (new nodes, deletions, selection changes, history reveal). All consumers go through the view's `flattenNodes()`: React hooks, `send()` (for the HTTP POST body), `view.loadOlder()` (for pagination snapshots). See [Message lifecycle](message-lifecycle.md#cached-message-list).
 
 ### Flatten
 
-`view.flattenNodes()` - the sole path from tree state to a message array. (`flattenNodes()` is internal to `TreeInternal`, not on the public `Tree` interface.) Walks the sorted node list, checks parent reachability and sibling selection, and returns the linear message sequence for the currently selected conversation path. See [Conversation tree: flatten](conversation-tree.md#flatten-producing-the-linear-path).
+`view.flattenNodes()` - the sole path from tree state to a message array. Returns the View's cached node list in O(1). The cache is rebuilt by an internal `_computeFlatNodes()` method that walks the sorted node list, checks parent reachability and sibling selection, and produces the linear message sequence for the currently selected conversation path. (`flattenNodes()` on `TreeInternal` does the actual tree walk; the View's public method returns cached results.) See [Conversation tree: flatten](conversation-tree.md#flatten-producing-the-linear-path).

docs/internals/message-lifecycle.md

@@ -118,13 +118,18 @@ Every `'update'` event triggers a full `flattenNodes()` call, which rebuilds the
 
 Each turn needs its own accumulator because events from interleaved concurrent turns would corrupt each other's message assembly - a text-delta from turn A would be accumulated into turn B's message.
 
-## Why no cached message list
+## Cached message list
 
-The tree is a DAG with branch selection state. The "current conversation" depends on which sibling is selected at each fork point. There is no single cached `TMessage[]` - every call to `flattenNodes()` rebuilds from scratch.
+The View caches the result of `flattenNodes()` in a `_cachedNodes` field. The public `flattenNodes()` method returns this cache in O(1). The cache is refreshed by `_computeFlatNodes()` - a private method that performs the actual tree walk - whenever the visible output may have changed:
 
-This is a deliberate tradeoff: no cache invalidation complexity, at the cost of repeated traversals. Since message counts are conversation-sized (tens to low hundreds), this is cheap.
+| Trigger                                                       | What refreshes the cache                              |
+| ------------------------------------------------------------- | ----------------------------------------------------- |
+| Tree structural change (new node, deletion, serial promotion) | `_onTreeUpdate()` calls `_computeFlatNodes()`         |
+| Branch selection change                                       | `select()` calls `_computeFlatNodes()`                |
+| Fork auto-selection after `send()`                            | `send()` auto-select path calls `_computeFlatNodes()` |
+| History page revealed                                         | `_releaseWithheld()` calls `_computeFlatNodes()`      |
 
-All consumers go through `view.flattenNodes()`:
+All consumers go through the cached `view.flattenNodes()`:
 
 | Consumer                                  | When it calls `flattenNodes()`                    |
 | ----------------------------------------- | ------------------------------------------------- |
@@ -133,4 +138,6 @@ All consumers go through `view.flattenNodes()`:
 | `send()` / `regenerate()`                 | To build the HTTP POST body's message history     |
 | `view.loadOlder()`                        | To snapshot the current tree state for pagination |
 
+Because all consumers read the cache, a tree update triggers one tree walk (inside the View), not one per consumer. React hooks calling `flattenNodes()` after an `'update'` event get the pre-computed result without a redundant traversal.
+
 See [Conversation tree](conversation-tree.md) for how `flattenNodes()` works. See [Codec interface](codec-interface.md#accumulator) for the accumulator's role. See [History hydration](history.md) for the history decode pipeline.

src/core/transport/view.ts

@@ -145,6 +145,13 @@ export class DefaultView<TEvent, TMessage> implements View<TEvent, TMessage> {
   /** Unsubscribe functions for tree event subscriptions. */
   private readonly _unsubs: (() => void)[] = [];
 
+  /**
+   * Cached result of the last flattenNodes computation. Public `flattenNodes()`
+   * returns this in O(1); internal callers use `_computeFlatNodes()` when a
+   * fresh tree walk is needed (structural changes, selection changes, history reveal).
+   */
+  private _cachedNodes: MessageNode<TMessage>[] = [];
+
   private _loadingOlder = false;
   private _processingHistory = false;
   private _closed = false;
@@ -159,8 +166,9 @@ export class DefaultView<TEvent, TMessage> implements View<TEvent, TMessage> {
     this._logger.trace('DefaultView();');
     this._emitter = new EventEmitter<ViewEventsMap>(this._logger);
 
-    // Snapshot initial visible state
-    this._updateVisibleSnapshot();
+    // Compute initial cache and snapshot visible state
+    this._cachedNodes = this._computeFlatNodes();
+    this._updateVisibleSnapshot(this._cachedNodes);
 
     // Subscribe to tree events and re-emit scoped versions
     this._unsubs.push(
@@ -186,6 +194,17 @@ export class DefaultView<TEvent, TMessage> implements View<TEvent, TMessage> {
 
   // Spec: AIT-CT9, AIT-CT11c
   flattenNodes(): MessageNode<TMessage>[] {
+    return this._cachedNodes;
+  }
+
+  /**
+   * Walk the tree and compute a fresh visible node list, applying branch
+   * selections and withheld-message filtering. Use this instead of the
+   * public `flattenNodes()` when the cache may be stale (structural
+   * changes, selection changes, history reveal).
+   * @returns A fresh array of visible nodes.
+   */
+  private _computeFlatNodes(): MessageNode<TMessage>[] {
     const nodes = this._tree.flattenNodes(this._resolveSelections());
     if (this._withheldMsgIds.size === 0) return nodes;
     return nodes.filter((n) => !this._withheldMsgIds.has(n.msgId));
@@ -255,7 +274,8 @@ export class DefaultView<TEvent, TMessage> implements View<TEvent, TMessage> {
     if (!selected) return; // unreachable: clamped is always in bounds
     this._branchSelections.set(groupRootId, { kind: 'user', selectedId: selected.msgId });
     this._logger.debug('DefaultView.select();', { msgId, index: clamped, selectedId: selected.msgId });
-    this._updateVisibleSnapshot();
+    this._cachedNodes = this._computeFlatNodes();
+    this._updateVisibleSnapshot(this._cachedNodes);
     this._emitter.emit('update');
   }
 
@@ -311,7 +331,8 @@ export class DefaultView<TEvent, TMessage> implements View<TEvent, TMessage> {
         const lastMsgId = result.optimisticMsgIds.at(-1);
         if (lastMsgId) {
           this._branchSelections.set(groupRoot, { kind: 'auto', selectedId: lastMsgId });
-          this._updateVisibleSnapshot();
+          this._cachedNodes = this._computeFlatNodes();
+          this._updateVisibleSnapshot(this._cachedNodes);
           this._emitter.emit('update');
         }
       } else {
@@ -584,7 +605,8 @@ export class DefaultView<TEvent, TMessage> implements View<TEvent, TMessage> {
       this._withheldMsgIds.delete(n.msgId);
     }
     if (nodes.length > 0) {
-      this._updateVisibleSnapshot();
+      this._cachedNodes = this._computeFlatNodes();
+      this._updateVisibleSnapshot(this._cachedNodes);
       this._emitter.emit('update');
     }
   }
@@ -621,10 +643,11 @@ export class DefaultView<TEvent, TMessage> implements View<TEvent, TMessage> {
     this._pinBranchSelections();
     this._resolvePendingSelections();
 
-    const nodes = this.flattenNodes();
+    const nodes = this._computeFlatNodes();
     const newIds = nodes.map((n) => n.msgId);
     const newMessages = nodes.map((n) => n.message);
     if (this._visibleChanged(newIds, newMessages)) {
+      this._cachedNodes = nodes;
       this._updateVisibleSnapshot(nodes);
       this._emitter.emit('update');
     }

test/core/transport/view.test.ts

@@ -1176,6 +1176,17 @@ describe('DefaultView', () => {
   // -------------------------------------------------------------------------
 
   describe('flattenNodes caching and reference stability', () => {
+    it('returns the same array reference on consecutive calls without intervening changes', () => {
+      tree.upsert('m1', { id: '1', content: 'hi' }, makeHeaders('m1'), 'serial-1');
+
+      const ref1 = view.flattenNodes();
+      const ref2 = view.flattenNodes();
+
+      // flattenNodes() should return a cached result - the same array
+      // reference - when nothing has changed between calls.
+      expect(ref2).toBe(ref1);
+    });
+
     it('preserves unchanged message references after a content-only update', () => {
       const msg1 = { id: '1', content: 'stable' };
       const msg2 = { id: '2', content: 'will-change' };