Skip tree walk for content-only updates during streaming

During token streaming, every tree update triggered a full O(n) tree
walk even though the tree structure hadn't changed - only a single
message's content was updated. With conversations of hundreds of
messages and tokens arriving at high frequency, this became the
dominant cost on the hot path.

The Tree now exposes a structuralVersion counter (on TreeInternal)
that increments on insert, delete, and serial promotion - but not
on content-only upsert of an existing node. The View compares this
counter against its last-seen version: when unchanged, it takes a
fast path that compares cached message references against the
snapshot in O(visible_count) instead of re-walking the tree.

A monotonic counter was chosen over alternatives (tagged events,
dirty sets, boolean flags) because it requires no public API change,
is multi-View safe (each View tracks its own last-seen version),
and cannot produce false negatives.

Author: VeskeR

docs/internals/conversation-tree.md

@@ -16,10 +16,11 @@ Note that serial order is not necessarily delivery order - messages published co
 ## Data structures
 
 ```
-_nodeIndex:     Map<msgId, InternalNode>        Primary index
-_sortedList:    InternalNode[]                  All nodes, sorted by serial
-_parentIndex:   Map<parentId, Set<msgId>>       Children of each parent
-_selections:    Map<groupRootId, index>         Selected sibling at each fork
+_nodeIndex:          Map<msgId, InternalNode>        Primary index
+_sortedList:         InternalNode[]                  All nodes, sorted by serial
+_parentIndex:        Map<parentId, Set<msgId>>       Children of each parent
+_selections:         Map<groupRootId, index>         Selected sibling at each fork
+_structuralVersion:  number                          Monotonic counter (see below)
 ```
 
 Each `MessageNode` stores:
@@ -50,6 +51,10 @@ Each `MessageNode` stores:
 
 Serial promotion handles the common case where a client inserts an optimistic message (null serial), then the server publishes it to the channel (with serial). The node moves from the end of the sorted list to its correct serial-order position.
 
+### Structural version
+
+The tree maintains a `structuralVersion` counter (exposed via `TreeInternal`) that increments on changes affecting the `flattenNodes()` output structure - node inserts, deletions, and serial promotions (which reorder the sorted list). Content-only updates (replacing an existing node's message) do not increment the counter. The [View](message-lifecycle.md#cached-message-list) uses this to skip full tree walks during streaming - when only message content changed, the cached node list is still structurally valid.
+
 ## Sibling groups and fork chains
 
 When a user calls `regenerate(msgId)` or `edit(msgId)`, the new message carries an [`x-ably-fork-of`](wire-protocol.md#branching-headers) header pointing to `msgId`. Messages that fork the same target (or transitively fork each other) form a **sibling group** - alternative messages at the same point in the conversation.
@@ -91,20 +96,20 @@ Sibling group resolution is cached per `flattenNodes()` call using a `resolvedGr
 
 The public `Tree` interface exposes:
 
-| Method | Returns |
-|---|---|
+| Method               | Returns                                              |
+| -------------------- | ---------------------------------------------------- |
 | `getSiblings(msgId)` | All messages in the sibling group containing `msgId` |
-| `hasSiblings(msgId)` | Whether the message has alternative versions |
-| `getNode(msgId)` | The `MessageNode` by msg-id |
-| `getHeaders(msgId)` | Headers for a specific message |
+| `hasSiblings(msgId)` | Whether the message has alternative versions         |
+| `getNode(msgId)`     | The `MessageNode` by msg-id                          |
+| `getHeaders(msgId)`  | Headers for a specific message                       |
 
 The following are on the `View`, not the public `Tree` interface:
 
-| Method | Returns |
-|---|---|
-| `flattenNodes()` | Linear message list following selected branches |
-| `select(msgId, index)` | Switch to a different sibling at a fork point |
-| `getSelectedIndex(msgId)` | Currently selected index in the sibling group |
+| Method                    | Returns                                         |
+| ------------------------- | ----------------------------------------------- |
+| `flattenNodes()`          | Linear message list following selected branches |
+| `select(msgId, index)`    | Switch to a different sibling at a fork point   |
+| `getSelectedIndex(msgId)` | Currently selected index in the sibling group   |
 
 ## Delete
 

docs/internals/message-lifecycle.md

@@ -125,10 +125,15 @@ The View caches the result of `flattenNodes()` in a `_cachedNodes` field. The pu
 | Trigger                                                       | What refreshes the cache                              |
 | ------------------------------------------------------------- | ----------------------------------------------------- |
 | Tree structural change (new node, deletion, serial promotion) | `_onTreeUpdate()` calls `_computeFlatNodes()`         |
+| Content-only update (streaming token)                         | `_onTreeUpdate()` shallow-copies the cached array     |
 | Branch selection change                                       | `select()` calls `_computeFlatNodes()`                |
 | Fork auto-selection after `send()`                            | `send()` auto-select path calls `_computeFlatNodes()` |
 | History page revealed                                         | `_releaseWithheld()` calls `_computeFlatNodes()`      |
 
+### Content-only fast path
+
+The tree exposes a [`structuralVersion`](conversation-tree.md#structural-version) counter that increments on insert, delete, and serial promotion - but not on content-only message updates. When `_onTreeUpdate()` sees the version unchanged, it skips the full tree walk entirely. The cached node list is still structurally valid because only a message reference changed on an existing `MessageNode`. The View compares each cached node's `.message` against the last-seen snapshot to detect which message changed, creates a new array reference (`[...cache]`) so React sees a state change, and emits `'update'`. This reduces the streaming hot path from O(total_nodes) to O(visible_count).
+
 All consumers go through the cached `view.flattenNodes()`:
 
 | Consumer                                  | When it calls `flattenNodes()`                    |
@@ -138,6 +143,6 @@ All consumers go through the cached `view.flattenNodes()`:
 | `send()` / `regenerate()`                 | To build the HTTP POST body's message history     |
 | `view.loadOlder()`                        | To snapshot the current tree state for pagination |
 
-Because all consumers read the cache, a tree update triggers one tree walk (inside the View), not one per consumer. React hooks calling `flattenNodes()` after an `'update'` event get the pre-computed result without a redundant traversal.
+Because all consumers read the cache, a structural tree update triggers one tree walk (inside the View), not one per consumer. Content-only updates (streaming tokens) trigger zero tree walks - only a reference comparison over visible messages. React hooks calling `flattenNodes()` after an `'update'` event get the pre-computed result without a redundant traversal.
 
 See [Conversation tree](conversation-tree.md) for how `flattenNodes()` works. See [Codec interface](codec-interface.md#accumulator) for the accumulator's role. See [History hydration](history.md) for the history decode pipeline.

src/core/transport/tree.ts

@@ -39,6 +39,14 @@ interface InternalNode<TMessage> {
 
 /** Internal tree surface used by View — not part of the public Tree API. */
 export interface TreeInternal<TMessage> extends Tree<TMessage> {
+  /**
+   * Monotonic counter that increments on structural changes (node insert,
+   * delete, serial promotion/reorder) but NOT on content-only updates
+   * (existing node's message replaced). Allows the View to skip full
+   * tree walks when only message content changed.
+   */
+  readonly structuralVersion: number;
+
   /**
    * Flatten the tree along selected branches into a linear node list.
    * The `selections` map provides the selected sibling's msgId at each
@@ -108,6 +116,13 @@ export class DefaultTree<TMessage> implements TreeInternal<TMessage> {
   /** Monotonically increasing counter for insertion sequence. */
   private _seqCounter = 0;
 
+  /** Incremented on structural changes; unchanged on content-only updates. */
+  private _structuralVersion = 0;
+
+  get structuralVersion(): number {
+    return this._structuralVersion;
+  }
+
   constructor(logger: Logger) {
     this._logger = logger;
     this._emitter = new EventEmitter<TreeEventsMap>(logger);
@@ -394,6 +409,7 @@ export class DefaultTree<TMessage> implements TreeInternal<TMessage> {
         // Re-sort: remove from current position, re-insert at correct position.
         this._removeSorted(existing);
         this._insertSorted(existing);
+        this._structuralVersion++;
       }
       this._emitter.emit('update');
       return;
@@ -415,6 +431,7 @@ export class DefaultTree<TMessage> implements TreeInternal<TMessage> {
     this._nodeIndex.set(msgId, internal);
     this._addToParentIndex(parentId, msgId);
     this._insertSorted(internal);
+    this._structuralVersion++;
     this._emitter.emit('update');
   }
 
@@ -437,6 +454,7 @@ export class DefaultTree<TMessage> implements TreeInternal<TMessage> {
 
     // Children are NOT deleted — they become unreachable in flattenNodes()
     // because their parent is no longer on the active path.
+    this._structuralVersion++;
     this._emitter.emit('update');
   }
 

src/core/transport/view.ts

@@ -152,6 +152,9 @@ export class DefaultView<TEvent, TMessage> implements View<TEvent, TMessage> {
    */
   private _cachedNodes: MessageNode<TMessage>[] = [];
 
+  /** Last seen tree structural version - used to distinguish content-only from structural updates. */
+  private _lastStructuralVersion = -1;
+
   private _loadingOlder = false;
   private _processingHistory = false;
   private _closed = false;
@@ -168,6 +171,7 @@ export class DefaultView<TEvent, TMessage> implements View<TEvent, TMessage> {
 
     // Compute initial cache and snapshot visible state
     this._cachedNodes = this._computeFlatNodes();
+    this._lastStructuralVersion = this._tree.structuralVersion;
     this._updateVisibleSnapshot(this._cachedNodes);
 
     // Subscribe to tree events and re-emit scoped versions
@@ -641,6 +645,27 @@ export class DefaultView<TEvent, TMessage> implements View<TEvent, TMessage> {
     // updates arriving during the async history fetch are still forwarded.
     if (this._processingHistory) return;
 
+    const currentVersion = this._tree.structuralVersion;
+
+    // Content-only fast path: the tree structure hasn't changed (no new
+    // nodes, deletions, or serial reorders), so the cached node list is
+    // still structurally valid. The tree mutated an existing node's
+    // .message in place - check if any visible message reference changed.
+    // JS single-threaded: structuralVersion cannot change between the
+    // check and the response within this synchronous handler invocation.
+    if (currentVersion === this._lastStructuralVersion) {
+      const changed = this._cachedNodes.some((node, i) => node.message !== this._lastVisibleMessages[i]);
+      if (changed) {
+        this._lastVisibleMessages = this._cachedNodes.map((n) => n.message);
+        this._cachedNodes = [...this._cachedNodes];
+        this._emitter.emit('update');
+      }
+      return;
+    }
+
+    // Structural update: full re-walk required.
+    this._lastStructuralVersion = currentVersion;
+
     // Pin selections for previously-visible nodes that now have siblings.
     // This prevents new forks (from other views' edits/regenerates) from
     // shifting this view to a branch the user didn't navigate to.

test/core/transport/view.test.ts

@@ -1158,6 +1158,26 @@ describe('DefaultView', () => {
       expect(ref2).toBe(ref1);
     });
 
+    it('does not re-walk the tree during a content-only message update', () => {
+      tree.upsert('m1', { id: '1', content: 'first' }, makeHeaders('m1'), 'serial-1');
+      tree.upsert('m2', { id: '2', content: 'second' }, makeHeaders('m2', 'turn-1'), 'serial-2');
+
+      // Capture the current cached state so the view has a baseline
+      view.flattenNodes();
+
+      const spy = vi.spyOn(tree, 'flattenNodes');
+      spy.mockClear();
+
+      // Content-only update: same msgId, different message content, no serial change
+      tree.upsert('m2', { id: '2', content: 'streaming token' }, makeHeaders('m2', 'turn-1'), 'serial-2');
+
+      // The view should detect this is a content-only update and skip the
+      // full tree walk - using the cached node list instead.
+      expect(spy).not.toHaveBeenCalled();
+
+      spy.mockRestore();
+    });
+
     it('preserves unchanged message references after a content-only update', () => {
       const msg1 = { id: '1', content: 'stable' };
       const msg2 = { id: '2', content: 'will-change' };