perf(chat): window long transcripts to fix input lag while streaming by jaeeyoungkim · Pull Request #831 · fathah/hermes-desktop

jaeeyoungkim · 2026-07-08T02:17:46Z

Problem

#748 reported input lag in long conversations; #769 fixed the paint half with content-visibility: auto on message rows. But the React-tree half remains: every transcript row still mounts, and each streaming delta re-runs the whole MessageList render, so filtering + reconciliation stays O(all rows).

In a session with ~770 rows this reliably reproduces: typing is smooth while the app is idle, and lags precisely while the agent is streaming a reply (deltas arrive many times per second; the per-delta O(n) work competes with keystroke/IME events on the main thread). CJK IME composition is hit hardest.

Fix

Render only the trailing TRANSCRIPT_WINDOW (100) visible rows. The head collapses behind a "Show N earlier messages" pill; each click reveals one more window. Expansion is stored as a count of extra rows, not an absolute index, so incoming streamed rows never shift what the user chose to reveal.
The window cut is nudged back past any contiguous tool-call/tool-result run, so a ToolActivityGroup is never split in half.
Dropped the [...messages].reverse().find() array clone that ran on every delta.

Windowing composes with #769: content-visibility keeps paint cheap inside the window; the window keeps the tree itself bounded.

Testing

New MessageList.test.tsx (5 tests): under-window renders all rows; long transcripts hide the head + newest kept; button expands one window per click and disappears at the end; tool runs never split at the boundary; empty streaming placeholders stay hidden.
npx vitest run src/renderer/src/screens/Chat/ — 20 files, 193 tests pass.
npx tsc --noEmit -p tsconfig.web.json --composite false — clean.
npx eslint on touched files — clean.
Manually verified against a real ~770-row session: typing during streaming no longer stutters; expanding history works; auto-scroll behavior unchanged (sentinel/scroll container untouched).

content-visibility (fathah#769) bounded the paint cost of off-screen rows, but every row still lives in the React tree: each streaming delta re-runs the transcript render, so reconciliation stays O(all rows). In long sessions (hundreds of messages) that competes with typing on the main thread — the remaining half of the fathah#748 input lag, felt precisely while the agent is streaming a reply. Render only the trailing TRANSCRIPT_WINDOW (100) rows and collapse the head behind a 'Show N earlier messages' button (one more window per click). The cut is nudged back so a tool-call run is never split through a ToolActivityGroup. Also drop the [...messages].reverse().find() clone that ran on every delta. Expansion state is kept as a count of extra rows rather than an absolute index, so new streaming rows never shift what the user chose to reveal.

greptile-apps · 2026-07-08T02:19:21Z

Greptile Summary

This PR addresses the React-tree half of the #748 input lag issue: long transcripts now mount only the trailing 100 rows by default, with earlier rows revealed on-demand via a "Show N earlier messages" pill or by scrolling to the top. A second change coalesces streaming delta commits to one requestAnimationFrame per burst, cutting setMessages calls from O(deltas) to O(frames). A new useTranscriptState hook provides a write-through ref that ensures every transcript writer (user turns, /btw, clear, failures) always builds on the newest in-flight snapshot rather than a stale React commit.

Windowing: windowStart is clamped to lastVisibleBubbleIndex (so the approval bubble is never cut out), nudged past contiguous tool runs within a bounded walk, and informed by beforeWindow so mid-turn cuts don't spawn duplicate avatars. extraRows resets on conversation change. IntersectionObserver enables scroll-driven expansion.
Delta coalescing: message.delta, thinking.delta, and similar high-frequency events write only to messagesRef and schedule at most one rAF flush; lifecycle events (message.complete, tool boundaries) cancel any pending frame and commit synchronously.
useTranscriptState: wraps useState so every functional updater is resolved against the live ref at dispatch time, preventing the dropped-chunk class of bug (Bug: Message Text Truncation in Streaming Responses #757 and its coalesced twin).

Confidence Score: 5/5

Safe to merge — the windowing, coalescing, and write-through ref changes are all consistent with each other, and a thorough test suite exercises the corner cases that were flagged in earlier review rounds.

All issues raised in prior review rounds (approval-bar disappearing behind trailing tool/reasoning rows, expansion carrying over across conversations, spurious avatars at the window cut, mixed-language locale) are explicitly fixed and covered by new tests. The delta coalescing, write-through ref, and scroll-restore logic have their own tests that verify the specific failure scenarios documented in the PR. No new correctness gaps were found in this pass.

No files require special attention — the most complex logic lives in MessageList.tsx and useDashboardChatTransport.ts, both of which are well-tested.

Important Files Changed

Filename	Overview
src/renderer/src/screens/Chat/MessageList.tsx	Core windowing logic: per-render window cut with bubble-preservation clamp, tool-run nudge with bounded walk, beforeWindow avatar fix, IntersectionObserver-driven auto-expansion, and scroll-restore useLayoutEffect — all addressed cleanly and covered by the new test suite.
src/renderer/src/screens/Chat/hooks/useTranscriptState.ts	New hook that wraps useState with a write-through ref; functional updaters are resolved against the ref at dispatch time, ensuring coalesced flushes can never resurrect a stale transcript.
src/renderer/src/screens/Chat/hooks/useDashboardChatTransport.ts	rAF-based delta coalescing added: high-frequency events defer to scheduleDeltaFlush; lifecycle events call flushDeltasNow to cancel pending frames and commit immediately. Error paths use setMessages directly without explicit flush cancellation, but the write-through ref ensures the subsequent rAF flush publishes the same failed state — a no-op extra dispatch via React's same-reference bailout.
src/renderer/src/screens/Chat/hooks/useChatScroll.ts	Adds initialScrollDoneRef to detect history (re)loads and scroll instantly instead of smoothly, preventing the race between the smooth multi-frame scroll and the IntersectionObserver's first auto-expand callback.
src/renderer/src/screens/Chat/MessageList.test.tsx	Comprehensive new test file covering: under-window, windowing, per-click expansion, tool-run no-split, bounded nudge, conversation reset, IntersectionObserver auto-expansion, approval-bar preservation, avatar grouping at the cut, and the never-window-away-the-newest-bubble regression.
src/renderer/src/screens/Chat/hooks/useDashboardChatTransport.test.tsx	New delta-coalescing test suite: burst-of-deltas coalesces to one commit, message.complete supersedes a pending frame, and a functional writer mid-frame builds on the ref without forking from stale committed state.
src/renderer/src/screens/Chat/Chat.tsx	Swaps useState + separate messagesRef effect for useTranscriptState; passes messagesRef directly to the dashboard transport instead of the messages array, eliminating the per-render adopt-back effect.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[messages prop] --> B[useTranscriptState]
    B --> C[messagesRef - sync source of truth]
    B --> D[messages - React committed state]

    subgraph DashboardTransport["useDashboardChatTransport"]
        E[gateway event arrives] --> F{isCoalescableDelta?}
        F -- yes: delta/thinking/tool.progress --> G[apply to messagesRef]
        G --> H[scheduleDeltaFlush - rAF, at most 1/frame]
        F -- no: start/complete/clarify/boundary --> I[apply to messagesRef]
        I --> J[flushDeltasNow - cancels pending rAF, commits immediately]
    end

    C --> E

    subgraph MessageList["MessageList render"]
        K[visibleMessages - filter empty bubbles] --> L[window math]
        L --> M[windowStart - clamp to lastVisibleBubbleIndex]
        M --> N[nudge back past tool run, bounded to 1 window]
        N --> O[windowedMessages.slice]
        O --> P[render rows]
    end

    D --> K
    H --> D
    J --> D

    subgraph Expansion["Scroll-driven expansion"]
        Q[IntersectionObserver on earlierMarkerRef] -- marker near top --> R[expandEarlier]
        S[button click] --> R
        R --> T[setExtraRows - renderedCountRef.current]
        T --> L
        T --> U[useLayoutEffect - restore scrollTop pre-paint]
    end

%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
flowchart TD
    A[messages prop] --> B[useTranscriptState]
    B --> C[messagesRef - sync source of truth]
    B --> D[messages - React committed state]

    subgraph DashboardTransport["useDashboardChatTransport"]
        E[gateway event arrives] --> F{isCoalescableDelta?}
        F -- yes: delta/thinking/tool.progress --> G[apply to messagesRef]
        G --> H[scheduleDeltaFlush - rAF, at most 1/frame]
        F -- no: start/complete/clarify/boundary --> I[apply to messagesRef]
        I --> J[flushDeltasNow - cancels pending rAF, commits immediately]
    end

    C --> E

    subgraph MessageList["MessageList render"]
        K[visibleMessages - filter empty bubbles] --> L[window math]
        L --> M[windowStart - clamp to lastVisibleBubbleIndex]
        M --> N[nudge back past tool run, bounded to 1 window]
        N --> O[windowedMessages.slice]
        O --> P[render rows]
    end

    D --> K
    H --> D
    J --> D

    subgraph Expansion["Scroll-driven expansion"]
        Q[IntersectionObserver on earlierMarkerRef] -- marker near top --> R[expandEarlier]
        S[button click] --> R
        R --> T[setExtraRows - renderedCountRef.current]
        T --> L
        T --> U[useLayoutEffect - restore scrollTop pre-paint]
    end

_{Reviews (7): Last reviewed commit: "fix(chat): address review — streaming ra..." | Re-trigger Greptile}

greptile-apps · 2026-07-08T02:19:27Z

    error: "Couldn't deliver your answer — the turn may have ended. Try again.",
  },
  thinking: "Thinking…",
+  showEarlierMessages: "Show {{count}} earlier messages",


Locale Key Falls Back

The new chat.showEarlierMessages label is only added to the English locale. In other supported locales, the new button falls back to English inside an otherwise localized chat, so long transcripts show mixed-language UI.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

…ar at cut, locales - isLast now keys off the newest visible bubble id instead of the trailing row index, so an approval prompt keeps its approve/deny controls when reasoning/tool rows stream after the bubble. - Expansion (extraRows) resets when the component is reused for a different conversation (first-message id as conversation identity), so a large expanded budget doesn't carry into the next long chat. - Avatar grouping at the window cut consults the hidden row above the cut, so a mid-turn cut no longer renders a duplicate avatar for one turn. - showEarlierMessages translated for all 11 non-English locales.

jaeeyoungkim · 2026-07-08T02:30:34Z

Addressed all four review comments in 728e77f:

P1 last-bubble marker: isLast now keys off the newest visible bubble id rather than the trailing row index — an approval prompt keeps its approve/deny bar when reasoning/tool rows stream after it. Covered by the new "keeps the last-bubble marker when tool/reasoning rows trail it" test.
P1 expansion crosses conversations: extraRows resets when the mounted component switches to a different conversation, using the first message id as conversation identity (transcripts are append-only, so it's stable within a session). Covered by "resets expansion when the conversation changes".
P2 avatar at the window cut: avatar grouping now consults the hidden row just above the cut (beforeWindow), so a mid-turn cut doesn't render a duplicate avatar. Covered by "does not fake a new turn at the window cut".
P2 locales: showEarlierMessages added to all 11 non-English locales.

Full Chat + i18n suites pass (208 tests), tsc/eslint clean.

jaeeyoungkim · 2026-07-08T02:49:07Z

12dd5ea adds scroll-driven expansion on top of the windowing:

An IntersectionObserver on the top marker auto-reveals the next window when the user scrolls near it (rootMargin: 300px preloads one step early, so scrolling up through history feels continuous — same UX as claude.ai).
Scroll anchoring: the position is snapshotted before rows are prepended and restored in a useLayoutEffect (pre-paint), so the content being read never jumps.
The "Show N earlier messages" button remains as a fallback for keyboard users and environments without IntersectionObserver.
Also clamps the window cut so the newest bubble is never sliced out by a long trailing run of reasoning/tool rows (it owns the approval bar / last-bubble marker).

10 tests now, incl. an IntersectionObserver stub test for the auto-expand path. Chat suite 198 passing, tsc/eslint clean.

Greptile round 2 (P1, Last Bubble Can Disappear): lastVisibleBubbleId is computed from visibleMessages, but rows render from windowedMessages. When an approval bubble is followed by more than a full window of reasoning/ tool rows, the naive cut slices that bubble out while its id still wins the scan — no rendered row receives isLast and the approval controls vanish while the turn is waiting for input. Compute the last-bubble index and clamp windowStart to it, so the actionable bubble always stays rendered (the tool-run nudge then walks further back as before). Regression test included.

An IntersectionObserver on the top marker reveals the next window when the user scrolls near it (rootMargin 300px preloads a step early, so scrolling up through history feels continuous). The scroll position is snapshotted before rows are prepended and restored in a layout effect (pre-paint), so the content being read never jumps. The 'Show N earlier messages' button stays as a fallback for keyboard users and engines without IntersectionObserver.

jaeeyoungkim · 2026-07-08T02:57:00Z

Round-2 P1 (Last Bubble Can Disappear) addressed in e05db97 — good catch, this was a real regression path in my round-1 fix:

lastVisibleBubbleId was scanned from visibleMessages, but rows render from windowedMessages: with more than a full window of reasoning/tool rows trailing an approval bubble, the bubble got sliced out while its id still won the scan, so no rendered row carried isLast and the approve/deny controls vanished mid-turn.
Fix: compute the last-bubble index and clamp windowStart to it, so the actionable bubble always stays rendered (the tool-run nudge then walks further back as before). Regression test: "never windows away the newest bubble behind a long trailing run".

Also pushed 3c0af19 (separate commit): scroll-driven expansion — an IntersectionObserver on the top marker auto-reveals the next window as the user scrolls up (rootMargin: 300px preloads a step early), with the scroll position restored pre-paint in a useLayoutEffect so the content being read never jumps. The button remains as a fallback for keyboard users and engines without IntersectionObserver.

Note: the branch was rewritten to split these into two clean commits (12dd5ea → e05db97 + 3c0af19); final tree content is identical to what was already verified. MessageList suite 10/10, Chat suite 198 passing, tsc/eslint clean.

Streaming events (message.delta, reasoning.delta, tool.progress, ...) arrive many times per second and each called setMessages, costing a full transcript reconciliation per event. With windowing this is bounded but still competes with typing/IME on the main thread while a turn streams. Deltas keep applying to messagesRef synchronously (no data loss; the ref remains the source of truth for the next event), but the React commit now rides one animation frame — render work becomes O(frames), independent of event rate. Lifecycle events (start/complete/clarify/tool boundaries) flush immediately and cancel any queued frame, so isLoading/toolProgress/ approval state never observe a stale transcript. The flush callback only publishes if messagesRef still holds the queued array — if a non-delta path (user turn, clear, clarify) took over meanwhile, it already committed newer state and the stale frame is a no-op.

jaeeyoungkim · 2026-07-08T03:38:55Z

e233752 adds the second half of the streaming-lag fix: delta coalescing.

Windowing bounds how much re-renders; this bounds how often. Streaming events fire many times per second and each called setMessages → a full reconciliation per event, competing with keystrokes/IME on the main thread for the whole turn.

Deltas (message.delta, reasoning.delta, tool.progress, tool.generating, thinking.delta) still apply to messagesRef synchronously — the ref remains the source of truth for the next event, so no chunk is ever lost (Bug: Message Text Truncation in Streaming Responses #757-safe) — but the React commit rides one requestAnimationFrame. Render work becomes O(frames), independent of event rate.
Lifecycle events (message.start/complete, clarify.request, tool boundaries) flush synchronously and cancel any queued frame, so isLoading/toolProgress/approval state never observe a stale transcript.
The queued flush publishes only if messagesRef still holds the queued array; if a non-delta path (new user turn, clear, clarify resolve) took over meanwhile, it already committed newer state and the stale frame no-ops.

2 new tests with a controllable rAF stub: a 5-delta burst commits once with full text; message.complete before the frame commits immediately, cancels the queued flush, and a stale frame firing later can't resurrect pre-complete state. Chat suite 200 passing, tsc/eslint clean.

…g bounds - Route all transcript writers through a write-through ref (useTranscriptState): functional updates resolve against the ref at dispatch time, and the coalesced flush publishes the ref instead of a pinned snapshot. Fixes two races where a mid-frame writer (/btw, failure marking, clarify) could permanently drop streamed chunks or be erased by a stale frame. Replaces the transport's adopt-back effect and guard. - History loads scroll to the bottom instantly: a smooth multi-frame scroll raced the auto-expand observer, which aborted it and stranded the view mid-transcript with no self-heal. - Bound the tool-run nudge to one window so a pathological run of hundreds of contiguous tool rows splits instead of defeating the windowing cap, and derive each expansion budget from the current effective cut so every click makes progress. - Recreate the auto-expand observer per expansion: IntersectionObserver only reports transitions, so a short reveal (collapsed tool group) left it silent and auto-load stalled. - Count hidden bubbles, not raw rows, in the "show earlier" label. - Update lat.md (chat-performance windowing/coalescing, chat-commands streaming ref invariants) to match. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

greptile-apps Bot reviewed Jul 8, 2026

View reviewed changes

Comment thread src/renderer/src/screens/Chat/MessageList.tsx Outdated

jaeeyoungkim added 2 commits July 8, 2026 11:55

jaeeyoungkim force-pushed the fix/transcript-windowing branch from 12dd5ea to 3c0af19 Compare July 8, 2026 02:57

fathah and others added 2 commits July 12, 2026 14:37

Merge branch 'main' into pr/831

27c4cb1

Navigation Menu

perf(chat): window long transcripts to fix input lag while streaming - #831

perf(chat): window long transcripts to fix input lag while streaming#831
jaeeyoungkim wants to merge 7 commits into
fathah:mainfrom
jaeeyoungkim:fix/transcript-windowing

jaeeyoungkim commented Jul 8, 2026

Uh oh!

greptile-apps Bot commented Jul 8, 2026 •

edited

Loading

Flowchart

Uh oh!

Uh oh!

Uh oh!

Uh oh!

greptile-apps Bot Jul 8, 2026

Uh oh!

jaeeyoungkim commented Jul 8, 2026

Uh oh!

Uh oh!

jaeeyoungkim commented Jul 8, 2026

Uh oh!

jaeeyoungkim commented Jul 8, 2026

Uh oh!

jaeeyoungkim commented Jul 8, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

jaeeyoungkim commented Jul 8, 2026

Problem

Fix

Testing

Uh oh!

greptile-apps Bot commented Jul 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Greptile Summary

Confidence Score: 5/5

Important Files Changed

Flowchart

Uh oh!

Uh oh!

Uh oh!

Uh oh!

greptile-apps Bot Jul 8, 2026

Choose a reason for hiding this comment

Uh oh!

jaeeyoungkim commented Jul 8, 2026

Uh oh!

Uh oh!

jaeeyoungkim commented Jul 8, 2026

Uh oh!

jaeeyoungkim commented Jul 8, 2026

Uh oh!

jaeeyoungkim commented Jul 8, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

greptile-apps Bot commented Jul 8, 2026 •

edited

Loading