feat(todo-state): realtime Todos panel + INFLIGHT recovery

[v2psv]

1. Problem

The Todos panel only repainted once per turn, after the agent settled and the tool message hit state.db. loadTodos() then reverse-scanned S.messages to find the latest todo write. Three user-visible consequences:

• Not realtime. A todo tool call mid-turn was invisible until the turn finished. The user saw a frozen list while the agent was actively crossing items off.
• State lost on reload / reconnect. Refreshing during a running stream wiped the panel until the next todo write happened. Stream errors / reconnects had the same effect.
• Occasional cross-session pollution. A late SSE event tagged for session A could land after the user had already navigated to session B and overwrite S.todos.

2. What changed

ecd8a247 feat(todo-state): frontend live updates + INFLIGHT recovery (Phase 2)
c347a11f feat(todo-state): server-side realtime todo state (Phase 1)

Based on master 5528e2c5. Two commits, 17 files, +5,504 / −20 lines.

• Production code: 7 files / +573 / −20.
• Tests: 10 files / +4,931 (8 new files).

Phase 1 — server-side realtime pipeline (c347a11f)

File	Change
api/todo_state.py (new, 235 lines)	Single source of truth for the wire protocol: derive_todo_state / emit_todo_state / attach_todo_state / parse_todo_tool_result / _normalize_snapshot. Constants EVENT_NAME / PAYLOAD_KEY / VERSION live here. Every error boundary swallows-and-degrades — never breaks tool delivery or session GET.
api/streaming.py (+36)	Both the legacy tool_progress_callback (event_type='tool.completed') path and the modern structured on_tool_complete path emit todo_state SSE events. Full snapshot, idempotent under run-journal replay.
api/routes.py (+21)	The session GET handler calls attach_todo_state on both the WebUI session branch and the CLI fallback so opening any session from the sidebar hydrates the Todos panel immediately.

Phase 2 — frontend live consumption + INFLIGHT recovery (ecd8a247)

File	Change
static/ui.js (+133/−4)	S.todos + S.todoStateMeta as the single source of truth. null is a sentinel that means "no signal seen — fall through to legacy reverse-scan", distinct from "signal seen, list is empty". _compactInflightState persists todos into the localStorage snapshot. _todosHash + _todosLastRenderedHash short-circuit identical re-renders. scheduleTodosRefresh coalesces bursty events through requestAnimationFrame. _hydrateTodosFromSession reconciles cold-load vs. INFLIGHT at every S.session= settle point.
static/messages.js (+43/−3)	todo_state SSE listener: swallow JSON.parse errors, validate Array.isArray(d.todos), drop on session_id !== activeSid, drop on S.session.session_id !== activeSid (double check — late events arriving after navigation), drop on incomingTs < currentTs, full-snapshot replace (never merge). 'todo_state' added to the run-journal cursor whitelist so Last-Event-ID advances past it on reconnect.
static/sessions.js (+18)	INFLIGHT restore on tab return / hard reload carries todos and todoStateMeta alongside the existing fields. Session-deletion paths call _hydrateTodosFromSession(null) so the panel clears synchronously.
static/panels.js (+84/−13)	loadTodos() trusts S.todos whenever S.todoStateMeta is set; otherwise falls through to _legacyTodosFromMessages() (reverse-scan over tool messages). Every user-controlled string goes through esc() before innerHTML.

Data flow

agent.todo()
   │
   ▼
streaming.py  ──emit_todo_state──▶  SSE: 'todo_state' (full snapshot)
   │                                          │
   ▼                                          ▼
state.db (tool message)              messages.js listener
   │                                          │
   ▼                                          ▼
routes.py session GET                S.todos / S.todoStateMeta
   │   ──attach_todo_state──▶ payload.todo_state    │
   ▼                                                 │
ui.js  _hydrateTodosFromSession  ◀── reconcile ─────┤
   │       (cold-load vs INFLIGHT, by ts)            │
   ▼                                                 │
panels.js loadTodos ──▶ DOM         localStorage ◀───┘

3. Performance

Render cost

• Hash short-circuit. _todosHash builds an (id, content, status) fingerprint via string concatenation (no intermediate object allocation in V8). Identical snapshots return early without touching the DOM.
• RAF coalescing. scheduleTodosRefresh collapses any number of todo_state events landing in the same animation frame into one loadTodos() call. The e2e suite asserts that 100 emits share a single RAF tick.
• Inactive-panel skip. _todosPanelIsActive() short-circuits the entire path when the Todos panel is hidden — background events do zero DOM work.
• Linear-scaling regression guard. Instead of a fragile absolute timeout, the test compares 50-item vs. 200-item render time and asserts the ratio stays < 8. An O(n²) regression would push the ratio toward 16.

Persistence budget (INFLIGHT_STATE_DEFAULT_LIMITS)

maxSessions: 8
messages:    24
toolCalls:   48
stringChars: 60,000   per string field
jsonChars:   1,500,000 per full snapshot

LRU by updated_at when the cap is hit. Quota errors trigger graceful degradation: drop everything except the active session, and if that still fails, clear the whole bucket. INFLIGHT entries older than 10 minutes are evicted on read.

Network footprint

Each todo_state event is the same JSON the todo tool already returns in its tool message, plus five metadata fields (session_id, stream_id, source, ts, version). It's a single per-event mirror — no diffs, no polling.

4. Reliability & stability

Error isolation boundaries

Boundary	Behavior
parse_todo_tool_result gets non-string / non-JSON / wrong shape	Returns None; emit and attach skip.
emit_todo_state raises in put	Swallowed, debug-logged, returns False. Tool delivery is unaffected.
attach_todo_state errors during message iteration (broken generators, corrupted store)	Swallowed, returns False. Session GET responds normally without the todo_state field.
Frontend JSON.parse failure	Listener returns; S.todos is unchanged.
localStorage corrupted	_readInflightStateMap returns {}; the next save rebuilds.
localStorage quota exceeded	Falls back to active-session-only; if still over, clears the bucket.
INFLIGHT entry stale (>10 min)	loadInflightState evicts and returns null.
Stream reconnects with a different streamId	loadInflightState returns null — a new run cannot inherit a stale snapshot from the previous one.

Ordering & merge invariants

• Full-snapshot protocol. Every emission carries the entire list, so SSE replay is idempotent — no merge, no diff reconciliation.
• Strict-older-ts drop, equal-ts allowed. A compression-source refresh can legitimately land on the same wall-clock second as the tool emission it follows.
• Cross-session double gate. Both payload.session_id !== activeSid and S.session.session_id !== activeSid drop the event. The latter prevents pollution when the SSE stream is still wired to the previous session but the user has already navigated away.
• Recency-aware hydrate. When both cold-load (server settled view) and INFLIGHT (locally persisted snapshot) are present, _hydrateTodosFromSession picks the one with the newer ts. This stops a stale cached cold-load from regressing fresher INFLIGHT state on reload of a still-running session.

Compatibility

• S.todoStateMeta = null is a sentinel; pre-Phase-1 backends produce no todo_state events, so loadTodos() falls through to _legacyTodosFromMessages() and the panel still works.
• version: 1 reserves room for future wire-format evolution.
• The run-journal whitelist entry means a reconnect's Last-Event-ID advances past prior todo_state events. Replay remains correct (idempotent) but doesn't re-trigger renders.

Test coverage

Category	File	Purpose
Protocol unit	test_todo_state.py (365) / test_todo_state_emission.py (407) / test_todo_state_robustness.py (283)	parse / derive / emit / attach contracts; malformed items, non-dict summary, concurrent-emit thread safety, ts propagation
Real scenarios	test_todo_state_scenarios.py (741)	Multi-write history, last-write-truncated fallback, large-history performance sentinel
Wiring (AST)	test_streaming_todo_state.py (158) / test_session_todo_state_route.py (130)	AST call-shape checks: pin call sites' kwargs and positional shape, not source strings
Frontend wiring	test_phase2_frontend_static.py (505)	Balanced-brace function-body extraction + AST-style regex; tolerates whitespace and harmless reformatting
Frontend behavior	test_phase2_todo_behavior.py (831)	Node driver loads real JS function bodies and runs unit-level assertions against them
Frontend scenarios	test_phase2_e2e_scenarios.py (1082)	Lifecycle / multi-session / event-robustness / render-scheduling end-to-end through the real JS
Persistence round-trip	test_phase2_inflight_persistence.py (449)	Real in-memory localStorage shim drives saveInflightState ⇄ loadInflightState round-trips, hard-reload recovery, cross-session isolation, journal-replay regression

280 tests pass in ~4s.

5. Out of scope

• No upstream Hermes Agent protocol change; todo tool semantics unchanged.
• No SSE infrastructure change; one new event name plus one journal-whitelist entry.
• No persistence-layer change (still localStorage, not IndexedDB); current budgets are far above what todo lists need.

---

Review follow-ups already absorbed in these two commits

• P1: messages.js cross-session double check; ui.js ts-aware reconciliation in _hydrateTodosFromSession; api/todo_state.derive_todo_state propagates the source message timestamp so cold-load and INFLIGHT can be compared on the same axis.
• P2: replaced grep-only structural tests with AST call-shape checks; added malformed-item / concurrent-emit / extra malformed-payload coverage; reduced brittleness in static-shape assertions; replaced flake-prone absolute timing with linear-scaling ratio; added the real-localStorage round-trip + reconnect test file.

[nesquena-hermes]

Triage: HOLD pending deep review — labels: hold, maintainer-review

This is a large addition: a realtime Todos panel + INFLIGHT recovery surface, +5,504 LOC across 17 files including new modules (api/todo_state.py), api/routes.py cold-load hooks, and api/streaming.py integration.

Two reasons to slow down:

1. New always-visible UI surface. A realtime Todos panel is a major UI addition. We want multi-viewport screenshots (390/1280/1440/1920), behavior with no todos, behavior mid-stream, behavior after reload, behavior with INFLIGHT recovery scenarios, and dark vs. light theme.

2. New architectural module + streaming.py integration. api/streaming.py is in our sensitive-paths list and api/todo_state.py adds a parallel store concept. We need our deep-streaming reviewer to walk this carefully. INFLIGHT recovery is also one of the sharpest paths in the codebase — the test coverage looks present but we want to spend some time on the recovery contract before this lands.

Action items on your side when you have time:

• Multi-viewport screenshots for the Todos panel (empty / streaming / hydrated / inflight-recovery)
• Short README in docs/ (or in the PR body) explaining the todo_state contract and how it interacts with the existing run-journal / partial-output recovery
• Confirm whether the agent side already has a todo store this could consume vs. being WebUI-only

Will follow up here after the deep review. Thanks for the work — it's substantial and we want to land it well rather than rush it.