Model-context rebuild materializes every visible image; compaction is token-denominated and cannot prevent image-byte OOM

[FredKSchott]

Problem

rebuildCanonicalContext (session.ts) resolves every attachment visible on the active path into memory — raw bytes loaded from the store, base64-encoded (~2.4× stored size resident, transiently more), injected into agentLoop.state.messages for the life of the submission. Rebuild runs at session open, at the prompt/steer/join seams, throughout turn recovery, and after compaction. There is no bound of any kind on the materialized image bytes.

The only eviction mechanism, compaction, cannot help, for three independent reasons:

1. Its trigger is token-denominated, and estimateTokens counts user-message image blocks at zero (only text is counted; tool-result images count ~1200) — the cut-point math is blind to exactly the content that causes the OOM.
2. Images are token-cheap and byte-huge (~1.5k tokens vs ~10 MB each): a 200k-token window "fits" an order of magnitude more image bytes than a 128 MB isolate holds. Memory dies long before any token trigger fires.
3. runCompaction itself first materializes all attachments before summarizing — the overflow-recovery mechanism needs the memory that overflowed.

This is also the second leg of the wake crash loop (#530): inspecting or terminalizing an interrupted submission opens a session, and opening a session materializes the full visible image context — so even the give-up path OOMs on an image-heavy instance.

Proposed direction

A byte budget on the materialized model context, decided in the projection layer from AttachmentRef.size before loading any bytes: walk visible entries newest-to-oldest summing ref sizes; refs within the budget (and a count cap) are inline, older ones are evicted. The store is never asked for evicted bytes, so peak memory is bounded at every call site that shares the resolve helper — context rebuild, join steer, and compaction's pre-summarization pass.

• An evicted attachment projects to the model as a text placeholder naming the attachment id. The model already receives an <attachments> manifest and can re-reference ids (e.g. the task tool's attachments: [{id}]), so evicted is degraded, not destroyed. History/UI projections are untouched (they never carried bytes).
• The inline/evicted partition is a pure function of the active path's refs and constants — live rebuilds and crash-resumed rebuilds stay byte-identical; no new record types, no reducer state.
• Compaction keeps owning tokens (with the user-image estimator fix from Message admission has no aggregate attachment budget; tool-result images have no size check at all #529); eviction owns bytes; they compose.

Open design point: whether the inline set is frozen per response (friendlier to the model and provider prompt caches) or re-partitioned between turns.

Related: #529 (admission budgets), #530 (submission-payload byte residency / crash loop).