Compaction aims deep: trigger, target, and honest numbers (#53, part 1)

[webmatze]

Part 1 of #53. Makes the numbers compaction acts on describe the actual request, and splits the one threshold that caused the reported symptom into a trigger and a target.

The symptom

Context compacted (truncated): ~122887 → ~107023 tokens

Context.compact used a single max_tokens as both the "should I act?" test and every stage's stop condition, so it stopped the instant the history dipped under the ceiling. The next assistant turn put it straight back over. It ran every turn, reclaimed almost nothing, and paid a full prompt-cache creation charge each time.

What changed

Three numbers, not one. A budget (context.max_tokens), a trigger at 80% of it (compact_at), and a target at 50% (compact_to) that every stage compacts down to. Both are fractions, so raising the budget for a wider-window model scales all three. Below the trigger the message array is returned untouched — the same object, not an equal copy — so a non-compacting turn keeps the prompt cache. Because the target now sits below what truncation alone can usually reach, the summarizer stage starts running for the first time.

The whole request counts. System prompt, project context, skills and tool definitions are measured through the same Breakdown that smith context renders, so the breakdown and the compaction decision cannot disagree. It is recomputed every turn rather than cached: @system_prompt is rewritten by the mode switch and the registry grows when an MCP server connects.

The estimate gets corrected. Thinking signatures are counted at all, which they were not, and the byte heuristic is scaled by how far it has been off from the prompt-token counts the provider reports back — smoothed, clamped to 1.0..2.0, and persisted with the session so the first turn after a resume is not blind.

Falling short is said out loud. Missing the target warns; being unable to reach the ceiling ends the run with a new ContextExhausted event rather than paying the provider to reject the request. Its own event, not budget_exceeded, so a script can tell "start a fresh session" apart from "you spent your money".

A latent bug this activates

Checkpoints stores absolute message indices (checkpoints.cr:122) and a rewind truncates the transcript at one (cli.cr:1086). Replacing an N-message prefix with a single summary moves every stored index. The bug has been there as long as the summarizer has — it was simply unreachable, and this PR's whole point is to make the summarizer run.

Indices are now shifted by what compaction removed. A checkpoint whose messages were replaced outright is marked rather than silently clamped onto index 1: clamping several onto the same position would make rewind --to 0012 cut the whole conversation back to the summary and report that as the point picked. Marked ones restore their files and leave the transcript alone, and the CLI says why. Replacing indices with stable message ids is the correct fix and is follow-up work.

Review fixes

A high-effort review found ten issues; nine are fixed in this branch. Two were serious:

• The calibration was inert. usage.prompt_tokens on Anthropic is only the uncached remainder — a few hundred tokens for a 100k history on a cache hit. The observed ratio was therefore always below 1.0 and clamped back to the floor every turn, so the feature meant to make the estimate honest would never have worked on the default provider. Now driven by LLM::Usage#billed_prompt_tokens.
• allowance could floor at 0 when a fat tool set filled the target, truncating every result and cutting back to the newest turn while half the window sat unused. It now aims at the ceiling when the target is unreachable, and leaves the history alone when even the ceiling is.

Also fixed: an exhausted subagent reported to the parent as completed; max_tokens = 0 aborting every run; Ctrl+C and the TUI abort dropping the learned ratio (three hand-copied save blocks, now all through persist); fork not copying it; smith context rows no longer summing to their total; and already-truncated results being re-cut for zero reclaim, a nonsensical "0 KiB truncated" note and a busted cache entry.

One finding was kept deliberately: return if compact_history ends the run on an estimate where the old code would have sent the request. Aborting on a breached budget is the agreed design, and the realistic accidental path into it — max_tokens = 0 — is now closed.

Testing

828 examples, all green. Every compaction path still asserts assert_tool_pairing. The two tests worth naming:

• Idempotence — compacting an already-compacted history returns None and a byte-identical array.
• Convergence — a simulated 30-turn session compacts fewer than five times in total. The only test that exercises the reported symptom as a symptom; the complaint was never about one call, it was about the cadence.

Not in this PR

Stages 0–2 (drop stale thinking, supersede duplicate reads, staged truncation caps), the synthetic flag for agent-injected turn boundaries, incremental token accounting, and the raw transcript log. That is part 2.

🤖 Generated with Claude Code