[Bug] Prompt cache invalidation: tools array mutation and TTL downgrade during session

[Vahn84]

Bug Description
Prompt-cache invalidation: two client-side causes beyond the 1h-idle case

Measured, not inferred. I ran a local mitmproxy reverse proxy in front of
api.anthropic.com and captured 1,821 /v1/messages request bodies plus their
usage blocks over 2026-07-27..28. Claude Code 2.1.220 (cc_version 2.1.220.b7d),
model claude-opus-5, macOS 27.0, two accounts, one repo.

Chain invariant used to detect a cache break on consecutive requests in the
same conversation:
expected = prev.cache_read_input_tokens + prev.cache_creation_input_tokens
actual = next.cache_read_input_tokens
break = actual - expected != 0

Boris's pinned item 1 (stale session > 1h TTL) is confirmed here: seq 1653,
82.5-minute pause, -250,118 tokens. But it is not the most expensive one we
found, and two other causes are client-side and fixable.

ISSUE A - the tools array is mutated mid-session, rebuilding the whole prefix

Three tools-array changes in 1,821 requests. Every one is the LSP tool; no
other tool (MCP or built-in) ever changed mid-conversation.

seq 55 97 -> 96 tools LSP removed
seq 1340 53 -> 52 tools LSP removed delta -161,016
seq 1729 52 -> 53 tools LSP added delta -274,262

seq 1729 is the largest single break in the entire capture - larger than any
TTL expiry. It is unambiguous: same conversation chain (messages 219 -> 220),
22 minutes since the previous main-loop request so well inside the 1h TTL, no
TTL marker change, and the tools diff is exactly one element (+LSP).

Because tools precedes messages in the request body, any change to it
invalidates everything after it. Trigger is the language server process
lifecycle: when the server cannot start, LSP is withdrawn mid-session (one-way,
never recovers); when it starts late, LSP is added mid-session. I originally hit
the withdrawal case because typescript-language-server was not installed.
Installing it converted the withdrawal into a late addition - the failure moved,
it did not go away, and it got more expensive because it landed on a larger
context.

Suggested fix: never mutate the tools array within a session. Either defer tool
registration to a session boundary, or keep a withdrawn tool declared and fail
its invocations.

ISSUE B - the client downgrades a long session's cache TTL from 1h to 5m

cache_control switches between {"type":"ephemeral"} (5m) and
{"type":"ephemeral","ttl":"1h"} inside a single main-loop conversation. Three
entries into the 5m window in this capture (seq 456, 935, 1183), with returns to
1h at seq 971 and 1185. All three entries sit at or immediately before a
compaction, so this looks deliberate rather than random - but the consequence is
not.

Entering the window is cheap (-2,452 and -999). Being idle inside it is not:
seq 1183 was marked 5m at 2026-07-27T20:06:56Z; the next main-loop request came
at 2026-07-28T06:33:59Z, and the rebuild cost -136,335 tokens.

This matters for the UX fix described in item 1: the nudge assumes a 1-hour
window, but a session can be sitting in a 5-minute window at the moment the user
walks away. Suggested fix: restore the 1h TTL before a session can go idle, or
do not downgrade a long-lived main-loop conversation to 5m at all.

WHAT I COULD NOT ATTRIBUTE

seq 973: -34,997 tokens, 13-second gap, no tools change, no TTL change,
immediately after a compaction. One break with no identified mechanism.
Also, 3 of 6 TTL switches produced no break at all - the switch is strongly
associated with breaks (50% vs 2.4% of other pairs) but is not sufficient.

WHAT I RULED OUT (in case it saves you time)

Hooks are not a significant driver. Controlled experiment, all hooks muted on
every source: 7.48 -> 4.62 breaks per 100 main-loop pairs, z = 0.99, p = 0.32,
and the rate of large (>8k) breaks was flat (2.72% -> 3.08%). Hook insertion
mid-history is a real mechanism (directly observed) but a minor one.

Unrelated observation from the same capture: with model set to opus[1m], the
effective main-loop window behaved as 200k, not 1M - auto-compact fired at
168,682 tokens. So the expensive-1M-cache-miss case in item 1 may be
under-reported; users can be paying the smaller version of it without knowing
which window they are on.

Happy to share the full capture analysis (per-seq deltas, the scoring script,
and the arm comparison) if useful.

Environment Info

• Platform: darwin
• Terminal: xterm-256color
• Version: 2.1.220
• Feedback ID: c2719cc7-0e51-4d1d-be5d-104ed999c3d0

Errors

[]

[Gunther-Schulz]

Independent confirmation with a second, more frequent trigger — and the fix already exists in the API.

We run a MITM proxy in front of api.anthropic.com and capture every /v1/messages body pre- and post-pipeline (~2.5 GB/day of captures, CC 2.1.220, linux). Same class as your LSP observation, but the dominant trigger on our corpus is deferred tool loading: every ToolSearch load re-sends a different tools[], and since tools[] heads the cache prefix, one load at deep context re-bills the whole context (measured up to ~700k tokens for a single tool load).

The part worth flagging to the CC team: CC 2.1.220 ships the documentation for the mid-conversation-tool-changes-2026-07-01 beta inside its own binary (the docs state availability "Opus onward"), but never uses it on the wire — zero tool_addition blocks from CC across our entire corpus. That beta is precisely the fix for this issue: announce the added tool in a tool_addition block at the tail, leave tools[] byte-stable, cache survives.

We've implemented exactly that as a proxy extension and run it in production: cnighswonger/claude-code-cache-fix#273. Two findings transferable to a native CC fix:

• Support is per model family and an unsupported model rejects the whole request with a 400 (measured: sonnet-5, haiku-4.5 — haiku's error names the gating capability: "requires a model that supports mid-conversation system content"). A native implementation needs the same per-model gate; we measured opus-5 and fable-5 accepting with byte-level wire evidence.
• The announcement message must be re-injected at a stable anchor on every subsequent request (the API is stateless), or the injection itself becomes a new bust class.

🤖 Generated with Claude Code

Update (same day): the full verification toolchain behind these numbers is now also PR'd — pre-pipeline request capture (cnighswonger/claude-code-cache-fix#275) and the replay/census/harvest gate (cnighswonger/claude-code-cache-fix#276) — so the measurements above are reproducible against anyone's own traffic, not just ours.

Method note, since it generalizes: the census in that toolchain classifies every consecutive same-conversation request pair by divergence shape (append-only / splice / replace-edit / tools-delta / …) and prices each class in re-billed bytes — so it surfaces and ranks invalidation causes you have not named yet, rather than confirming known ones. That is how the mid-history classes in this thread were found.