feat(daemon): attribute stable-prefix drift to the input that caused it

[lefarcen]

Why

My use case. I was following up on the 0.14.1 prompt-cache drift investigation (a colleague's write-up + the drift dashboard I built for #622). Both stall at the same wall: stable_prompt_cache_miss_reason='stable-prompt-changed' tells us that the cached prefix moved, but never which input moved. Every diagnosis so far has been a human pairing Langfuse traces and eyeballing two multi-KB prompts for a diff. That does not scale, and it is the concrete blocker on driving drift down.

The pain. Drift is real, large, and worsening every release. AMR, from PostHog:

	07-09	07-16
drift events	77	1,032
eligible resume turns	506	2,583
drift rate	15.2%	40.0%

Absolute drift grew 13.4×: traffic 5.1× × rate 2.6×. Both are real — the rate alone went 15.2% → 40.0%. It worsens monotonically per release: 0.13.0 20.2% → 0.14.0 26.0% → 0.14.1 37.7% → 0.15.0 50.5%.

And it is not an AMR story. Over 30 days, by drift rate on resume turns: codex_cli 52.2%, opencode 40.3%, claude_code 36.6%, amr 34.0% — AMR is the best of the four and only ~5.6% of the ~132k total drift events. (The other 15 runtimes are structurally 0: only native-resume runtimes can report drift at all.)

The part that decides scope: over 14 days on AMR, later-resume turns (turn ≥2) carry 4,812 of 6,828 drift events — 71% of the total, despite a much lower rate (27.0% vs 67.2%) than first-resume's smaller base. First-resume drift has a known story (memory extraction sediments new facts mid-session). Nothing in today's telemetry can say what drives the other 4,812. This PR is what makes that question answerable.

This is instrumentation, deliberately not a fix. It ships no behavior change so it can land fast and start collecting; the actual remediations (freezing the memory snapshot at session start being the big one) need that attribution to be targeted and to be verifiable afterwards.

What users will see

Nothing. No UI, no CLI, no prompt bytes, no model-visible change. This adds one diagnostic property to an existing analytics event and one nullable SQLite column.

Two notes for reviewers, since both are questions this PR invites:

1. The prompt is byte-identical. The only prompt-adjacent change is hoisting the existing composeSystemPrompt({...}) argument literal into a const and passing it — so one object both composes the prompt and feeds attribution, instead of a second hand-maintained copy that would drift from the real inputs and mislabel the very telemetry it exists to explain. The literal is unchanged. See Validation for how this is proven rather than asserted (the obvious guard — the system-prompt snapshot suite — is already red on main, so it could not carry the claim on its own).
2. No CLI surface. AGENTS.md's UI/CLI dual-track rule covers user-facing capabilities. This is internal instrumentation on an existing event — there is no capability for od to expose.

Surface area

• Default behavior change — additive nullable SQLite column agent_sessions.stable_prompt_sections, guarded by the existing PRAGMA table_info migration pattern. Old rows read back null and report unattributed for exactly one turn, then self-heal on the next write. No backfill, no downgrade hazard (an older daemon ignores the column).
• UI / Keyboard shortcut / CLI / env var / API / contract / Extension point / i18n / New top-level dependency

Design

Attribution is computed over the composer's inputs, not by slicing the composed prompt on its --- separators. A drift event has to answer "what caused the prefix to move", and the cause is always an input — splitting the output would report which bytes differ but still leave the mapping back to a cause to a human reading Langfuse, which is the manual step this exists to remove. It also means zero risk of touching prompt assembly.

prompts/stable-sections.ts holds one table mapping each composer argument to its section (memory, intent, mode, design-system, skill, craft, plugin, instructions, locale, media, critique, mcp, runtime, client-system) — covering the memory / intent / mode / design-system / skill / runtime split the investigation asked for.

Two properties worth calling out:

• stablePromptHash stays the only source of truth for hit/miss. Section hashes never gate a re-send. A bug in the section table can mislabel a drift event but can never suppress or fabricate one.
• unattributed is a designed outcome, not a failure mode. If the prefix moves and no tracked section did, the input that moved is missing from the table. That is a coverage-gap alarm rather than a silently clean turn — and its rate tells us how good our coverage is. This is what makes the design self-validating: adding a future input to the stable prefix without registering it here shows up in telemetry instead of hiding.

The mcp section is deliberately dormant: #5336 moved connectedExternalMcp out of the cached prefix (it reflects live OAuth token validity and flipped mid-conversation). Nothing passes that key today, so the section never reports — it exists so re-admitting a live-token input is attributed on sight rather than rediscovered by hand a third time. memory, intent, and MCP tokens are three instances of one disease: live per-turn state ending up in a prefix that is supposed to be stable.

Bug fix verification

Not a bug fix (instrumentation), but the e2e assertions are falsifiable and were verified as load-bearing rather than assumed:

• apps/daemon/tests/intent-signal-stable-prompt-cache.test.ts — real daemon, real session resume, real drift:
  • a genuine deck-intent flip → changedSections: ['intent']
  • a memory entry landing mid-conversation → changedSections: ['memory'] and nothing else (the follow-up carries no deck/media/platform vocabulary, so attribution must isolate one cause)
  • a cache hit → changedSections: null
• Mutation-tested. Moving memoryBody out of the memory section turns the memory test red (changedSections stops being ['memory']). The assertions read real production-path data — they are not vacuously green.

Adjacent issues

Found while working here, deliberately not fixed in this PR to keep the diff's answer to "did you change the prompt?" an unambiguous no:

• apps/daemon/src/server.ts carries a stale comment above promptCoreVariant: "VALIDATION DEFAULT — feat/system-prompt integration branch only … main keeps classic as the default — do NOT carry this flip into a PR against main." feat(prompts): land the slim system-prompt line as the default charter #5603 intentionally landed slim as the default charter on main, so the code is right and the comment now tells the next reader the opposite of the truth. Comment-only fix, worth its own PR.
• The drift dashboard's alert (mine) fires on absolute daily count >150, which the investigation correctly flags as traffic-sensitive. Fixed on the dashboard side, not here: ratio as the regression signal, absolute retained as the scoreboard (the goal is 0, and 0 does not scale with traffic).
• The dashboard's eligible denominator was wrong, and I fixed it while writing this up. stable_prompt_cache_miss_reason != 'new-session' also matches turns from builds predating this telemetry (≤0.11.0 — 35k AMR turns per 30d), where the property is simply absent. Counting those as eligible inflated every denominator ~2.5× and understated drift accordingly (AMR read 12.7% when it is 34.0%). The honest marker is stable_prompt_cache_hit is set, which is present iff the daemon computed a cache decision. The figures in this PR are the corrected ones. This is a query-side fix only — no code change — but it is worth knowing that published drift rates before today were low by ~2.5×.

Validation

• pnpm guard — 86/86 pass
• pnpm --filter @open-design/daemon typecheck — clean
• pnpm --filter @open-design/daemon test — full daemon suite: 7 failed | 5846 passed | 6 skipped
• apps/daemon/tests/prompts/stable-sections.test.ts — 18 new unit tests (determinism, key-order independence, section isolation, appearing/disappearing sections, unattributed, malformed-row degradation)
• apps/daemon/tests/chat-route.test.ts — describeStablePromptCache's existing case updated for the new field, plus 3 new cases (named sections, unattributed, no attribution without a baseline)

Byte-identity of the prompt, proven not asserted. The natural guard, tests/prompts/system-prompt-matrix.test.ts → "keeps the section gating matrix stable across scenarios", is already failing on main (stale snapshot, pre-dates this branch), so a green run could not have carried the claim. Instead I compared the snapshot mismatch output itself between baseline and this branch: 271 lines, byte-for-byte identical. The hoist changes no prompt bytes. (Refreshing that stale snapshot is main's problem, not this PR's — touching it here would be exactly the prompt change this PR must be able to deny.)

All 7 failures diffed against baseline (origin/main @ 1dd813f00) rather than assumed:

	Test Files	Tests
baseline (chat-route + connection-test)	2 failed	3 failed | 220 passed
this branch (same two files)	2 failed	3 failed | 223 passed

• Six are reproduced identically on baseline: launches Kimi connection tests without the legacy acp positional arg, passes keyless BYOK provider config without auth fields to OpenCode, does not leave a pinned assistant message queued when legacy chat fails before spawning, returns signed_out without reading an upstream wallet API, drives representative failed runs through analytics and Langfuse diagnostics, keeps the section gating matrix stable across scenarios.
• The seventh, discovers the default tools-dev daemon URL when no sidecar IPC path is available, is a load-dependent flake: tests/daemon-url.test.ts is 5/5 green in isolation on both baseline and this branch.

This branch adds 3 passing tests to the two files that already had failures, and nothing new to the failure set.

[PerishCode]

@lefarcen This is a clean, well-scoped instrumentation PR — I verified the correctness-sensitive parts end to end and everything holds together.

What I checked:

• Migration safety. The stable_prompt_sections column follows the existing PRAGMA table_info additive-guard pattern; new DBs get it via CREATE TABLE, old DBs via ALTER, and the .some(...) guard prevents a double-add. Old rows read back null and degrade to unattributed for one turn as documented.
• SQL binding integrity. upsertAgentSession now has 9 columns / 9 placeholders / 9 bindings in matching order, and every writer (all upsertAgentSession and persistCapturedAgentSession call sites in server.ts) passes stablePromptSections, so a stored hash never lands without its section map.
• Attribution coherence. The section table's input keys line up with the hoisted systemPromptInputs literal plus the two caller-merged inputs (runtimeToolPrompt, clientSystemPrompt), which are exactly the three parts of stableInstructionFingerprint. stablePromptHash stays the sole re-send decider — section hashes only label a decision already made, so a table bug can mislabel but never suppress or fabricate a drift. The hoist is byte-identical to the prior call argument.
• End-to-end plumbing. changedSections flows from describeStablePromptCache → run.promptCache → the /api/runs serializer (runtimes/runs.ts spread), the run_finished analytics event, and the Langfuse turn payload, so the e2e assertions read real production-path data.

The unit + e2e coverage is genuinely load-bearing (determinism, key-order independence, section isolation, appear/disappear, unattributed, malformed-row degradation, and the mutation-tested memory-isolation case). No behavior change, no prompt bytes moved. Nice, careful work — the unattributed-as-coverage-alarm design is a good touch that keeps the table honest as new inputs enter the prefix.

_{🔁 Powered by Looper · runner=reviewer · agent=claude-code · An autonomous AI dev team for your GitHub repos.}