fix(proxy/ccr): self-heal dangling headroom_retrieve reference after model switch

[nangsontay]

Description

Through the Headroom proxy, switching models mid-session (/model in Claude Code) — or restarting the proxy — 400'd every subsequent turn with Tool reference 'headroom_retrieve' not found in available tools.

Root cause: the sticky CCR retrieve-tool injection is keyed by a model-scoped, in-memory session id (compute_session_id hashes model + leading system text). A /model switch or restart rotates/loses that key, so apply_session_sticky_ccr_tool stops injecting headroom_retrieve — while the client transcript still carries a tool_reference block naming it (Anthropic validates every tool_reference against the request's tools array).

Fix: make the invariant self-heal from the transcript instead of from tracker state. Key-tweaking alone can't fix it (the system-prompt half of the hash rotates on model switch, and restarts lose the tracker regardless).

Plan: plans/260723-1004-ccr-dangling-tool-reference-reinjection/.

Closes #

Type of Change

• Bug fix (non-breaking change that fixes an issue)
• New feature (non-breaking change that adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)
• Documentation update
• Performance improvement
• Code refactoring (no functional changes)

Changes Made

• headroom/proxy/ccr_marker_policy.py — new transcript_references_ccr_tool() that scans about-to-forward messages for a bare headroom_retrieve in anthropic tool_reference/tool_use blocks (one level of nesting into tool_result/tool_search_tool_result) or an openai-chat assistant.tool_calls entry; extended should_inject_ccr_tool() with a transcript_requires_tool override.
• headroom/proxy/tool_injection_logging.py — added inject_transcript_recovery to the closed ToolInjectionDecision literal.
• headroom/proxy/helpers.py — re-export the policy fn; thread transcript_requires_tool through the should_inject_ccr_tool wrapper and apply_session_sticky_ccr_tool (both the no-session-id and fresh-session paths now inject on the transcript signal, log inject_transcript_recovery, and record sticky state so subsequent turns resume normal sticky replay).
• headroom/proxy/handlers/anthropic.py — compute the flag over optimized_messages, pass it through, add a distinct recovery log branch.
• headroom/proxy/handlers/openai.py — same wiring on the chat path.
• tests/test_ccr_tool_always_on.py — 15 new tests (policy wire-shapes, mcp-prefixed exclusion for both providers, override semantics, and end-to-end recovery through the sticky helper).
• docs/content/docs/troubleshooting.mdx — new troubleshooting entry (symptom, cause, fix, and workarounds for older versions).

Exact bare-name match only, so a client-owned mcp__headroom__headroom_retrieve (MCP, client-owned lifecycle) never triggers proxy injection. The openai /v1/responses path never proxy-injects the CCR tool, so recovery is N/A there (plan decision D1).

Testing

• Unit tests pass (pytest) — scoped to the CCR/tool-injection blast radius (16 files, 256 tests); the full repo suite has a known noisy baseline unrelated to this change.
• Linting passes (ruff check) — on the touched files.
• Type checking passes (mypy headroom) — not run this pass.
• New tests added for new functionality
• Manual testing performed — live proxy /model switch not exercised; reproduced via unit tests (see Real Behavior Proof).

Test Output

$ uv run pytest tests/test_ccr_marker_policy.py tests/test_ccr_tool_injection.py \
    tests/test_tool_injection_logging.py tests/test_tool_injection_policy.py \
    tests/test_ccr_tool_always_on.py tests/test_ccr_tool_calls.py \
    tests/test_anthropic_beta_session_sticky.py tests/test_openai_beta_session_sticky.py \
    tests/test_proxy_ccr.py tests/test_ccr_session_tracker.py tests/test_ccr_golden_policy.py \
    tests/test_ccr_response_handler.py tests/test_ccr_response_handler_openai_responses.py \
    tests/test_memory_tool_session_sticky.py tests/test_tool_injection_tracker.py \
    tests/test_tool_injection_config.py -q
======================= 256 passed, 3 warnings in 6.14s ========================
# (the 3 warnings are pre-existing deprecations, not from this change)

$ uv run ruff check headroom/proxy/ccr_marker_policy.py headroom/proxy/tool_injection_logging.py \
    headroom/proxy/helpers.py headroom/proxy/handlers/anthropic.py \
    headroom/proxy/handlers/openai.py tests/test_ccr_tool_always_on.py
All checks passed!

Real Behavior Proof

• Environment: local, macOS (darwin), Python 3.13.14, uv run pytest (pytest 9.0.3).
• Exact command / steps: the pytest/ruff commands above. The /model-switch failure is reproduced as a unit test — test_transcript_recovery_injects_on_fresh_tracker constructs the exact failing shape (dangling tool_reference, freshly-rotated/empty tracker, no fresh CCR markers) and asserts the tool is re-injected with decision=inject_transcript_recovery; test_transcript_recovery_records_sticky_state asserts the following turn resumes sticky replay.
• Observed result: 256/256 pass; recovery injects and re-records sticky state; reference-free requests and mcp-prefixed names keep prior behavior.

Review Readiness

• I have performed a self-review
• This PR is ready for human review

Checklist

• My code follows the project's style guidelines
• I have performed a self-review of my code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes
• I did not edit CHANGELOG.md — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this)

Screenshots (if applicable)

N/A — proxy/backend change, no UI surface.

Additional Notes

• Cache impact: the recovery injection changes the tools prefix once on the first post-switch turn (same accepted cost as the existing [BUG] Proxy compresses content and emits retrieval markers when headroom_retrieve tool injection is deferred, causing irrecoverable data loss #1006 marker-override — one cache miss beats a hard 400 loop). It does not re-inject per turn once sticky state is re-recorded.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[github-actions]

PR governance

This PR does not yet satisfy the required template fields:

• Fill in Real Behavior Proof → Not tested.

Please update the PR body, or move the PR back to draft while it is still in progress.

[codecov-commenter]

⚠️ Please install the to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

❌ Patch coverage is 90.38462% with 5 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
headroom/proxy/ccr_marker_policy.py	93.02%	1 Missing and 2 partials ⚠️
headroom/proxy/handlers/anthropic.py	50.00%	1 Missing and 1 partial ⚠️

📢 Thoughts on this report? Let us know!