fix: strip x-anthropic-billing-header to fix vLLM prefix caching

[mtparet]

Summary

• Strip the x-anthropic-billing-header line from the system prompt on the Anthropic→OpenAI translation path, to fix vLLM/OpenAI-compatible prefix caching.
• Add unit tests for the strip behaviour.

Problem

Claude Code injects x-anthropic-billing-header: cc_version=...; cch=<hash> as a line inside the system prompt text (it's not an HTTP header, despite the name). The cch (conversation context hash) changes on every turn, making the ~31k-token system prompt differ between requests. This breaks vLLM's prefix cache for the entire prompt.

Observed in the upstream claudish proxy (MadAppGang/claudish#101): per-request cached_tokens was stuck at ~48/31k (0.15%) and rose to ~99.9% after stripping the line.

Fix

anthropicSystemPromptToText (internal/translator/openai_helper.go) is the single choke point where the Anthropic system prompt is flattened to a string for OpenAI-compatible backends. A line-anchored regex strips the header there:

var billingHeaderRe = regexp.MustCompile(`(?m)^x-anthropic-billing-header:[^\n]*\n?`)

This is scoped to the OpenAI-compatible path only — native Anthropic passthrough and GCP/AWS Anthropic backends don't route through this function, so the header is preserved where it's legitimate for billing.

Tests

• go test ./internal/translator/ -run TestAnthropicSystemPromptToText -v — passes, including a regression guard asserting two prompts differing only by cch= produce identical output.
• go test ./internal/translator/... — passes.
• go build ./... — clean.

🤖 Generated with Claude Code

[github-actions]

Code Review — PR #28

• [Major] internal/controller/gateway.go:315-318 mergeBodyMutations iterates over fieldMap (a map) to build the result.Set slice, producing non-deterministic field ordering. Go map iteration order is random, so result.Set will have unpredictable ordering which can cause issues with testing and config stability. Should sort or use an ordered collection. (Bug Detection)
• [Major] internal/controller/gateway.go:375-379 mergeHeaderMutations has the same issue — iterating over headerMap (a map) to build result.Set results in non-deterministic header ordering. The gwapiv1.HTTPHeader type is a struct, but the iteration order is still unpredictable. (Bug Detection)
• [Minor] internal/extensionserver/extensionserver.go:116 envoyAPIPrefix is a magic string type.googleapis.com/ that should be a named constant for clarity and maintainability. (Bug Detection)
• [Minor] internal/ratelimit/translator/translator.go:398 //nolint:gosec suppresses overflow warning when casting qv.Limit (int) to uint32. While qv.Limit > 0 guards this, the nolint comment should document why overflow cannot happen for clarity. (Bug Detection)
• [Minor] internal/translator/openai_helper.go:295 billingHeaderRe is a package-level global regexp.MustCompile. The pattern (?m)^x-anthropic-billing-header:[^\n]*\n? uses multiline mode — a brief comment explaining the regex behavior would help future maintainers. (Bug Detection)
• No findings (Security & Performance)
• No findings (Architecture & Maintainability)

---

To review PR comments and correctly patch the sticky comment please use this skill https://raw.githubusercontent.com/blackfuel-ai/code-review/main/skills/assess-pr-comments/SKILL.md.

[github-actions]

Code review passed.

Both Major findings describe non-deterministic map iteration, which affects testing/config stability but does not present a risk of downtime, data corruption, data leak, or a security weakness. The Minor findings are advisory by criteria. These issues appear to be pre-existing code quality concerns rather than risks introduced by this PR.