fix(models): update context limits and error types for paid plans - #233
fix(models): update context limits and error types for paid plans#233sendsaf wants to merge 14 commits into
Conversation
…tem prompt Claude Code 2.1.x prepends a per-request system text block of the form: x-anthropic-billing-header: cc_version=2.1.153.d02; cc_entrypoint=sdk-cli; cch=<5hex>; The cch segment is a fresh random hex token on every request. Because extract_system_prompt() concatenates all system text blocks into a single string before forwarding to Kiro, this random prefix poisons the prompt seen by the upstream and invalidates any prefix-keyed prompt cache (the attribution header is meant to be an HTTP header but ships in body here). Add _strip_billing_attribution() and call it from extract_system_prompt() for both string and list system formats. Blocks that are pure attribution are dropped; mixed blocks have the leading attribution line removed. The behavior is gated by STRIP_BILLING_HEADER (default true) so it can be disabled for A/B comparison.
When the Kiro API sends tool arguments split across multiple tool_input
events as dicts, the parser was serializing each dict independently and
concatenating the JSON strings — producing invalid JSON like
`{"file_path":"..."}{"content":"..."}`. The brace-counting diagnostic
then misidentified this as upstream truncation.
Fix: accumulate dict fragments via .update() and serialize once at
finalize time. String fragment path remains unchanged.
Fixes jwadow#153
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Bedrock rejects tool input_schema containing oneOf, allOf, or anyOf at the top level (TOOL_SCHEMA_INVALID). This resolves them before sending: - anyOf/oneOf: picks the first object-type variant - allOf: merges all variants into a single schema Nested composition keywords are preserved since Bedrock only rejects them at the root of input_schema.
…d plans - Add per-model tokenLimits to FALLBACK_MODELS matching Kiro's documented context windows: 1M for sonnet-4-6, opus-4-6/4-7/4-8 and auto; 200k for older models; 128k/256k for non-Claude models - Raise DEFAULT_MAX_INPUT_TOKENS from 200k to 1M to match the highest-tier models available on paid plans - Map CONTENT_LENGTH_EXCEEDS_THRESHOLD errors to the standard error types that Claude Code and compatible clients recognise (invalid_request_error for Anthropic format, context_length_exceeded for OpenAI format), enabling automatic context compaction instead of a hard failure
|
Thanks for the PR! 🎉 Before merge, we need a one-time CLA confirmation. Full CLA text: Please reply once with: You need to write once, all further messages from me can be ignored. |
|
I have read the CLA and I accept its terms |
|
Thanks for the PR! 🎉 Before merge, we need a one-time CLA confirmation. Full CLA text: Please reply once with: You need to write once, all further messages from me can be ignored. |
…handling AnthropicMessage.role is now str (not Literal) so unknown roles pass validation and are normalized downstream — test updated to expect 200. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
|
Thanks for the PR! 🎉 Before merge, we need a one-time CLA confirmation. Full CLA text: Please reply once with: You need to write once, all further messages from me can be ignored. |
|
I have read the CLA and I accept its terms |
5 participants
Problem
The gateway had two related issues that caused hard failures for users on paid Kiro plans using newer models:
1. Wrong context window limits in fallback model list
FALLBACK_MODELShad notokenLimitsper model, so every model fell back toDEFAULT_MAX_INPUT_TOKENS = 200000. This was appropriate for the free plan era (sonnet-4.5 etc.), but Kiro's current paid plans offer 1M context on sonnet-4.6, opus-4.6/4.7/4.8, and auto. Users would hit a 400 error well before reaching the actual model limit.2. Context overflow errors not recognised by clients
When Kiro returned
CONTENT_LENGTH_EXCEEDS_THRESHOLD, the gateway forwarded it as"type": "kiro_api_error"(OpenAI) or"type": "api_error"(Anthropic). Claude Code and claude-code-router do not recognise these types as context errors, so automatic compaction (/compact) failed — clients had no way to recover without starting a fresh session.Changes
kiro/config.pytokenLimitstoFALLBACK_MODELSbased on Kiro's documented context windows:auto,claude-sonnet-4.6,claude-opus-4.6,claude-opus-4.7,claude-opus-4.8claude-sonnet-4,claude-sonnet-4.5,claude-haiku-4.5,claude-opus-4.5deepseek-3.2qwen3-coder-nextglm-5,minimax-m2.1,minimax-m2.5DEFAULT_MAX_INPUT_TOKENSfrom 200k → 1M to match the highest-tier flagship modelskiro/routes_openai.pyCONTENT_LENGTH_EXCEEDS_THRESHOLD→"type": "context_length_exceeded"in both the legacy and account-system error paths (standard OpenAI error type that clients use for context recovery)kiro/routes_anthropic.pyCONTENT_LENGTH_EXCEEDS_THRESHOLD→"type": "invalid_request_error"in both error paths (standard Anthropic error type recognised by Claude Code for automatic compaction)Test plan
pytest tests/unit/ -q)/v1/modelsreturns models with correct context windowscontext_length_exceeded/invalid_request_errorinstead of generic API error/compactcompletes successfully after a context limit hit