Skip to content

fix(claude): handle max-token Responses truncation - #5438

Open
Reekin wants to merge 1 commit into
router-for-me:devfrom
Reekin:fix/claude-max-token-terminal-pr
Open

fix(claude): handle max-token Responses truncation#5438
Reekin wants to merge 1 commit into
router-for-me:devfrom
Reekin:fix/claude-max-token-terminal-pr

Conversation

@Reekin

@Reekin Reekin commented Sep 3, 2026

Copy link
Copy Markdown

Summary

  • default Fable Responses requests to 64K output tokens while respecting lower registered model limits
  • preserve explicit max_output_tokens and document model-scoped payload.override configuration for budgets such as 128K
  • map Claude stop_reason: max_tokens to response.incomplete with incomplete_details.reason: max_output_tokens
  • preserve partial reasoning/text and keep already completed message item statuses stable
  • apply the terminal mapping to both streaming and non-streaming Responses paths

Problem

Claude can consume the full output budget in reasoning without producing a visible assistant message. The Claude-to-Responses translator currently ignores message_delta.delta.stop_reason and always emits response.completed, so Codex can silently finalize the turn with no final answer.

Configuration

payload:
  override:
    - models:
        - name: "claude-fable-5-1"
          protocol: "claude"
      params:
        max_tokens: 128000

The override is applied after protocol translation and works for both OAuth and API-key Claude execution.

Verification

  • go test ./internal/translator/claude/openai/responses -count=1
  • go test ./internal/runtime/executor -run TestClaudeExecutorPayloadOverrideMaxTokens -count=1
  • go build -o <temp>/cli-proxy-api ./cmd/server
  • real HTTP smoke: /v1/responses -> configured Claude upstream received max_tokens: 128000; downstream ended with response.incomplete and preserved reasoning output

go test ./... -count=1 was also exercised. Changed packages pass; the existing XAI TTFT timing assertion can intermittently report ttft = 0s on Windows and passes independently.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant