Skip to content

Skip exhausted Claude accounts during failover #41

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
lawrencecchen merged 2 commits into from
Jul 3, 2026
Merged

Skip exhausted Claude accounts during failover #41

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
9e9539c
Add Claude exhausted retry regression
lawrencecchen Jul 3, 2026
dfc7e49
Skip exhausted Claude retry accounts
lawrencecchen Jul 3, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 41 additions & 0 deletions internal/proxy/claude_ratelimit_routing_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -510,6 +510,47 @@ func TestClaudeFailoverExhaustionIsLogged(t *testing.T) {
}
}

func TestClaudeFailoverSkipsMarkedExhaustedAlternates(t *testing.T) {
server, store := claudeFailoverServer(t)
if _, err := store.Put("claude", "session-exhausted-alt", "cooked@example.com", ""); err != nil {
t.Fatal(err)
}
server.SchedulerRef.MarkExhaustedUntil(accounts.ProviderClaude, "fresh@example.com", time.Now().Add(time.Hour))

var calls int
stub := &stubRoundTripper{responses: func(req *http.Request) *http.Response {
calls++
if strings.Contains(req.Header.Get("Authorization"), "tok-fresh") {
t.Fatal("fresh account is marked exhausted and must not be retried")
}
h := http.Header{}
h.Set("Anthropic-Ratelimit-Unified-Status", "rejected")
h.Set("Anthropic-Ratelimit-Unified-Reset", strconv.FormatInt(time.Now().Add(time.Hour).Unix(), 10))
return &http.Response{StatusCode: http.StatusTooManyRequests, Header: h, Body: io.NopCloser(strings.NewReader(`{"type":"error","error":{"type":"rate_limit_error","message":"Error"}}`))}
}}
transport := usageLimitRetryTransport{
base: stub, server: &server, provider: accounts.ProviderClaude,
agent: "claude", session: "session-exhausted-alt", account: "cooked@example.com",
method: http.MethodPost, path: "/v1/messages", maxAttempts: 6,
}
req, err := http.NewRequest(http.MethodPost, "https://api.anthropic.com/v1/messages", bytes.NewReader([]byte(`{}`)))
if err != nil {
t.Fatal(err)
}
req.Header.Set("Authorization", "Bearer tok-cooked")
response, err := transport.RoundTrip(req)
if err != nil {
t.Fatal(err)
}
defer response.Body.Close()
if response.StatusCode != http.StatusTooManyRequests {
t.Fatalf("status=%d, want original 429", response.StatusCode)
}
if calls != 1 {
t.Fatalf("upstream calls=%d, want 1 because marked-exhausted alternates are skipped", calls)
}
}

// TestClaudeRejectedHeaderOn200FailsOver is the Aziz regression: a depleted
// account answers 200 via overage but sets anthropic-ratelimit-unified-status:
// rejected, which Claude Code hard-blocks on. subrouter must treat that 200 as
Expand Down
13 changes: 8 additions & 5 deletions internal/proxy/proxy.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3163,22 +3163,25 @@ func (s Server) oauthRetryAccount(ctx context.Context, provider accounts.Provide
// though healthy accounts remained untried.
var lastErr error
for {
scheduler := s.scheduler()
if s.Sessions != nil {
scheduler = scheduler.WithSessionCounts(s.Sessions.CountByAccount())
}
candidates := make([]accounts.Account, 0, len(allCandidates))
for _, account := range allCandidates {
if _, ok := tried[account.ID]; ok {
continue
}
if scheduler.Exhausted(account.Provider, account.ID) {
continue
Comment on lines +3175 to +3176

@chatgpt-codex-connector chatgpt-codex-connector Bot Jul 3, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Do not drop stale-exhausted failover candidates

When an alternate account has a zero scheduler score from stale usage data but no active request-time exhaustion mark, this continue removes it from failover entirely. In that scenario a Claude request whose current account returns a rejected 429 will return the 429 to the client with no_alternate_account, even if the alternate token is valid and upstream would return 200; the code below still documents that these apparently exhausted accounts should be tried because scores can be stale. Please distinguish actively marked-exhausted accounts from stale zero scores, or keep the previous bounded probe behavior for score-only exhaustion.

Useful? React with 👍 / 👎.

}
candidates = append(candidates, account)
}
if len(candidates) == 0 {
if lastErr != nil {
return accounts.Account{}, lastErr
}
return accounts.Account{}, fmt.Errorf("no untried OAuth %s accounts available", provider)
}
scheduler := s.scheduler()
if s.Sessions != nil {
scheduler = scheduler.WithSessionCounts(s.Sessions.CountByAccount())
return accounts.Account{}, fmt.Errorf("no untried non-exhausted OAuth %s accounts available", provider)
}
account, err := scheduler.Pick(candidates)
if err != nil {
Expand Down