Skip to content

fix(models): count non-200 responses against OpenAI retry limit - #2752

Open
pcbeingused333 wants to merge 1 commit into
open-compass:mainfrom
pcbeingused333:fix/openai-non200-retry-limit
Open

pcbeingused333 wants to merge 1 commit into
open-compass:mainfrom
pcbeingused333:fix/openai-non200-retry-limit

Conversation

@pcbeingused333

Copy link
Copy Markdown
Contributor

Symptom. An evaluation using OpenAI with an invalid key (HTTP 401) hangs forever instead of failing, since the non-200 branch never consumes a retry.

Root cause. In OpenAI._generate (opencompass/models/openai_api.py), a non-200 HTTP response logs the error and does continue. That skips max_num_retries += 1 (which sits after the finally), so a persistent 401/400/500 never reaches the RuntimeError after self.retry attempts and the evaluation loops forever.

Fix. Increment max_num_retries before the continue on the non-200 branch, so it consumes one retry like the connection-error path does. HTTP 429 is the exception: it is transient, so it sleeps 10 s and retries without consuming one (same as the existing rate_limit_exceeded handling), so rate limiting does not turn into a failure.

Behaviour change. A persistent non-429 error (401/400/5xx) now raises RuntimeError after retry attempts instead of retrying forever, the same as the connection-error path; raise retry for flaky backends.

Test. test_generate_non_200_respects_retry_limit mocks a permanent 401 with retry=2 and asserts RuntimeError after exactly 2 requests (before the fix the loop never terminates; the mock aborts after 10 calls).
test_generate_429_then_success covers 429, 429, 200 with retry=2 succeeding.


Disclosure: this change was prepared with the assistance of an AI coding tool (Claude Code) and submitted for review by the contributor.

OpenAI._generate hit 'continue' on any non-200 response before reaching
max_num_retries += 1, so a persistent 401/400/500 looped forever instead
of raising after self.retry attempts.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant