Skip to content

Commit df66e38

Browse files
committed
Switch judge evaluator from GPT-4o to Claude to avoid OpenAI quota limits
The judge evaluator was using GPT-4o (OpenAI) which hit quota limits during testing. Switching to Claude 3.5 Sonnet (Anthropic) to continue running CI tests without interruption. Changes: - Added --judge-model claude-3-5-sonnet-20241022 flag - Added --judge-provider anthropic flag - Judge evaluations will now use Claude instead of GPT-4o Benefits: - Avoids OpenAI API quota limits - Uses available Anthropic credits - Same evaluation quality (both are frontier models)
1 parent 3e02e31 commit df66e38

File tree

1 file changed

+2
-0
lines changed

1 file changed

+2
-0
lines changed

.github/workflows/ai-config-validation.yml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -294,6 +294,8 @@ jobs:
294294
--api-url http://localhost:8000 \
295295
--endpoint /chat \
296296
--minimal-payload \
297+
--judge-model claude-3-5-sonnet-20241022 \
298+
--judge-provider anthropic \
297299
--report test-report.json
298300
299301
TEST_EXIT_CODE=$?

0 commit comments

Comments
 (0)