-
Notifications
You must be signed in to change notification settings - Fork 2
Open
Description
Problem
The run-evals
job in the CI workflow currently executes on every PR, including those with unrelated changes (e.g., docs-only). This wastes OpenAI/Phoenix API tokens, increases CI time, and risks throttling.
Suggested Solution
Add a paths filter (e.g., dorny/paths-filter
) to conditionally run the eval steps only when relevant files change (eval configs, connector configs, or eval-related code).
Context
This issue was identified during the review of PR #116.
References:
Metadata
Metadata
Assignees
Labels
No labels