Skip to content

Gate eval runs to avoid burning tokens on unrelated PRs #119

@coderabbitai

Description

@coderabbitai

Problem

The run-evals job in the CI workflow currently executes on every PR, including those with unrelated changes (e.g., docs-only). This wastes OpenAI/Phoenix API tokens, increases CI time, and risks throttling.

Suggested Solution

Add a paths filter (e.g., dorny/paths-filter) to conditionally run the eval steps only when relevant files change (eval configs, connector configs, or eval-related code).

Context

This issue was identified during the review of PR #116.

References:

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions