Skip to content

fix(ci): skip review policy for check events on non-head commits - #1027

Open
rl-block wants to merge 1 commit into
mainfrom
ci/review-policy-skip-non-head-check-events
Open

fix(ci): skip review policy for check events on non-head commits#1027
rl-block wants to merge 1 commit into
mainfrom
ci/review-policy-skip-non-head-check-events

Conversation

@rl-block

@rl-block rl-block commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

Reviewable diff: +12/-0 across 1 file (excludes generated, test, and story files).

Summary

Review Policy currently starts one evaluation for every check_run / check_suite completion in the repository. External check apps post one check run per commit of a pull request (the DCO check does), so a single push or body edit on a 56-commit PR started 56 Review Policy runs, 55 of which allocated a runner only to log No pull request was associated with this event and exit. The workflow's run counter is at ~100,900 against ~6,700 for PR Gate, almost all from this fan-out. This change drops check events for commits that are not an open pull request head before a runner is allocated. Fork PRs keep today's behaviour.

How it works

GitHub fills check_run.pull_requests (and check_suite.pull_requests) only when the check's commit is the head of an open pull request in the same repository, and leaves check_suite.head_branch null when the commit came from a fork (Checks API, webhook payloads). The workflow already classifies some events as ignorable in three places: the concurrency group (so ignored events never cancel a real evaluation), cancel-in-progress, and the evaluate job's if:. The same new condition is added to all three:

github.event_name == 'check_run'
  && github.event.check_run.check_suite.head_branch != null   # same-repo commit
  && github.event.check_run.pull_requests[0] == null          # not an open PR head

and its check_suite twin. A matching event still creates a workflow run entry (GitHub creates one for every subscribed event), but the job is skipped in the scheduler with no runner time. The head-commit event still has pull_requests populated, so the policy re-evaluates exactly once when an external check lands, as before. Fork events have head_branch == null, fail the first clause, and go through the existing API-based resolution.

Diagrams

flowchart LR
    A["Push / PR edit"] --> B["DCO app posts one check run per commit"]
    B --> C["check_run: completed x N"]
    C --> D{"head_branch set and pull_requests empty?"}
    D -- "yes: not a PR head (N-1 events)" --> E["concurrency group ignored-run_id; evaluate job skipped, no runner"]
    D -- "no: PR head, or fork" --> F["Resolve pull request via API"]
    F --> G["Evaluate Review Policy"]
Loading

Areas of the code involved

Area / file What changed Why it matters for review
.github/workflows/review-policy.yml Two conditions added to the ignore list in the concurrency group, cancel-in-progress, and the evaluate job if:; a comment above concurrency: explains the fan-out and the fork caveat The three copies must stay identical; the fork guard (head_branch != null) is what keeps this from silently skipping external contributors' PRs
.github/scripts/evaluate_review_policy_test.py test_workflow_skips_check_events_for_commits_that_are_not_a_pr_head asserts the condition, including the fork guard, is present in all three expressions Test — pins the fix against later edits to the ignore lists

Key technical decisions & trade-offs

  • Filter in if: and concurrency, not in the resolve script. The script already returns "no pull request" for these events, but only after a runner has been allocated; the expression-level filter is the only place GitHub lets a workflow decline work for free. Alternative: leave as is and accept ~10 s of runner time per commit per push.
  • Fail open for forks. When head_branch is null the event is evaluated as today rather than skipped, at the cost of keeping the fan-out for fork PRs. Alternative: skip whenever pull_requests is empty, which would stop the policy from re-evaluating when the DCO check completes on a fork PR.
  • Keep the check_run trigger. The policy requires DCO Check from block-dco-check (.github/review-policy.json), so the event is load-bearing for the head commit; dropping the trigger was not an option.

Testing & validation

  • python3 .github/scripts/evaluate_review_policy_test.py: 73 tests pass, including the new one (the loader parses the workflow with Ruby's YAML, so the expressions are checked as written).
  • Pre-commit hooks pass (python-ruff).
  • Not covered: the expressions run only in GitHub's evaluator. Evidence for the behaviour they rely on comes from the DCO runs on feat(rollout): RolloutService API contract for firmware release channels #1014 at 2026-09-08T01:33:32Z: 56 DCO Check check runs (one per PR commit) followed by 56 Review Policy runs, 55 resolving to no pull request. After merge, the next push to a multi-commit PR should show those 55 as skipped runs with zero duration.

External check apps post one check run per commit of a pull request (the
DCO check does), and Review Policy subscribes to every check_run and
check_suite completion, so one push to a 56-commit PR started 56 evaluations
of which 55 found no pull request and exited. Drop events whose commit is not
an open pull request head before a runner is allocated, in the concurrency
classification and the job condition alike. GitHub fills pull_requests only
for same-repository heads and leaves head_branch null for forks, so fork
events keep today's behaviour and are still evaluated.
@rl-block
rl-block requested a review from a team as a code owner September 8, 2026 02:09
@github-actions github-actions Bot added github_actions Pull requests that update GitHub Actions code review-policy: needs-review Managed by the Review Policy workflow. labels Sep 8, 2026
@github-actions

github-actions Bot commented Sep 8, 2026

Copy link
Copy Markdown

🔐 Codex Security Review

Note: This is an automated security-focused code review generated by Codex.
It should be used as a supplementary check alongside human review.
False positives are possible - use your judgment.

Scope summary

  • Reviewed pull request diff only (f1715b3f5a541be82853efd3a39e6a248fb1eefc...26b6a20a241a9f79bdc49ab68456edfa13df84fc, exact PR three-dot diff)
  • Model: gpt-5.6-sol

💡 Click "edited" above to see previous reviews for this PR.


Review Summary

Overall Risk: NONE

Findings

No security, correctness, or reliability findings identified in the changed hunks.

Notes

The targeted workflow invariant test passes. The full test script could not run in the read-only review environment because no writable temporary directory was available.


Generated by Codex Security Review |
Triggered by: @rl-block |
Review workflow run

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

github_actions Pull requests that update GitHub Actions code review-policy: needs-review Managed by the Review Policy workflow.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant