Skip to content

ci: make Chromatic visual check a real approval gate (and unblock non-UI/fork PRs)#18809

Draft
pettinarip wants to merge 1 commit into
devfrom
worktree-chromatic-required-gate
Draft

ci: make Chromatic visual check a real approval gate (and unblock non-UI/fork PRs)#18809
pettinarip wants to merge 1 commit into
devfrom
worktree-chromatic-required-gate

Conversation

@pettinarip

Copy link
Copy Markdown
Member

Problem

PR #18806 (a content-only change from an external fork) is stuck on the required check "Visual regression (Chromatic)", which never reports. Root cause is two-fold:

  1. The required check comes from a conditionally-skipped job. The visual-tests job (whose name is the required context) only runs for dev PRs that touch UI paths. Content-only PRs skip it. For internal PRs a skipped job reports skipped = success, so they merge fine.
  2. Fork PRs never even get that far. External-fork CI runs are held at action_required (they carry secrets). Until a maintainer approves the run, the job never executes, so the required context is never posted — the PR sits in "Expected — waiting for status to be reported" forever.

Separately, the check enforces nothing today: exitZeroOnChanges: true makes it go green even when there are unaccepted visual changes.

Changes

1. ci.yml — make the check a real approval gate.
exitZeroOnChanges: truefalse on the required visual-tests job. Now a PR with unaccepted visual changes fails the check and stays blocked until a maintainer accepts the diffs in Chromatic and re-runs the job. (autoAcceptChanges: {dev,staging,master} still auto-accepts release-train head branches, so only feature PRs require manual acceptance.)

2. New chromatic-required-status.yml — unblock non-UI PRs, including forks.
A pull_request_target reporter that posts the required Visual regression (Chromatic) status as success for dev PRs that touch no UI code. It runs in base-repo context (no approval needed) and only reads the PR's changed-file list — it never checks out or executes fork code, so there's no secret-exfiltration surface. Its path filter mirrors the chromatic-gate filter in ci.yml.

Behavior after this PR

PR type Result
Content-only, internal ✅ merges (reporter + skipped job both green)
Content-only, fork (e.g. #18806) ✅ merges without CI approval — reporter posts success
UI change, internal ⛔ blocked until visual diffs accepted in Chromatic + job re-run
UI change, fork ⛔ needs run-approval, then visual acceptance (both required, by design)

Notes / limitations

  • Fork UI PRs still require a maintainer to approve the CI run before visual review can happen. That's inherent to keeping secrets in CI and can't be removed safely.
  • The Actions job going red on unaccepted changes clears by accepting in Chromatic and re-running the job (or pushing a new commit) — accepting alone doesn't retroactively green a past run.
  • Reporter is scoped to branches: [dev] because that's the only base where visual-tests is skipped for content PRs; master/staging/test always run the real job.
  • Does not touch page-visual-tests (not a required check).

Immediate unblock for #18806

Independent of this PR, a maintainer can unblock #18806 now by clicking "Approve and run workflows" (the skipped job then reports success), or by admin-merging (enforce_admins is off).

Set exitZeroOnChanges: false on the required visual-tests job so PRs with
unaccepted visual changes fail the check until a maintainer accepts them in
Chromatic and re-runs the job.

Add a pull_request_target reporter that posts the required
"Visual regression (Chromatic)" status as success for dev PRs that touch no
UI code. This unblocks content-only PRs -- including fork PRs whose CI
workflow is held for manual approval and therefore never reports the check
(e.g. #18806).
@netlify

netlify Bot commented Jul 15, 2026

Copy link
Copy Markdown

Deploy Preview for ethereumorg ready!

Name Link
🔨 Latest commit 39dcb79
🔍 Latest deploy log https://app.netlify.com/projects/ethereumorg/deploys/6a57aaf571cfa50008ad1f2b
😎 Deploy Preview https://deploy-preview-18809.ethereum.it
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.
Lighthouse
Lighthouse
7 paths audited
Performance: 58 (🟢 up 2 from production)
Accessibility: 95 (no change from production)
Best Practices: 100 (no change from production)
SEO: 98 (no change from production)
PWA: 59 (🔴 down 1 from production)
View the detailed breakdown and full score reports

To edit notification comments on pull requests, go to your Netlify project configuration.

@github-actions github-actions Bot added the tooling 🔧 Changes related to tooling of the project label Jul 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

tooling 🔧 Changes related to tooling of the project

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant