Skip to content

ci: stop e2e blaming PRs for the base branch's specs and for cancellations #15660

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
christian-byrne wants to merge 2 commits into
base: main
Choose a base branch
from
+12 −2
Open

ci: stop e2e blaming PRs for the base branch's specs and for cancellations #15660

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
08b8bd7
ci: detect new spec files against the merged base, not the stale PR base
christian-byrne Aug 23, 2026
d68dc7e
ci: report a cancelled e2e run as cancelled, not failed
christian-byrne Aug 23, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 12 additions & 2 deletions .github/workflows/ci-tests-e2e.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -220,6 +220,8 @@ jobs:
VIDEO: ${{ needs.playwright-video-new-tests.result }}
run: |
[[ "$SHOULD_RUN" != "true" ]] && echo "E2E skipped" && exit 0
echo "sharded=$SHARDED browsers=$BROWSERS video=$VIDEO"
[[ "$SHARDED" == "cancelled" || "$BROWSERS" == "cancelled" || "$VIDEO" == "cancelled" ]] && echo "E2E cancelled, not failed - rerun to get a result" && exit 1
[[ "$SHARDED" != "success" || "$BROWSERS" != "success" || ( "$VIDEO" != "success" && "$VIDEO" != "skipped" ) ]] && echo "E2E failed" && exit 1
echo "E2E passed"

Expand Down Expand Up @@ -268,8 +270,16 @@ jobs:
id: detect
run: |
git config --global --add safe.directory "$GITHUB_WORKSPACE"
BASE_SHA="${{ github.event.pull_request.base.sha }}"
NEW_FILES=$(git diff --name-only --diff-filter=A "$BASE_SHA"...HEAD -- browser_tests/tests | grep '\.spec\.ts$' || true)
# HEAD is refs/pull/N/merge, so its first parent is the base tip the PR was merged
# onto. pull_request.base.sha is recorded when the PR opens and does not follow the
# base branch, so diffing against it attributes every spec the base gained since
# then to this PR.
if git rev-parse --verify --quiet HEAD^2 >/dev/null; then
BASE_SHA=$(git rev-parse HEAD^1)
else
BASE_SHA="${{ github.event.pull_request.base.sha }}"
fi
NEW_FILES=$(git diff --name-only --diff-filter=A "$BASE_SHA" HEAD -- browser_tests/tests | grep '\.spec\.ts$' || true)

if [ -z "$NEW_FILES" ]; then
echo "has-new-tests=false" >> "$GITHUB_OUTPUT"
Expand Down
Loading