Skip to content

Commit d95d94e

Browse files
Unify /ok-to-test as single trigger for all E2E suites (#898)
* Unify /ok-to-test as single trigger for all E2E suites Replace /trigger-e2e-full, /test-e2e-full, and /test-full with /ok-to-test in ci-pr-checks.yaml so maintainers use one command to trigger both the full Kind E2E and OpenShift E2E workflows. /retest remains as an OpenShift-only re-run command. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Address review comments: fix permission wording and clarify concurrency - Docs: change "Maintainers/admins only" to "Users with write access" to match the actual workflow permission check (admin/maintain/write) - Clarify concurrency header comment to note workflow_dispatch falls back to run_id since no PR number is available Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Lionel Villard <villard@us.ibm.com> --------- Signed-off-by: Lionel Villard <villard@us.ibm.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
1 parent d09f5a8 commit d95d94e

2 files changed

Lines changed: 29 additions & 32 deletions

File tree

.github/workflows/ci-pr-checks.yaml

Lines changed: 23 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -1,23 +1,22 @@
11
name: CI - PR Checks
22

33
# Cancel previous runs on the same PR when new commits are pushed
4-
# Only group by PR number for legitimate triggers (pull_request, workflow_dispatch, e2e trigger comments)
4+
# Group by PR number for pull_request and /ok-to-test comment triggers
55
# Regular comments get a unique group (run_id) so they don't cancel in-progress test runs
6+
# workflow_dispatch falls back to run_id (no PR number available)
67
#
78
# Logic:
8-
# - Regular comments (not e2e trigger): unique group prevents cancellation of real tests
9-
# - Valid triggers: group 'ci-pr-checks-{pr_number}' (can cancel previous runs for same PR)
9+
# - Regular comments (not /ok-to-test): unique group prevents cancellation of real tests
10+
# - pull_request / /ok-to-test comments: group 'ci-pr-checks-{pr_number}' (can cancel previous runs for same PR)
11+
# - workflow_dispatch: group 'ci-pr-checks-{run_id}' (unique per run)
1012
# - Fallback chain for ID: pull_request.number -> issue.number -> run_id
1113
#
12-
# NOTE: Accepted commands (/trigger-e2e-full, /test-e2e-full, /test-full) must stay in sync
13-
# with check-full-tests job validation
14+
# NOTE: Accepted command (/ok-to-test) must stay in sync with check-full-tests job validation
1415
concurrency:
1516
group: >-
1617
${{
1718
github.event_name == 'issue_comment' &&
18-
!contains(github.event.comment.body, '/trigger-e2e-full') &&
19-
!contains(github.event.comment.body, '/test-e2e-full') &&
20-
!contains(github.event.comment.body, '/test-full')
19+
!contains(github.event.comment.body, '/ok-to-test')
2120
&& format('comment-isolated-{0}', github.run_id)
2221
|| format('ci-pr-checks-{0}',
2322
github.event.pull_request.number
@@ -39,7 +38,7 @@ on:
3938
required: false
4039
default: false
4140
type: boolean
42-
# Allow triggering via PR comments
41+
# Allow triggering via /ok-to-test PR comment
4342
issue_comment:
4443
types: [created]
4544

@@ -85,7 +84,7 @@ jobs:
8584
ref: ${{ github.event_name == 'issue_comment' && steps.pr-info.outputs.pr_head_sha || github.event.pull_request.head.sha || github.sha }}
8685
token: ${{ secrets.GITHUB_TOKEN }}
8786
fetch-depth: 0
88-
87+
8988
- name: Check for code changes
9089
uses: dorny/paths-filter@v3
9190
id: filter
@@ -99,12 +98,12 @@ jobs:
9998
- '!LICENSE'
10099
- '!OWNERS'
101100
- '!PROJECT'
102-
101+
103102
- name: Set output with default
104103
id: set-output
105104
run: |
106105
# Use filter output if available, otherwise default to 'true' for issue_comment events
107-
# This ensures /trigger-e2e-full works even if PR context is unclear
106+
# This ensures comment triggers work even if PR context is unclear
108107
FILTER_OUTPUT="${{ steps.filter.outputs.code }}"
109108
if [ "${{ github.event_name }}" == "issue_comment" ] && [ -z "$FILTER_OUTPUT" ]; then
110109
echo "has_code_changes=true" >> $GITHUB_OUTPUT
@@ -191,21 +190,21 @@ jobs:
191190
core.setOutput('run_full', 'true');
192191
return;
193192
}
194-
195-
// Check for /trigger-e2e-full comment trigger
193+
194+
// Check for /ok-to-test comment trigger
196195
if (context.eventName === 'issue_comment') {
197196
const comment = context.payload.comment.body.trim();
198197
const issue = context.payload.issue;
199198
200-
// Only process /trigger-e2e-full comments on PRs
199+
// Only process /ok-to-test comments on PRs
201200
if (!issue.pull_request) {
202201
console.log('Comment is not on a PR, skipping');
203202
core.setOutput('run_full', 'false');
204203
return;
205204
}
206205
207-
// Accept multiple command variants (case-sensitive, exact match)
208-
const validCommands = ['/trigger-e2e-full', '/test-e2e-full', '/test-full'];
206+
// Accept /ok-to-test (same command triggers both Kind full E2E and OpenShift E2E)
207+
const validCommands = ['/ok-to-test'];
209208
if (!validCommands.includes(comment)) {
210209
console.log(`Comment "${comment}" is not a valid trigger command, skipping`);
211210
core.setOutput('run_full', 'false');
@@ -245,13 +244,13 @@ jobs:
245244
owner: context.repo.owner,
246245
repo: context.repo.repo,
247246
issue_number: issue.number,
248-
body: `🚀 **Kind E2E (full)** triggered by \`${comment}\`\n\n[View the Kind E2E workflow run](${runUrl})`
247+
body: `🚀 **Kind E2E (full)** triggered by \`/ok-to-test\`\n\n[View the Kind E2E workflow run](${runUrl})`
249248
});
250249
251250
core.setOutput('run_full', 'true');
252251
return;
253252
}
254-
253+
255254
core.setOutput('run_full', 'false');
256255
257256
# E2E smoke tests - run automatically on every PR with code changes
@@ -305,13 +304,13 @@ jobs:
305304
IMAGE_TAG="pr-${GITHUB_RUN_ID}-${CHECKOUT_SHA:0:7}"
306305
# Use localhost prefix for local-only image (Kind will load it directly)
307306
FULL_IMAGE="localhost/${IMAGE_NAME}:${IMAGE_TAG}"
308-
307+
309308
echo "Building local image: $FULL_IMAGE"
310309
echo "Image will be loaded into Kind cluster (no push needed)"
311-
310+
312311
# Build image locally (no push needed for Kind)
313312
make docker-build IMG="$FULL_IMAGE"
314-
313+
315314
echo "image=$FULL_IMAGE" >> $GITHUB_OUTPUT
316315
echo "image_tag=${IMAGE_TAG}" >> $GITHUB_OUTPUT
317316
echo "Image built locally: $FULL_IMAGE"
@@ -339,7 +338,7 @@ jobs:
339338
run: |
340339
make test-e2e-smoke-with-setup
341340
342-
# Full Kind E2E - runs only when triggered (/trigger-e2e-full or workflow_dispatch with run_full_tests).
341+
# Full Kind E2E - runs only when triggered (/ok-to-test or workflow_dispatch with run_full_tests).
343342
# Add "e2e-tests-full" as a required status check in branch protection so it appears on every PR
344343
# and must run and pass before merge (shows as "Expected" until triggered).
345344
e2e-tests-full:
@@ -445,7 +444,7 @@ jobs:
445444
make test-e2e-full-with-setup
446445
447446
# Report status back to PR for issue_comment triggered runs
448-
# This ensures fork PRs show the correct status after /trigger-e2e-full runs complete
447+
# This ensures fork PRs show the correct status after /ok-to-test runs complete
449448
report-status:
450449
runs-on: ubuntu-latest
451450
needs: [check-code-changes, check-full-tests, e2e-tests-full]

docs/developer-guide/testing.md

Lines changed: 6 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -262,20 +262,18 @@ Runs OpenShift E2E tests on dedicated cluster:
262262

263263
#### Triggering E2E via PR Comments
264264

265-
You can trigger specific e2e runs by commenting on a PR:
265+
You can trigger E2E runs by commenting on a PR:
266266

267267
| Comment | Workflow | Who can use | Effect |
268268
|--------|----------|-------------|--------|
269-
| **`/trigger-e2e-full`** | `ci-pr-checks.yaml` | Anyone with PR access | Runs the **full** e2e suite on Kind (instead of smoke only). Aliases: `/test-e2e-full`, `/test-full`. |
270-
| **`/ok-to-test`** | `ci-e2e-openshift.yaml` | Maintainers/admins only | **OpenShift E2E:** Approve and trigger the OpenShift E2E (GPU) run on this PR. Use on fork PRs (required before E2E can run) or to start E2E on demand. |
271-
| **`/retest`** | `ci-e2e-openshift.yaml` | Maintainers/admins only | **OpenShift E2E:** Re-run the OpenShift E2E workflow (e.g. after a failure, flake, or new commits). Same workflow as `/ok-to-test`, different trigger intent. |
272-
273-
Both `/ok-to-test` and `/retest` trigger the **same** OpenShift E2E workflow; the first is “approve and run,” the second is “run again.”
269+
| **`/ok-to-test`** | `ci-pr-checks.yaml` + `ci-e2e-openshift.yaml` | Users with write access | Runs the **full** Kind E2E suite **and** the OpenShift E2E (GPU) run on this PR. On fork PRs, this is required before OpenShift E2E can run. |
270+
| **`/retest`** | `ci-e2e-openshift.yaml` | Users with write access | **OpenShift E2E only:** Re-run the OpenShift E2E workflow (e.g. after a failure, flake, or new commits). Same workflow as `/ok-to-test`, different trigger intent. |
274271

275272
**When to use:**
276273

277-
- **Full e2e on Kind**: Comment `/trigger-e2e-full` when you want the full e2e suite to run on your PR (e.g. after making scaling or saturation changes). By default, PRs only run smoke e2e.
278-
- **Fork PRs (OpenShift E2E)**: If you opened a PR from a fork, OpenShift E2E will not run until a maintainer or admin comments **`/ok-to-test`** (approve and run OpenShift E2E). Use **`/retest`** to re-run OpenShift E2E (e.g. after failure or new commits). Branch protection should require the **e2e-openshift** status check so merge stays blocked until that run passes (the gate check is intentionally green on fork PRs to avoid a false failure that cannot be updated from upstream).
274+
- **`/ok-to-test`**: Comment this when you want the full E2E suite to run on your PR. It triggers both the full Kind E2E (instead of smoke only) and the OpenShift E2E. By default, PRs only run smoke E2E on Kind.
275+
- **`/retest`**: Use to re-run only the OpenShift E2E workflow (e.g. after a failure or new commits).
276+
- **Fork PRs**: If you opened a PR from a fork, OpenShift E2E will not run until a maintainer or admin comments **`/ok-to-test`**. Branch protection should require the **e2e-openshift** status check so merge stays blocked until that run passes (the gate check is intentionally green on fork PRs to avoid a false failure that cannot be updated from upstream).
279277

280278
### Running CI Tests Locally
281279

0 commit comments

Comments
 (0)