ci: run full e2e tests automatically on every PR#881
ci: run full e2e tests automatically on every PR#881lionelvillard wants to merge 1 commit intollm-d:mainfrom
Conversation
Remove the /trigger-e2e-full comment-gated mechanism and run e2e-tests-full alongside e2e-tests-smoke on every PR with code changes. This removes the check-full-tests job, report-status job, issue_comment trigger, and all associated fork-PR workarounds. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Lionel Villard <villard@us.ibm.com>
There was a problem hiding this comment.
Pull request overview
This PR updates the PR CI workflow to always run the full Kind e2e suite automatically (in addition to smoke) on PRs that include code changes, removing the prior comment-gated “full e2e” trigger mechanism and associated jobs/logic.
Changes:
- Removes
/trigger-e2e-fullcomment trigger, validation, and status-reporting jobs. - Runs
e2e-tests-fullautomatically alongsidee2e-tests-smokewhen code changes are detected. - Simplifies workflow concurrency grouping and PR-change detection wiring.
| FILTER_OUTPUT="${{ steps.filter.outputs.code }}" | ||
| if [ "${{ github.event_name }}" == "issue_comment" ] && [ -z "$FILTER_OUTPUT" ]; then | ||
| echo "has_code_changes=true" >> $GITHUB_OUTPUT | ||
| elif [ -n "$FILTER_OUTPUT" ]; then | ||
| if [ -n "$FILTER_OUTPUT" ]; then | ||
| echo "has_code_changes=$FILTER_OUTPUT" >> $GITHUB_OUTPUT | ||
| else | ||
| echo "has_code_changes=true" >> $GITHUB_OUTPUT | ||
| fi |
There was a problem hiding this comment.
check-code-changes defaults has_code_changes to true when paths-filter produces no output (e.g., if the step fails; it's currently continue-on-error: true). That can unintentionally run the e2e jobs on docs-only PRs, defeating the optimization. Consider failing fast when change detection fails, or defaulting to false for pull_request events (and explicitly forcing true for workflow_dispatch if you want manual runs to always execute e2e).
|
/ok-to-test |
|
🚀 OpenShift E2E — approve and run ( |
|
/trigger-e2e-full |
|
🚀 Kind E2E (full) triggered by |
GPU Pre-flight Check ✅GPUs are available for e2e-openshift tests. Proceeding with deployment.
|
clubanderson
left a comment
clubanderson
left a comment
There was a problem hiding this comment.
A few concerns about removing the /trigger-e2e-full gate entirely:
-
Abuse/cost exposure: Any fork contributor can now trigger 60-minute full Kind e2e runs on every push. The old comment-gate intentionally required write access — that was a deliberate security/cost decision, not accidental complexity.
-
Kind alone is not sufficient: With the OpenShift nightly e2e tests currently failing, settling for Kind-only validation on PRs is risky. Kind doesn't match the real deployment environment described in the guides. Maintainers need to be confident that
/ok-to-test(OpenShift e2e) or/trigger-e2e-fullare being used regularly before merge — removing the trigger mechanism makes it easier to skip that step. -
Smoke already runs on every PR: If smoke tests are catching regressions on every PR, the full Kind suite was designed as an escalation path that maintainers invoke when they want extra confidence. Running it unconditionally on every push adds cost without a clear benefit over the existing smoke + on-demand full model.
Suggestion: if the goal is to run full e2e more frequently, consider auto-running it only on non-fork PRs (from contributors with write access) while keeping the comment gate for forks. That simplifies the common case without removing the abuse protection.
|
@clubanderson what about merging both |
|
That sounds a bit better. Sure. |
|
will open a new PR. |
Pull request was closed
Summary
/trigger-e2e-fullcomment-gated mechanism for running full e2e testse2e-tests-fullautomatically alongsidee2e-tests-smokeon every PR with code changescheck-full-testsjob,report-statusjob,issue_commenttrigger, and all associated fork-PR workaroundscheck-code-changesjobTest plan
e2e-tests-smokeande2e-tests-fullboth trigger on a PR with code changesworkflow_dispatchstill works for manual runs🤖 Generated with Claude Code