Skip to content

ci: make compat-matrix engine-job failures fail the run - #3376

Merged
bpamiri merged 1 commit into
developfrom
peter/3302-flip-continue-on-error
Aug 5, 2026
Merged

ci: make compat-matrix engine-job failures fail the run#3376
bpamiri merged 1 commit into
developfrom
peter/3302-flip-continue-on-error

Conversation

@bpamiri

@bpamiri bpamiri commented Aug 5, 2026

Copy link
Copy Markdown
Collaborator

What

Deletes continue-on-error: true from the tests (engine) job in .github/workflows/compat-matrix.yml. The rustcfml job keeps its flag — it is an informational lane by design.

Why now — the gate is cleared

The issue's own sequencing rule was: burn down the pre-existing leg debt first, then flip. That happened today:

  1. PR fix(compat): burn down four compat-matrix root causes (#3302) #3365 (debt burn-down) merged as 6bff05441.
  2. Confirming dispatch 30976993285 on develop: all 5 engine jobs green, aggregate 132,170 pass with only the 6 oracle soft-fails (non-blocking via SOFT_FAIL_DBS, tracked under ci: Oracle datasource broken across all engines — Invalid URL, missing DBMS_LOCK, constraint cleanup #2663).
  3. The safe-slice hardening (zero-test guard, neutral soft-fail check, honest summary grid) merged earlier via ci: compat-matrix safe-slice hardening — zero-test guard, neutral soft-fail check, honest summary #3366 and validated live on dispatch runs.

With this flip, a failing engine leg fails the run — the weekly banner becomes a real alarm instead of decoration.

Fixes #3302

🤖 Generated with Claude Code

Removes continue-on-error from the tests job now that the debt gate is
cleared: PR #3365 merged and dispatch run 30976993285 on develop came back
fully green (all 5 engine jobs, aggregate 6 oracle soft-fails only, which
stay non-blocking via SOFT_FAIL_DBS under #2663). The rustcfml job keeps
its continue-on-error — it is an informational lane by design.

From now on a red engine leg turns the weekly run banner red instead of
being silently swallowed.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Peter Amiri <petera@pai.com>

@wheels-bot wheels-bot Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wheels Bot — Reviewer

TL;DR — This PR removes continue-on-error: true from the tests (engine) job in .github/workflows/compat-matrix.yml, so a failing engine leg now fails the compat-matrix run instead of being silently swallowed. The change is minimal, correct, and well-justified by the linked debt burn-down (#3365 / 6bff05441). One minor docs nit keeps this at comment rather than approve; nothing blocking.

Correctness

The one-line deletion does exactly what the title claims. Verified the surrounding mechanics hold up:

  • The job-internal gating already distinguishes soft-fails from real failures — SOFT_FAIL_DBS="oracle" only warns (.github/workflows/compat-matrix.yml:476-480), and OVERALL_STATUS drives the final exit 1 (:502-505). So removing the blanket continue-on-error promotes only genuine, non-soft-fail leg failures to run failures. Correct.
  • fail-fast: false is retained (:19), so one failing leg still lets the other four engines finish — you get the full matrix picture on a failure, not a truncated one.
  • Both downstream jobs guard with if: always() (publish-results at :631, test-matrix-summary at :668), so a now-failing tests job does not suppress the JUnit publish, PR matrix comment, or artifact upload. No regression there.
  • The rustcfml lane correctly keeps its own continue-on-error: true (:605), consistent with the in-file comment describing it as a never-gating informational lane (:594-601).

No blocking findings.

Docs

  • .github/workflows/compat-matrix.yml:1-2 — the file-header comment still reads Non-blocking — informational only. That is now stale for the engine legs: after this PR a failing engine job does block the run. Consider tightening it to note that only the rustcfml lane (and oracle soft-fail DBs) remain informational. Non-blocking nit.

Commits

ci: make compat-matrix engine-job failures fail the run conforms to commitlint.config.js — valid type ci, no scope, subject is 47 chars and not ALL-CAPS. Good.

No test changes are expected or needed for a CI-config-only diff.

@bpamiri
bpamiri merged commit a9d2ff4 into develop Aug 5, 2026
6 checks passed
@bpamiri
bpamiri deleted the peter/3302-flip-continue-on-error branch August 5, 2026 05:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

compat-matrix: engine-job failures are invisible (continue-on-error) — harden after burning down pre-existing leg debt

1 participant