Skip to content

Commit 88276fa

Browse files
stroland02claude
andcommitted
test: say why the pipeline abandoned, instead of only that it did
This test failed on a Linux runner and passed everywhere else, and the log carried `assert 'abandoned' == 'verified'` and nothing more. No production code had changed since the last run whose suite was green, so the cause is environmental -- but the run executes a compiler and a package download, and the failure message cannot tell a broken toolchain from a broken patch. The state already holds the answer. `abandon_reason`, `verify_ok`, `prepare_ok`, `static_fatal` and the rest are each set by the node that decided, so rendering them into the assertion costs nothing on a pass and turns the next failure into a diagnosis. Verified by flipping the expected value and reading what printed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
1 parent 4d2a4c3 commit 88276fa

1 file changed

Lines changed: 19 additions & 2 deletions

File tree

tests/test_pipeline_composes.py

Lines changed: 19 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -238,6 +238,23 @@ def _drive(finding: Finding, repo: RepoRef, store: GraphStore, adapter, remediat
238238
return state
239239

240240

241+
def _why(state: RunState) -> str:
242+
"""Everything the run recorded about why it ended where it did.
243+
244+
An outcome assertion that fails alone says a run abandoned and says nothing about what
245+
abandoned it, and this pipeline runs a compiler and a package download -- so a failure here
246+
is as likely to be the environment as the code, and the two are indistinguishable from
247+
`assert 'abandoned' == 'verified'`. That is not hypothetical: it is how this test failed on
248+
a Linux runner while passing locally, with the log carrying no reason.
249+
"""
250+
keys = (
251+
"abandon_reason", "diagnostics", "verify_ok", "prepare_ok", "verifiable", "verify_gap",
252+
"static_fatal", "fatal", "tier", "routing_row", "static_attempts",
253+
"replay_outcome", "replay_reason",
254+
)
255+
return "\n".join(f"{key}={state.get(key)!r}" for key in keys)
256+
257+
241258
def _verified_run(tmp_path, store):
242259
repo = _clone(tmp_path, "verified")
243260
finding = _seed(store, repo, "src/summarise.ts")
@@ -271,8 +288,8 @@ def test_a_finding_reaches_a_verified_patch_through_the_real_graph(tmp_path, sto
271288
"""
272289
state, repo, _ = _verified_run(tmp_path, store)
273290

274-
assert state["outcome"] == "verified"
275-
assert state["verify_ok"] is True
291+
assert state["outcome"] == "verified", _why(state)
292+
assert state["verify_ok"] is True, _why(state)
276293
source = Path(repo.local_path, "src", "summarise.ts").read_text(encoding="utf-8")
277294
assert REPLACEMENT in source
278295
assert RETIRED not in source

0 commit comments

Comments
 (0)