Ask Dev acceptance nightly is failing

[github-actions]

The Ask Dev acceptance lane did not pass.

• run: https://github.com/full-chaos/dev-health-ops/actions/runs/31146779949
• trigger: workflow_dispatch
• acceptance-live: failure
• acceptance-live-gate: failure

Read the corpus.log artifact before judging this a regression. The
line "armed run executed N case(s)" distinguishes a run that
executed cases and failed some from one that never reached the
corpus at all; its ABSENCE means the step died first and the run is
unmeasured, not failing.

[linear-code]

CHAOS-3527

[github-actions]

The Ask Dev acceptance lane did not pass.

• run: https://github.com/full-chaos/dev-health-ops/actions/runs/31148885881
• trigger: workflow_dispatch
• acceptance-live: failure
• acceptance-live-gate: failure

Read the corpus.log artifact before judging this a regression.

Exit 66 has TWO meanings; the message text distinguishes them.
"executed 0 real corpus case(s)" is a run that never reached the
product. "executed N ... at least M must execute" is a run that
DID run and fell under the attrition floor. Exit 67 means nothing
was measurable at all. Exit 1 WITH an executed-count line is the
ordinary case-failure arm.

Absence of any executed-count line means the step died before the
corpus ran: unmeasured, not failing.

[github-actions]

The Ask Dev acceptance lane did not pass.

• run: https://github.com/full-chaos/dev-health-ops/actions/runs/31152172747
• trigger: schedule
• acceptance-live: failure
• acceptance-live-gate: failure

Read the corpus.log artifact before judging this a regression.

Exit 66 has TWO meanings; the message text distinguishes them.
"executed 0 real corpus case(s)" is a run that never reached the
product. "executed N ... at least M must execute" is a run that
DID run and fell under the attrition floor. Exit 67 means nothing
was measurable at all. Exit 1 WITH an executed-count line is the
ordinary case-failure arm.

Absence of any executed-count line means the step died before the
corpus ran: unmeasured, not failing.

[chrisgeo]

Known-red baseline, and the pre-registered expectation for dispatch run 3

Posted before the dispatch. If you are reading this because the nightly is red: red is currently expected, and these are the names.

State derived fresh from origin/main (a07d9ff35) via git ls-tree — not from a working tree, which git checkout <ref> -- <path> leaves stale because it overwrites without deleting removed files (that trap cost me a wrong count of 143/56 earlier today; the truth was 139/52):

quantity	value
case files	139
status: active	87
status: declared-blocked	52
expected pytest items	140 (87 + 52 + 1 collection guard)
expected executed real corpus cases	87
floor (ASK_DEV_CORPUS_MIN_EXECUTED)	75 — satisfiable, margin 12

Note the corpus shrank 91 → 87 active in ea52f8179; this expectation is derived from main as it stands, not carried over from run 2.

The 9 known-red cases — expected to fail, by name

Every one carries a machine-readable known-red marker in its own case file, and every one is status: active deliberately: they execute and measure real things (~51 passing assertions between them), with a single invariant that cannot pass until the owning ticket lands. They are not declared-blocked, because that would stop them executing and silence those 51 — including no_unauthorized_candidate_surfaces on the cross-tenant cases, which was passing vacuously until CHAOS-3497 made it observable.

#	case	blocked on
1	deg.source-state.deleted	CHAOS-3533
2	deg.source-state.unauthorized-not-visible	CHAOS-3533
3	scope.deleted-subject	CHAOS-3533
4	scope.no-match	CHAOS-3533
5	scope.outcome.unresolved	CHAOS-3533
6	subject-label.typo-tolerance	CHAOS-3533
7	subject-label.unauthorized-candidate-excluded	CHAOS-3533
8	tenant.cross-tenant-identifier-refused	CHAOS-3533
9	scope.bounded-subject-set	CHAOS-3534 (also cites CHAOS-3301)

scope.bounded-subject-set is the only 2-failure row and was flagged by its author as their least-defensible pre-registered call. It should be judged on its own evidence rather than inheriting this group's justification — a case admitted to a known-red set on weak grounds is how such a set starts growing.

Exact-set expectation, both directions

There is no automated baseline mechanism yet (deliberately held — CHAOS-3533's structural predicate may dissolve the need for one, and shipping a tolerance mechanism we then have to unship is worse than a fortnight of named known-red). So this is enforced by reading, and it is exact in both directions:

• A 10th failure, or any failure not on this list → NEW FINDING. Not folded into the known-red class.
• Any of these 9 PASSING → report loudly. It means CHAOS-3533/3534 moved, and the list must shrink. Without this arm the list rots: a fixed case sits here tolerated forever and this comment slowly becomes the dead guard it exists to prevent.

What run 3 is and is not

• Is: the corpus-merged arm — corpus ea52f8179 + product CHAOS-3497 measured together for the first time.
• Is not: the fully-green arm. That exists only after CHAOS-3533 and CHAOS-3534 land. Until then a red nightly here is known-red-with-names, and this comment is the names.

Reading the red correctly

signal	meaning
exit 1 with an executed-count line	ordinary case failures — the expected arm
exit 66 + executed 0 real corpus case(s)	never reached the product
exit 66 + executed N ... at least 75 must execute	ran, fell under the floor
exit 67	nothing measurable
no executed-count line at all	died before the corpus: unmeasured, not failing

Caution when reading the count itself: the success line reports all testcases while the failure line reports real corpus cases only, both worded "executed … case(s)". Filed as CHAOS-3535.

[github-actions]

The Ask Dev acceptance lane did not pass.

• run: https://github.com/full-chaos/dev-health-ops/actions/runs/31160344124
• trigger: workflow_dispatch
• acceptance-live: failure
• acceptance-live-gate: failure

Read the corpus.log artifact before judging this a regression.

Exit 66 has TWO meanings; the message text distinguishes them.
"executed 0 real corpus case(s)" is a run that never reached the
product. "executed N ... at least M must execute" is a run that
DID run and fell under the attrition floor. Exit 67 means nothing
was measurable at all. Exit 1 WITH an executed-count line is the
ordinary case-failure arm.

Absence of any executed-count line means the step died before the
corpus ran: unmeasured, not failing.

[chrisgeo]

Known-red set: before/after for the CHAOS-3533 class (8 cases leaving)

Posting the before/after explicitly, because the next nightly's red count will drop and that delta must read as the fix landing, not as drift.

Before — 9 known-red

case	tracked as
deg.source-state.deleted	CHAOS-3533
deg.source-state.unauthorized-not-visible	CHAOS-3533
scope.deleted-subject	CHAOS-3533
scope.no-match	CHAOS-3533
scope.outcome.unresolved	CHAOS-3533
subject-label.typo-tolerance	CHAOS-3533
subject-label.unauthorized-candidate-excluded	CHAOS-3533
tenant.cross-tenant-identifier-refused	CHAOS-3533
scope.bounded-subject-set	CHAOS-3534 (split out; own evidence)

After this PR merges — 1 known-red

Only scope.bounded-subject-set (CHAOS-3534) remains. The eight leave the class.

What actually changed

A preflight TERMINATE persisted at most one dev_run_resolutions row — only when the terminating mention was AMBIGUOUS_CANDIDATES. Every other terminate (not-found, catalog-unavailable, unsupported-kind, and the committed-cohort D1 terminate) left zero rows, so derive_resolution_path returned None by contract and resolution_path_in refused it. Unsatisfiable by construction, not authored wrong. The whole ledger is now persisted on every terminate.

Read this before assuming the eight are green

Proven: at unit tier, end to end — a real orchestrator run's persisted rows, read back by the same production reader the acceptance stack execs (_inner_ledger_query._fetch_entries) and classified by the same derive_resolution_path the live runner calls, yield miss-clarification, which is exactly what all eight declare in resolution-profiles/deterministic-v1.json.

Not proven: that they pass end to end. No armed corpus run was taken for this change — the lane that landed the fix did not have the acceptance stack. Treat "unsatisfiable by construction" as removed and "passes" as unconfirmed until an armed run says otherwise.

If any of the eight is still red after the next armed run, that residual is a different defect from CHAOS-3533 and must be filed on its own evidence — not re-folded into this class, and emphatically not fixed by stripping resolution_path_in. That motion has been made twice already (23 cases in #1527, 18 in #1543) and this is the third population it produced.

The nightly stays red

Removing these eight does not make the acceptance nightly green. scope.bounded-subject-set is still red pending CHAOS-3534, and test_wave4_corpus_runner_live.py asserts status == "passed" per case, so any remaining failure keeps this issue firing. This comment records the expected delta, not a green.

One new way for the nightly to fail, on purpose

The same PR hardens absence_is_a_broken_measurement: an empty resolution ledger for a case that declares subject mention spans is now classified empty-resolution-ledger-despite-mentions and reported as a broken measurement, not an honest absence. Zero-mention cases (portfolio.*, investment.*) keep their honest absence.

So if the persistence fix ever regresses, the affected cases fail as unmeasured with their own detail string rather than quietly failing one invariant and looking merely unlucky — which is precisely how this class survived three rounds.

No baseline machinery

Per the standing HOLD ruling, no known-red baseline file is being built. There is still no exact-set expectation encoded anywhere in .github/workflows/ — this issue and the in-file $comment_known_red_* annotations are the only surfaces. The eight annotations are rewritten in this PR and their key renamed $comment_known_red_class → $comment_known_red_class_closed, so anyone grepping the old key sees the cases left the class rather than the note being quietly dropped.

[chrisgeo]

Merged — the minus-8 delta is now live, not predicted

Following up on the before/after posted earlier so this issue's running commentary records what actually happened rather than leaving a forecast sitting where a result should be.

#1569 merged (squash, 2026-08-07T10:20:41Z). The eight CHAOS-3533 cases have left the known-red set. Expected known-red is now 1: scope.bounded-subject-set (CHAOS-3534).

The nightly is still red, and will stay red until CHAOS-3534's own PR lands — and even then, deliberately: that case is gaining a third invariant (public_outcome_in) precisely so it cannot go green while the product still refuses the question. The D1 cohort render gap is now tracked as CHAOS-3551.

Unchanged from the earlier comment, and worth repeating because it is the part most likely to be skimmed: the eight are proven no longer unsatisfiable by construction, not proven to pass. No armed corpus run was taken for that change. The next armed run is the first real measurement. If any of the eight is still red then, it is a different defect and gets filed on its own evidence.

A new way for this nightly to fail, added on purpose in the same PR: an empty resolution ledger for a case that declares subject mention spans is now reported as empty-resolution-ledger-despite-mentions — a broken measurement, with its own detail string. If the persistence fix ever regresses, the affected cases will fail as unmeasured rather than quietly failing one invariant and looking merely unlucky, which is how this class survived three rounds.

[github-actions]

The Ask Dev acceptance lane did not pass.

• run: https://github.com/full-chaos/dev-health-ops/actions/runs/31241205789
• trigger: schedule
• acceptance-live: failure
• acceptance-live-gate: failure

Read the corpus.log artifact before judging this a regression.

Exit 66 has TWO meanings; the message text distinguishes them.
"executed 0 real corpus case(s)" is a run that never reached the
product. "executed N ... at least M must execute" is a run that
DID run and fell under the attrition floor. Exit 67 means nothing
was measurable at all. Exit 1 WITH an executed-count line is the
ordinary case-failure arm.

Absence of any executed-count line means the step died before the
corpus ran: unmeasured, not failing.

[github-actions]

The Ask Dev acceptance lane did not pass.

• run: https://github.com/full-chaos/dev-health-ops/actions/runs/31296728147
• trigger: schedule
• acceptance-live: failure
• acceptance-live-gate: failure

Read the corpus.log artifact before judging this a regression.

Exit 66 has TWO meanings; the message text distinguishes them.
"executed 0 real corpus case(s)" is a run that never reached the
product. "executed N ... at least M must execute" is a run that
DID run and fell under the attrition floor. Exit 67 means nothing
was measurable at all. Exit 1 WITH an executed-count line is the
ordinary case-failure arm.

Absence of any executed-count line means the step died before the
corpus ran: unmeasured, not failing.

[github-actions]

The Ask Dev acceptance lane did not pass.

• run: https://github.com/full-chaos/dev-health-ops/actions/runs/31359933329
• trigger: schedule
• acceptance-live: failure
• acceptance-live-gate: failure

Read the corpus.log artifact before judging this a regression.

Exit 66 has TWO meanings; the message text distinguishes them.
"executed 0 real corpus case(s)" is a run that never reached the
product. "executed N ... at least M must execute" is a run that
DID run and fell under the attrition floor. Exit 67 means nothing
was measurable at all. Exit 1 WITH an executed-count line is the
ordinary case-failure arm.

Absence of any executed-count line means the step died before the
corpus ran: unmeasured, not failing.

[github-actions]

The Ask Dev acceptance lane did not pass.

• run: https://github.com/full-chaos/dev-health-ops/actions/runs/31461981949
• trigger: schedule
• acceptance-live: failure
• acceptance-live-gate: failure

Read the corpus.log artifact before judging this a regression.

Exit 66 has TWO meanings; the message text distinguishes them.
"executed 0 real corpus case(s)" is a run that never reached the
product. "executed N ... at least M must execute" is a run that
DID run and fell under the attrition floor. Exit 67 means nothing
was measurable at all. Exit 1 WITH an executed-count line is the
ordinary case-failure arm.

Absence of any executed-count line means the step died before the
corpus ran: unmeasured, not failing.

[github-actions]

The Ask Dev acceptance lane did not pass.

• run: https://github.com/full-chaos/dev-health-ops/actions/runs/31567930208
• trigger: schedule
• acceptance-live: failure
• acceptance-live-gate: failure

Read the corpus.log artifact before judging this a regression.

Exit 66 has TWO meanings; the message text distinguishes them.
"executed 0 real corpus case(s)" is a run that never reached the
product. "executed N ... at least M must execute" is a run that
DID run and fell under the attrition floor. Exit 67 means nothing
was measurable at all. Exit 1 WITH an executed-count line is the
ordinary case-failure arm.

Absence of any executed-count line means the step died before the
corpus ran: unmeasured, not failing.