feat(harness): capture and check deliberate divergences instead of hand-writing them by Victoriakaey · Pull Request #151 · Victoriakaey/project-life-cycle

Victoriakaey · 2026-07-30T05:01:48Z

TL;DR

Problem — when a change to the context-floor hook is meant to change behaviour, the test harness needs both sides of that difference written down and measured by hand. That does not scale: one known one-line fix moves 57 of 105 rows.
What we did — added a capture-and-check pair. One command records what the new implementation actually does into a tracked, readable file; another re-runs everything and fails naming the row when a record no longer matches. CI runs only the checking half.
Why — hand-writing 57 expectations is how a pinned expectation stops being measured and starts being guessed, which is the one property this harness's own header says must never slip.
Result, with the honest boundary — proven end to end against the fix that motivated it: 12 rows go red, one capture command resolves them, reverting reddens them again. The fix itself is deliberately not in this PR, and the 45 rows that need real conversion are split into their own follow-up.

1. What was done

The differential harness compares two implementations of context-floor.sh row by row. Most rows must agree. A handful are deliberate leads — places where a phase intentionally made the new implementation behave differently — and those rows pin both sides to literals that the harness's own header insists must be captured from a real run, never reasoned out, because "a hand-derived expectation and a captured one are byte-identical in review".

Sixteen such rows are still hand-writable. Fifty-seven are not, and fifty-seven is what a single known one-line fix (B83) produces — measured, on the current corpus, not estimated.

This PR adds the missing half: the new side of a deliberate lead can now be captured into a git-tracked record and checked on every CI run.

Use cases

"I intentionally changed how the hook behaves and the harness went red."
Run the check to see which rows moved and how, then capture the new expectation for exactly those rows by naming them:

/bin/bash skills/project-lifecycle/hooks/test-context-floor-differential.sh --check-divergences
/bin/bash skills/project-lifecycle/hooks/test-context-floor-differential.sh --bless-divergences "marker/mk_dir" "marker/mk_grace"

Adding a record for a row that has none is unconditional. Touching a record you already have requires naming that row, because touching an existing expectation is the operation that can quietly turn a behaviour change green. A name that matches nothing is a hard error before anything runs — a typo silently taking the "add a new one" path would defeat the guard while the output read like success.

"Did anyone hand-edit an expectation instead of measuring it?"
CI answers this on every push, as its own invocation with its own exit code, naming any row that drifted. It has no path to the capture half by construction, so CI can never make the records agree with whatever the code currently does.

"Is this record still describing the implementation I'm running?"
Two separate verdicts are reported, never merged: DRIFT (the recorded text moved) and STALE (the record was captured from a different implementation). The plain suite run asks only the first, so "the port broke" stays distinguishable from "a record is out of date".

Files

skills/project-lifecycle/hooks/test-context-floor-differential.sh — the record format, both modes, the touch-existing gate, and the hybrid comparison (a row's NEW-side expectation comes from its record when it has one, from the corpus literal when it does not; OLD's side is always hand-written).
.claude/divergence-records.txt — 16 records, one per deliberate-lead row, with a prose header a reviewer reads before the first record.
.gitattributes — new file, path-scoped. The record format is length-prefixed, so a contributor whose git normalises line endings would break every stored byte count with nothing complaining until the check ran.
skills/project-lifecycle/hooks/test-differential-harness.sh — two new meta rows covering the record branch, plus their mutants in the file's mutation table.
.github/workflows/validate.yml — the check, inside the existing step, with an assertion that it actually ran.
CHANGELOG.md, docs/ ledger — corrections and four new filings.

2. Why this approach

The record is the expectation, not a copy of one. This is the load-bearing decision and the first implementation got it wrong. A pure sidecar — pins stay hand-written, the record only attests — was built across the first three tasks and then refuted by a run: under the B83 patch the twelve affected rows fail their hand-written literal first, so the comparison never reaches the record and no amount of re-capturing repairs them. That shape was the one the design document had already rejected in writing. It was rebuilt as the hybrid, with the human's approval at the point of discovery, because reinterpreting a locked decision is not an unattended call.

The generalisable form, which is worth more than the fix: a mechanism justified by a use case is not done until it has been run against that use case.

Old's side stays hand-written on purpose. The reference implementation is content-addressed by a pinned blob and the harness refuses to run if that blob moves, so it costs nothing to keep and it is the independent anchor the check compares against. Capturing both sides would leave the check nothing to disagree with.

Two verdicts rather than one. Requiring that a dirtied hook invalidate every record needs the implementation identity compared; keeping the 12-row text signal legible needs it not to be. Reporting both separately satisfies both and — more usefully — keeps did behaviour move and did identity move answerable independently.

Vacuous runs are a hard failure with no waiver, matching pytest's dedicated exit code for "no tests collected", Jest's fail-on-no-tests default, and this repo's own empty-corpus rule. Zero records, a record for a row that no longer exists, or a discovered row with no record all fail. The name comparison runs in both directions, because a copy-paste rename leaves the totals matching while one record describes a row that is gone.

3. Requirements satisfied

Every acceptance criterion is demonstrated with raw output in the PR comment, never prose.

AC	What it required	Where
AC1	a record per discovered lead row (16, not 12)	16 written against 16 discovered, names identical
AC2	unchanged tree exits 0	`checked 16 records`, 16 matched, rc=0
AC3	both the content-mismatch and unreadable-record branches, separately	both shown; attribution corrected — see spec §5.6 E2
AC4	the patch reddens the 12; one re-bless resolves	12 DRIFT; one invocation, 16 REBLESS, 0 REFUSED
AC5	vacuous run is a hard FAIL, no hardcoded constant	emptied file and removed record both shown
AC6	touching an existing record requires naming the row	16 REFUSED with none named; 1 REBLESS + 15 REFUSED with one named
AC7	the CI step fails if the check did not run	demonstrated against the shipped bytes with two controls
AC8	CI has no path to `--bless`	`grep -n bless .github/workflows/validate.yml` returns nothing
AC9	the control tally is unchanged	`85 passed, 0 failed, 0 skipped, 4 known-diverge, 16 deliberate-lead`, and a zero-delta 105-row per-line census
AC10	all 8 invariants still pass	8/8
AC11	the hash follows the implementation in use; dirtying it invalidates every record	`20b40056` → `5c77246f`; 16 STALE / 0 DRIFT
AC12	a path-scoped `.gitattributes`, proven by a normalising checkout	13278 → 13278 bytes while an unpinned control moved 2630 → 2735
AC13	an unrecognised row name is a hard error	rc=2 on a typo and on a stray trailing space; file byte-identical
AC14	the ran-it assertion fails against a deliberate early-exit stub	rc=1, with a control proving it is not always-red
AC15	an orphan record is reported even when the count matches	`checked 16 records` with 1 orphan + 1 missing

Three acceptance criteria were corrected by measurement rather than reworded to pass (spec §5.6): the sidecar-to-hybrid rebuild, AC3's branch attribution being off by one section, and AC4's "all 12" being exact but not exhaustive. Retired wording is kept verbatim.

Deliberately not here: B83's two-character fix and the 45 rows that need real conversion (split out as batch-map PR 10); a close-gate invariant for record freshness (rejected in the design, with the gap stated — the local pre-push layer cannot see a stale record, so red arrives from CI); user-story.md and dual-track smoke (internal test-rig tooling, no adopter-invoked surface — logged as SKIP: in the journal's Plan deviations).

… git-tracked provenance record A deliberate-lead row pins NEW's side as a hand-written literal in the corpus. That is affordable at 16 rows and unaffordable at the 57 rows B83's two-character fix breaks (measured 2026-07-29, against the filed 54), so this adds the capture half: `--bless-divergences` runs the corpus and writes a record for every LEAD row that does not have one. Comparison logic is deliberately NOT in this task. Record shape — one flat stream of the harness's own length-prefixed sections, no second encoder and specifically not JSON: every one of B79's four defects lived in a value encoder, and these payloads (`%%`, backslashes, multi-line stderr with significant trailing newlines) are exactly the shapes `emit_section`/`get_section` already handle byte-exactly. Four sections per row, and the identity fields are PER-ROW on purpose. D3 makes blessing granular, so a file-level hash could not describe the result of re-blessing one row — it would claim one implementation identity for text captured under two. The header states this, because the next reader's first instinct is to deduplicate it. `newimpl` is `git hash-object --no-filters` of the implementation ACTUALLY IN USE (C1/R2): the worktree hook, or the injected copy under PLC_CF_NEW. Hashing the committed blob instead would describe the tracked hook while the recorded text came from an injected copy, making the tagging inert exactly where it is load-bearing. C5 (spec R4) needed a value the plan did not name: `mk_dir`'s umask-derived bytes are a DIRECTORY mode, which the rig's existing `$FILEDEF` does not describe. A `$DIRDEF` probe is sampled the same way and both fold to placeholders in the record only — folding inside `normalize_record` would have moved all 16 pins and reddened the corpus for an unrelated reason. P1: the argument parser lives BELOW the library boundary and reads `${1:-}`. The meta-suite sources this file with no positional parameters, where a bare `$1` under `set -u` aborts the sourced library and takes a CI-gating suite red at this commit. C2/AC12: `.gitattributes` is created here, path-scoped. The repo had none, and `* text=auto` would renormalize the whole tree as a side effect of one file needing a guarantee. Evidence: differential 85 passed, 0 failed, 0 skipped, 4 known-diverge, 16 deliberate-lead (unchanged; AC9) per-line census zero delta against the pre-change baseline, 105 rows bless 16 records against 16 discovered LEAD rows, names identical; re-run writes 0 and leaves the file byte-identical round-trip 16/16 sections read back through get_section; header reads as prose; absent row refused meta 28/0 · contract 88/0/0/0 · test-hooks 101/0 · tasklist-view ALL PASS · validate OK · invariants 8/8 AC12 autocrlf checkout 13278 -> 13278 bytes, 0 CR, while the unpinned control file in the SAME checkout moved 2630 -> 2735 (+105 for 105 lines) AC11 (hash-source half) 20b4005 -> 5c77246f under PLC_CF_NEW, observation text byte-identical The first AC11 probe was vacuous — its `sed` matched nothing, so the "changed" copy was byte-identical and every line of the output looked like a pass. A `cmp -s` positive control in the same command caught it. Recorded in the journal rather than quietly rerun. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Rigp3VhvZdwJVUZKLLmfq2

…rison a count cannot make The capture half without a comparator is the wrong half to ship, so this adds the guard: re-run each deliberate-lead row, compare against its record, exit non-zero naming the row. A bare "mismatch occurred" is the filed complaint against `terraform fmt -check`'s exit 3; every finding here names its row. TWO VERDICTS, not one. AC11 requires that dirtying the hook invalidates every record, which needs the recorded implementation hash compared; AC4 requires that B83's patch reddens THE 12, which a hash-only comparison inflates to 16. `STALE` (captured under a different implementation) and `DRIFT` (the observation text moved) are reported independently, so behaviour and identity stay separable. Vacuous-run hard FAIL, no waiver: zero records, an orphan record, or a discovered row with no record. The set comparison is over NAMES in both directions, because AC15's copy-paste rename leaves the totals matching while one record describes a row that no longer exists. No constant is hardcoded -- the discovered side is accumulated from rows that actually reported LEAD, so a host that SKIPs marker/mk_samesecond compares 15 against 15. `checked N records` on stdout is the line AC7's CI-step assertion reads. A step that trusts only an exit code cannot tell "ran and found nothing wrong" from "never ran" -- the gofmt shape, and GitHub counting a skipped required check as passing. The record census walks the framed stream rather than grepping it: this harness's own observations embed framed sections, so a payload can legitimately contain a line shaped like a header, and a search-based census would count those. Demonstrations, run not argued (each rc=1 unless stated): AC2 unchanged tree -> rc=0, checked 16 records, 16 matched AC3a payload edited, prefix CORRECTED -> DRIFT, names marker/mk_dir, 15 matched 1 failed AC3b payload edited, prefix LEFT WRONG -> UNREADABLE RECORD branch, checked 5 of 16, 13 failed AC5a one record removed -> 1 missing, checked 15 records AC5b record file emptied -> VACUOUS RUN, checked 0 records, 17 failed AC15 a row renamed in the file -> count MATCHES at 16, yet 1 orphan + 1 missing AC11 inert hook change via PLC_CF_NEW -> 16 STALE, 0 DRIFT, 0 matched 16 failed MEASURED CORRECTION to AC3's wording, folded into the spec at T6: an uncorrected prefix does NOT make `get_section` refuse the EDITED row. A shortened payload under a too-large prefix over-reads, so the edited row reports DRIFT and its NEIGHBOUR reports UNREADABLE. Both branches are demonstrated and distinct; the attribution is off by one section, and nothing silently passes. Unchanged: differential 85/0/0/4 known-diverge/16 deliberate-lead, per-line census zero delta over 105 rows, meta 28/0, contract 88/0/0/0, test-hooks 101/0. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Rigp3VhvZdwJVUZKLLmfq2

…ires naming the row D3, adopted from syrupy -- the only surveyed tool that gates "add a new expectation" and "touch an existing expectation" separately. It maps onto B105's own Trigger text: the dangerous operation is touching a record, because that is where a behaviour change can be laundered into a green. `--bless-divergences [row...]`. A row with no record is written unconditionally. A row whose record already matches is a no-op. A row whose record has MOVED is refused unless the invocation names it. Deliberately not taken, noted in the code so the second path reads as a purchase: the cheaper `[row...]` where omitting names means "bless everything" -- the rubber-stamp shape. Pure per-row was declined too: one hook edit legitimately changes many rows identically, and N invocations add friction without scrutiny. C3/AC13 -- THE NAME CHECK RUNS BEFORE THE CORPUS. The failure mode is not a confusing message; it is that an unmatched name falls through to the unconditional new-record branch, defeating the guard while the output reads as a successful bless. Validating afterwards cannot undo the writes. Names are checked against the record file's own census, because naming a row authorises touching an EXISTING record; a name with no record is meaningless by construction. A second check the plan did not name: a row can pass the up-front test and still be wrong, because this run never discovered it as a lead (retired or renamed). Reported after the corpus rather than left as a silent no-op. Demonstrations: AC13 'marker/mk_dirr' -> rc=2, names the bad argument, record file byte-identical AC13b 'marker/mk_dir ' -> rc=2 on the stray space an exact-string match would wave through AC6a changed impl, no row named -> 16 REFUSED, 0 written, file byte-identical AC6b changed impl, one row named -> 1 REBLESS + 15 REFUSED, rc=1; a per-row hash dump shows exactly one newimpl moved extra naming a row that has a record but was not discovered -> reported, rc=1 extra --check-divergences with row arguments -> rc=2 Also fixed on the way: the argument-rejection message was passed as printf's FORMAT string and begins with `--`, so bash read it as an option and printed its own usage instead of the error. An error path, which is why only running it found it. Unchanged: differential 85/0/0/4/16 with a zero-delta 105-row census, --check 16 matched 0 failed, bless idempotent byte-for-byte, meta 28/0, contract 88/0/0/0, test-hooks 101/0. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Rigp3VhvZdwJVUZKLLmfq2

…g to one T4's first run refuted the shape T1-T3 built. Under a verified one-line copy of B83's `%%` fix (context-floor.sh:1038), the twelve affected rows fail their HAND-WRITTEN NEW literal first, so diverge_case enters its failure branch and never reaches the record: 0 DRIFT, 4 STALE, and no re-bless repairs them. AC4 was unsatisfiable, and the built shape was the pure sidecar D1 names and rejects -- "leaves B83 exactly as unaffordable as today, so this PR would ship a mechanism that cannot pay for the case that justified it". Reinterpreting a locked decision is not an unattended call; the human approved the rebuild at the point of discovery. D1's hybrid, as built now: NEW's expectation comes from the record when the row has one, from the corpus literal when it does not. OLD's side is always hand-written -- it is content-addressed by a pinned blob and the rig FATALs if that blob moves, so it stays the independent anchor. Two defects the rebuild exposed, both measured: record_has_row answered "does this row have a record" with get_section, conflating ABSENT with UNREADABLE. Harmless in a sidecar; under the hybrid it is a SILENT PASS, because an unreadable record falls back to the literal and can go green. Existence and readability are two calls now. The hybrid makes the plain suite run depend on the record, colliding with D4a ("a result merged into the suite's own pass/fail cannot tell the port broke from a record went stale"). Split by question instead of by file: the plain run compares BEHAVIOUR only; identity drift is enforced by --check-divergences alone. Measured -- an inert implementation change leaves the plain run at 85/0/0/4/16 rc=0 while --check reports 16 STALE rc=1. T4 demonstration, run not argued: 1 --check under the patched hook -> 12 DRIFT, 16 STALE, rc=1 2 ONE --bless-divergences, names generated from the census -> 16 REBLESS, 0 REFUSED 3 --check under the patched hook -> 16 matched, 0 failed (the 45 remaining reds are diff_case conversions -- B83's own work, out of scope by D5) 4 revert the hook -> the re-blessed file reddens again, 12 DRIFT / 16 STALE 4b the COMMITTED record file was never touched, so the repo checks green: rc=0, 16 matched Meta coverage for the new branch, because nothing reached it: meta09j (a recorded row uses the record, LEAD despite a literal deliberately set to fail) and meta09k (a record that no longer matches -> FAIL naming the record as the source). Mutants m26 `28 passed, 2 failed` and m27 `29 passed, 1 failed` -- two rather than one, because "which source is consulted" and "is it compared" are separate guards. The ids I first picked were already in use; the suite still reported 30 passed, which is what duplicate ids do -- they make a failure unattributable rather than colliding. The record file's header described the old semantics; corrected by delete-and-re-bless rather than a hand edit, verified header-only (all 64 non-header sections byte-identical). meta 30/0 · differential 85/0/0/4/16 with a zero-delta 105-row census · --check 16 matched 0 failed · contract 88/0/0/0 · test-hooks 101/0 · tasklist-view ALL PASS · validate OK · invariants 8/8 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Rigp3VhvZdwJVUZKLLmfq2

…on that it actually ran D4a: a fifth application of the `|| rc=$?` idiom already used four times in this step -- not a new step and not a new gate. Its own exit code and its own `::error::` per drifted row, because a result folded into the differential's own pass/fail cannot tell "the port broke" from "a record went stale" (the B96 shape this file paid for once). The plain differential asks the behaviour question; this one asks identity and completeness. THE RAN-IT ASSERTION reads the check's own stdout for `checked N records`, not whether a variable got assigned. An rc that merely holds a value proves nothing: a subshell early-exit leaves it looking set while the comparison never executed -- the bare-`tee` masking shape one layer over, and why `gofmt -l` in CI can be falsely green for years. GitHub also counts a SKIPPED required check as passing, so "the step reported no failure" is not evidence the step did anything. Demonstrated against the SHIPPED BYTES: the block is extracted verbatim from this file (dedented, nothing retyped) and run in a temp tree where the harness path holds a stub, so the tested text and the shipped text are the same. AC14 stub exits 0 immediately, block still assigns prc=0 -> rc=1, the assertion names it control1 stub prints the line and exits 0 -> rc=0, ZERO errors control2 stub prints the line, emits one RECFAIL, exits 1 -> rc=1, `::error::divergence record: marker/mk_dir` -- the row is named, not a bare non-zero real the same extracted block against the real harness -> rc=0, checked 16 records control1 is the load-bearing one: without it the AC14 result is equally consistent with an assertion that is simply always red. AC8 verified by reading the changed file: `grep -n bless .github/workflows/validate.yml` returns nothing. CI has no path to the mutate half, by construction rather than by convention. The guard forces prc=90 only when prc was 0, so a check that both failed AND printed nothing keeps its real exit code instead of having it overwritten by the guard's. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Rigp3VhvZdwJVUZKLLmfq2

…our new ones B104's instance half is closed: this harness now has three self-destructing label mechanisms (`known`'s reality check, `diverge_case`'s agreement check, and the divergence records), demonstrated red-then-green. Its repo-wide class is RE-FILED as B135 rather than closed with it. Closing both on the strength of one call site would be the exact "claim wider than mechanism" pattern that entry's own table is a record of, and B135's Trigger therefore requires the review-time rule to exist as a durable artifact a reviewer consults -- not a sentence in a backlog entry. Corrections landed rather than remembered: B83 57 of 105 rows, not 54 of 101 -- and the SHAPE was wrong: 12 of them are already diverge_case rows needing re-blessing, only 45 are conversion. Its "no cheaper shape available" blocker is gone. Both superseded corrections kept verbatim above the original, because a corrected number that erases its predecessor teaches nobody how far an estimate can drift, and this one drifted twice in two days. B105 closed as BUILT, with three of its own filed fields corrected: `observation digest` demoted to a ride-along (all 8 surveyed tools store full text; a digest says a pin moved but not into what), `fixture name + hash` dropped (a "fixture" here is a shell function -- nothing to hash), and the identity fields per-row rather than per-file. New filings, ids from scripts/backlog-toc.sh (129 entries, next free B139), never eyeballed: B135 the review-time rule has no durable artifact -- split from B104 B136 habituation: every record is tagged to one implementation hash, so an inert hook change reddens all of them (measured: 16 STALE / 0 DRIFT). D1's accepted risk, filed not remembered B137 tasklist-first's nonce is repo-global while its seen-marker is per-session, so any party's gate run re-arms every other session. Blocked FIVE times in this one phase against its own "exactly one block per phase" contract B138 a bad length prefix desyncs the record walk, so the corrupted row reports DRIFT and its NEIGHBOUR reports UNREADABLE -- diagnosis quality, not a correctness hole Spec §5.6 records three corrections MEASURED during execution: D1 was implemented as the sidecar D1 rejects and rebuilt as the hybrid (E1), AC3's branch attribution is off by one section (E2), and AC4's "all 12" is exact but not exhaustive under two verdicts (E3). Retired wording kept verbatim. ROADMAP: batch-map PR 5 flipped, recorded as NARROWED at the intent-gate, and B83's 45-row conversion split out as PR 10. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Rigp3VhvZdwJVUZKLLmfq2

…the pipe defect the fix reproduced Cross-family code review (codex, one lens: the expectation-source seam) returned two findings. Both were real. Applying the first reproduced a third. FINDING 2 (Important, fixed). `record_has_row` was `record_present`, which locates a header by SEARCHING. That predicate's own comment bounded it -- "a look-alike payload can only mis-attribute the DIAGNOSIS between two failing branches; it cannot turn either into a pass" -- and the comment was TRUE when written. T4 then promoted the same predicate to source selection without revisiting the bound, and in that role a payload line reading `observation <row>:<n>:` makes a row with NO record take the record branch. This harness's own observations embed framed sections, so that payload shape is legitimate. Fixed by making the predicate section-aware; `record_present` stays diagnosis-only and its comment now says the bound is conditional on that. New meta row meta09l, mutant m28 (`30 passed, 1 failed`). This is the class B135 -- filed in this same PR -- is about, committed inside the PR that files it. THE FIX REPRODUCED B102. First version: `record_row_names | LC_ALL=C grep -qxF "$1"`. `grep -q` exits at the first match, SIGPIPEs the still-producing left side, and `set -o pipefail` reports the whole pipeline as failed -- so a row that HAS a record was told it did not and fell back to its literal. B102's changelog entry describes this verbatim for the close-gate path matcher: "the search stopped at the first hit, the writer died of a broken pipe, and the pipeline's failure status was read as no match." WHAT STAYED GREEN WHILE IT WAS BROKEN, which is the part worth keeping: meta 31/0, differential 85/0/0/4/16, exit 0 -- because the literals the rows fell back to still matched. `--check` went from `16 matched` to `1 matched`, a number nothing asserted. So a COVERAGE ARM was added: the check now fails when the number of rows that actually consulted a record disagrees with the number of records, no waiver, same grounds as the vacuous-run check -- "the checker quietly stopped consulting the artefact it exists to consult" is the vacuous run one level in. Re-introducing the SIGPIPE version now gives rc=1 and `16 record(s) exist but only 1 row(s) actually read one`. Both sibling call sites (`is_named`, the up-front name validation) were rewritten pipe-free too. They do not fail at today's sizes; a latent instance of a defect this repo has shipped twice is not made safe by being small. FINDING 1 (filed Critical, folded into B138 as Important with the mechanism confirmed). `dd bs=1 count=N` returns 0 on a short read, so `get_section` never verifies it consumed the declared bytes. Mechanism confirmed by the reviewer's own probe. The attached claim -- a check can go green against an invalid record -- holds only for a missing FINAL newline, where the compared bytes are correct anyway; a truncated payload compares shorter and reddens. Not fixed here for the reason B138 was deferred in the first place: `get_section` is read by the whole corpus and two suites, and changing a CI-gating oracle's core reader inside a provenance phase mixes two risks. B138 now states both halves as one root cause -- the reader trusts the header instead of verifying against it -- and its Exit requires both demonstrations. meta 31/0 · differential 85/0/0/4/16, zero-delta 105-row census · --check 16 matched 0 failed · bless idempotent byte-for-byte · contract 88/0/0/0 · test-hooks 101/0 · tasklist-view ALL PASS · validate OK · invariants 8/8 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Rigp3VhvZdwJVUZKLLmfq2

Victoriakaey · 2026-07-30T05:01:58Z

Layer 1 — Golden path, real output

The whole point is a mutate/check pair, so here is the pair, on a clean tree.

Capture (adding is unconditional; a re-run is a no-op):

$ /bin/bash skills/project-lifecycle/hooks/test-context-floor-differential.sh --bless-divergences
  BLESS occupancy/bad_first
  BLESS occupancy/bad_after
  BLESS marker/mk_fresh
  BLESS marker/mk_symlink
  BLESS marker/mk_dir
  BLESS marker/mk_grace
  BLESS marker/mk_samesecond (sub-second)
  BLESS pathbytes/pb_sid_tab
  BLESS pathbytes/pb_sid_bs
  BLESS fieldtype/ft_sid_zero
  BLESS fieldtype/ft_sid_false
  BLESS fieldtype/ft_sid_estr
  BLESS fieldtype/ft_sid_earr
  BLESS fieldtype/ft_sid_eobj
  BLESS fieldtype/ft_sid_zero_grace
  BLESS processcwd/pw_cwd_zero
differential: 85 passed, 0 failed, 0 skipped, 4 known-diverge, 16 deliberate-lead

# run it again -> 0 BLESS lines, record file byte-identical (sha256 unchanged)

Check:

$ /bin/bash skills/project-lifecycle/hooks/test-context-floor-differential.sh --check-divergences
differential: 85 passed, 0 failed, 0 skipped, 4 known-diverge, 16 deliberate-lead
checked 16 records
divergence records: 16 matched, 0 failed
$ echo $?
0

What a record looks like — this is the product, so it has to be readable:

generator marker/mk_dir:1:
1
oldblob marker/mk_dir:40:
ef0d7e0bd29a40345e33a4aed885919fcea650da
newimpl marker/mk_dir:40:
20b40056878952c6532eb7a1f7fd247901489fd7
observation marker/mk_dir:577:
exit:1:
2
...
s.corrupt.<PID>.0 <DIRDEF>:<unreadable>
s.marker <FILEDEF>:

<SANDBOX>, <PID>, <FILEDEF> and <DIRDEF> are folded out because they are per-run or
per-umask. Without the last two, contributors would re-bless different bytes on an unchanged hook.

Layer 2 — Negative paths, because these are the product

Every one of these is rc=1 and names what is wrong. None of them is argued.

what was done to the record file	result
payload edited, length prefix corrected	`RECFAIL marker/mk_dir` · `DRIFT` · 15 matched, 1 failed
payload edited, prefix left wrong	`UNREADABLE RECORD` branch · `checked 5 records` of 16 · 13 failed
one row's four sections deleted	`1 discovered lead row(s) have no record: marker/mk_grace` · `checked 15 records`
file emptied	`VACUOUS RUN -- zero records. There is no waiver for this` · `checked 0 records`
a row renamed, count still 16	`checked 16 records` and `1 orphan` + `1 missing`
implementation changed (inert edit)	16 STALE, 0 DRIFT, `0 matched, 16 failed`
`--bless-divergences 'marker/mk_dirr'`	`rc=2`, names the bad argument, file byte-identical
`--bless-divergences 'marker/mk_dir '`	`rc=2` on the stray trailing space
implementation moved, no row named	16 REFUSED, 0 written, file byte-identical
implementation moved, one row named	1 REBLESS + 15 REFUSED; a per-row hash dump shows exactly one `newimpl` moved

The rename row is the one worth pausing on: the count matches and the check still fails. A count
comparison is blind to a copy-paste rename that trades one orphan for one missing.

Layer 3 — Before / after

	before	after
where a deliberate lead's NEW expectation lives	a hand-written literal in the corpus	a captured record; the literal is the fallback for rows without one
cost of a behaviour change touching 57 rows	57 hand-written observations, each of which must be measured not typed	12 re-captured by one command; 45 still need conversion (split to PR 10)
a hand-edited expectation	indistinguishable from a captured one in review	`--check-divergences` reddens, naming the row
an expectation captured from a different implementation	invisible	`STALE`, with both hashes printed
a record for a row that no longer exists	n/a	reported, even when the totals match
CI's ability to tell "the port broke" from "a record is stale"	n/a	two invocations, two exit codes

Layer 4 — Cost

(N/A — no paid API in the shipped path.) Nothing here calls a metered service; the check is local
bash. The one metered thing in this PR's process was the cross-family code review, which ran on
codex (status: succeeded) as one round with one lens.

Layer 5 — Performance

No stated go/no-go target for this phase. Measured on this machine so the next reader has a baseline:

	seconds
plain differential	36
differential + `--check-divergences`	37

+1s for the whole check. Both modes re-run the same corpus; the record comparison is 16 reads of
a 14 KB file. The record file is 14059 bytes for 16 rows.

Known cost, stated rather than hidden: record_has_row walks the census once per row, so the
membership test is O(rows x records). At 16 that is invisible; at B83's 45-row PR it is worth a
cache, and the walk is dd bs=1, which is not fast.

Layer 6 — Findings

S1 (blocks merge): none open.

S2 (shipped with a follow-up filed):

B136 — every record is tagged to one implementation hash, so an inert hook change reddens all of
them (measured: 16 STALE / 0 DRIFT). Fine at 16 rows; at 45+ it converges on the rubber-stamp risk
the design chose this shape to avoid. This is D1's accepted risk, filed rather than remembered.
B138 — the framed reader never verifies it consumed what its header declared. Two halves, one
root cause: a bad prefix desyncs the walk (so a corrupted row's neighbour takes the blame), and
dd bs=1 count=N returns 0 on a short read. Deferred deliberately: get_section is read by the
whole corpus and two suites.

S3 (captured):

B135 — the review-time rule split out of B104: a label is a guard only if a run can contradict
it now has mechanical enforcement in one harness and still no artifact a reviewer is handed.
B137 — tasklist-first's nonce is repo-global while its seen-marker is per-session. It blocked
five times in this one phase against its own "exactly one block per phase" contract.

Corrections landed rather than filed: B83's count (57 of 105, not 54 of 101) and its shape (12
re-bless + 45 convert, not 54 convert); three of B105's own filed fields.

Review round 1 — cross-family (codex), one lens: the expectation-source seam. 2 findings, both
real, both acted on.

Important, fixed — the source-selection predicate was grep-based, so a payload look-alike could
make a recordless row take the record branch. The interesting part is the provenance of the bug:
that predicate's own comment bounded it correctly, and the next change moved it out of those
bounds without revisiting the comment. That is the class B135 — filed in this same PR — is about.
Critical as filed, downgraded on inspection, folded into B138 — see S2.

And then applying the fix reproduced B102. record_row_names | grep -qxF : grep -q leaves at
the first match, SIGPIPEs the producer, pipefail reports the pipeline as failed, and a row that HAS
a record was told it did not. --check fell from 16 matched to 1 matched. Everything stayed
green — meta 31/0, differential 85/0/0/4/16, exit 0 — because the literals the rows fell back to
still matched. So the check now carries a coverage arm: it fails when the number of rows that
actually consulted a record disagrees with the number of records. Re-introducing the bug now gives
16 record(s) exist but only 1 row(s) actually read one, rc=1.

Layer 7 — Close gate

── close-gate phase b105 (gitignored-docs mode) ──
check-set (phase): running 1(product-tree) 2(CHANGELOG) 3(spec) 4(plan) 5(journal) 6(ROADMAP) 7(test-evidence) 8(invariants:8) 9(rationale)
check-set (phase): skipped none — nothing skipped — phase close runs every declared check
check-set (phase): product_paths declared [skills/ commands/ scripts/ .claude-plugin/] · touched by origin/main...HEAD [skills/]
✓ phase diff touches the product tree (origin/main...HEAD): skills/
✓ CHANGELOG.md touched in range
✓ spec doc exists (docs/superpowers/specs/*-phase-b105-*)
✓ plan doc exists (docs/superpowers/plans/*-phase-b105-*)
✓ journal fragment newer than the branch point, by mtime (2026-07-29-phase-b105-divergence-provenance.md)
✓ journal FACT fields present (Decision/Why/Backing)
✓ journal 'Plan deviations' header present
✓ ROADMAP.md names phase b105 and was updated this phase
✓ test-evidence non-empty; validate.py's declared inputs are unchanged since this file was generated (.claude/.last-test-run, set=phase) — NOT proof anything ran, and the recorded set is not checked against how the gate is being run now; hook + test suites are proven in CI, not here
✓ invariant 'no-tracked-docs' holds
✓ invariant 'manifest-version-sync' holds
✓ invariant 'no-icloud-conflict-copies' holds
✓ invariant 'context-floor-envelope' holds
✓ invariant 'no-hook-state-tracked' holds
✓ invariant 'ledger-contract-sync' holds
✓ invariant 'gate-jq-only-envelope' holds
✓ invariant 'named-interpreter' holds
✓ validator_input_paths_rationale present and shaped like a measurement (two numbers + a tracked-surface argument) — NOT a check that the measurement is current or true
✓ context-floor hook wired (settings reference context-floor.sh) and jq was resolvable when this gate ran — not proof the hook fires where your client spawns it
──
✓ close-gate PASS (phase b105)

Layer 8 — Reviewer asks

test-context-floor-differential.sh — line_in / record_has_row. The membership test is
now pipe-free and O(rows x records) with a dd bs=1 walk inside it. Is caching the census once
per run worth doing now, or is that a B83-PR concern?
test-context-floor-differential.sh — the coverage arm in check_records_complete. It
compares RECSEEN against the record count. Is there a legitimate state where a discovered lead
row should skip its record and the arm would false-positive?
test-context-floor-differential.sh — diverge_case, the bless-before-verdict ordering.
Blessing repairs a row and the verdict then reads the repaired record. Is there a mode interaction
where that ordering reports a green for a row that was red on entry and should have stayed red?
.github/workflows/validate.yml — [ "$prc" -eq 0 ] && prc=90. Under bash -e, this is the
last command of the if body when prc is non-zero, so the && yields 1. It has not killed the
step in testing because it is not the last command in the block — is that too subtle to leave?
.claude/divergence-records.txt header. It is prose inside a framed section. Does it say
enough for someone who arrives at a red --check with no context, or does the operator recipe
need to be in the header too rather than only in the harness source?

Layer 9 — What's next

LOCKED, on the batch map:

PR 6 — CHANGELOG surface (B114, B119). The next station.
PR 10 — B83's full rebaseline: the two-character fix plus the 45 rows that genuinely need
conversion. Split out of this PR at the intent-gate so B105's own red/green evidence would not be
buried under 45 rows of B83 noise.

Not scheduled: B135, B136, B137, B138 — all filed with Triggers, none of them blocking.

Victoriakaey · 2026-07-30T05:06:04Z

CI evidence — and the first cross-platform verification of the records

validate pass, 2m57s, all steps green. Run:
https://github.com/Victoriakaey/project-life-cycle/actions/runs/30515475133

The interesting part is not that it passed. The records were blessed on macOS and verified on
Linux, under a different stat formatter family, and nothing in the design guaranteed that would
work — it is exactly what the <FILEDEF> / <DIRDEF> folding was added for, and this is the first
run that could falsify it.

=== context-floor differential: OLD(blob ef0d7e0bd29a) vs NEW(worktree) ===
    fractional stat: gnu
    mode/type stat: gnu
differential: 85 passed, 0 failed, 0 skipped, 4 known-diverge, 16 deliberate-lead

meta-suite: 31 passed, 0 failed
contract: 88 passed, 0 failed, 0 xfail (expected), 0 xpass (UNEXPECTED)

checked 16 records
divergence records: 16 matched, 0 failed
divergence-record check exit code: 0

Locally this machine reports fractional stat: bsd and mode/type stat: bsd, with a umask of 022.
The mode strings differ between the two hosts before folding (Regular File vs regular file,
Directory vs directory) and the sandbox paths differ entirely; after folding, all 16 records
matched byte for byte with no re-blessing. The checked 16 records line is also the first real
confirmation that the ran-it assertion sees what it is meant to see in the actual CI environment
rather than only against the extracted-block stub.

The paragraph that used to end this comment claimed the no-formatter path was "handled in code
but reasoned, not measured". It has been measured since, and it was BROKEN — see the follow-up
commit c4a20a6. On a host where frac_row SKIPs marker/mk_samesecond, the run discovered 15 rows
against 16 records and the completeness arm called the sixteenth an orphan: --check-divergences
failed on that entire class of host for no reason but a legitimate skip. AC5 had said to exclude
skipped rows from both sides; only the discovered side was excluded.

Measured with a one-line mutant forcing FRACTIONAL_STAT="", before and after:

before   84 passed, 0 failed, 2 skipped, 4 known-diverge, 15 deliberate-lead
         RECFAIL <orphan records> ... marker/mk_samesecond (sub-second)      rc=1

after    1 record(s) set aside -- this host cannot run their rows
                 marker/mk_samesecond (sub-second)
         checked 15 records
         divergence records: 15 matched, 0 failed                            rc=0

this host, unchanged: checked 16 records / 16 matched, 0 failed              rc=0

Still not covered by a suite, and this time the sentence is a limitation rather than a defect in
disguise: no host available to this project lacks a fractional formatter, so the guard is a
documented mutant rather than an assertion. A FRACTIONAL_STAT test double is the durable answer —
the same shape the meta-suite already uses to drive the probe.

… red on any host without a fractional stat formatter AC5 says to exclude skipped rows from BOTH sides. Only the discovered side was excluded, so on a host where `frac_row` SKIPs `marker/mk_samesecond` the run finds 15 rows against 16 records and the completeness arm calls the sixteenth an orphan. `--check-divergences` therefore FAILED on that entire class of host for no reason but a legitimate skip. Latent here: this machine probes `bsd`, CI probes `gnu`, and both have the formatter. It would have surfaced as an unexplained red on a contributor's laptop, attributed to their change. Found by refusing to ship a sentence. The CI-evidence comment on the PR said the path was "handled in code but reasoned, not measured" -- accurate, and not the bar, because the sentence would have described a defect as a limitation. Measured with a one-line mutant forcing FRACTIONAL_STAT="": before 84 passed, 0 failed, 2 skipped, 4 known-diverge, 15 deliberate-lead RECFAIL <orphan records> ... marker/mk_samesecond (sub-second) rc=1 after 1 record(s) set aside -- this host cannot run their rows checked 15 records / 15 matched, 0 failed rc=0 this host, unchanged checked 16 records / 16 matched rc=0 Two things the fix had to get right, both measured rather than assumed: The record is keyed by the name `diverge_case` RECEIVES -- with the `(sub-second)` suffix -- while the skip line prints the bare `$1`. Recording the bare name would have matched nothing and the exclusion would silently have done nothing, which is the same shape as the defect it fixes. The set-aside message counts RECORDS, not skipped rows. The first version said "2 records set aside" when only 1 record existed: most skipped rows are diff_case rows that never had a record. Accepted gap, stated: no suite covers the no-formatter path. Every host available to this project has a fractional formatter, so a faithful fixture cannot be built here; the guard is a documented mutant. The durable answer is a FRACTIONAL_STAT test double, which is the shape the meta-suite already uses to drive the probe. Also logged in the journal's Plan deviations: the milestone-close retention drain is SKIPPED, because B132 makes the branch un-pushable while the PR is still open to review changes and B114 forbids draining the CHANGELOG at all. meta 31/0 · differential 85/0/0/4/16, zero-delta 105-row census · contract 88/0/0/0 · test-hooks 101/0 · validate OK · invariants 8/8 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Rigp3VhvZdwJVUZKLLmfq2

Victoriakaey · 2026-07-30T05:25:23Z

CI went red once, on a row this PR does not touch — same-SHA re-run is green

Recording it rather than quietly re-running, because "it passed the second time" is exactly the
sentence that turns a load-sensitive guard into a permanently-ignored one.

The red, run 30516182278, commit c4a20a6:

FAIL   B89-AC5b 8 concurrent batches over a corrupt marker grant AT MOST ONE allow (1 batches did not; worst=2)
contract: 87 passed, 1 failed, 0 xfail (expected), 0 xpass (UNEXPECTED)
##[error]context-floor suites failed (contract=1 differential=0 meta=0 provenance=0)

Note the attribution line: differential=0 meta=0 provenance=0. Everything this PR adds passed
in the same run — checked 16 records, 16 matched, 0 failed, meta-suite: 31 passed. The failure
is in the contract suite, which does not source the differential harness and was not touched by this
branch's diff.

Same SHA, re-run, no tree change: pass (3m0s). So the row is load-flaky on a shared runner.

Filed as B139, and the reason it earns an entry rather than a shrug: phase b89-b86 measured
this exact fix at 0/660 on an idle developer machine and used that number as its acceptance
evidence. Its own journal already said "an idle green on a concurrency row means nothing". This is
the other half of that sentence arriving from a loaded host. 0/660 idle and one red in ~2 CI runs
are not in contradiction — they are the same row measured at two load levels, and only one of those
levels resembles where the guard has to hold.

B139's Exit explicitly forbids fixing it with a retry: a retry converts a measurable flake into an
unmeasurable one, and the number the entry exists to distrust is a green obtained without load.

Victoriakaey and others added 7 commits July 29, 2026 20:01

Victoriakaey added the feature New user-visible capability label Jul 30, 2026

Victoriakaey merged commit be70e2b into main Jul 30, 2026
1 of 2 checks passed

Victoriakaey deleted the feat/phase-b105-divergence-provenance branch July 30, 2026 05:56

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(harness): capture and check deliberate divergences instead of hand-writing them - #151

feat(harness): capture and check deliberate divergences instead of hand-writing them#151
Victoriakaey merged 8 commits into
mainfrom
feat/phase-b105-divergence-provenance

Victoriakaey commented Jul 30, 2026

Uh oh!

Victoriakaey commented Jul 30, 2026

Uh oh!

Victoriakaey commented Jul 30, 2026 •

edited

Loading

Uh oh!

Victoriakaey commented Jul 30, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

Victoriakaey commented Jul 30, 2026

TL;DR

1. What was done

Use cases

Files

2. Why this approach

3. Requirements satisfied

Uh oh!

Victoriakaey commented Jul 30, 2026

Layer 1 — Golden path, real output

Layer 2 — Negative paths, because these are the product

Layer 3 — Before / after

Layer 4 — Cost

Layer 5 — Performance

Layer 6 — Findings

Layer 7 — Close gate

Layer 8 — Reviewer asks

Layer 9 — What's next

Uh oh!

Victoriakaey commented Jul 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

CI evidence — and the first cross-platform verification of the records

Uh oh!

Victoriakaey commented Jul 30, 2026

CI went red once, on a row this PR does not touch — same-SHA re-run is green

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Victoriakaey commented Jul 30, 2026 •

edited

Loading