Worked-example case records for the rules in
metacognitive-monitoring.md, moved here verbatim to keep them out of the
auto-loaded CLAUDE.md context.
Each heading names the rule the record supports.
(2026-07-30, auditing Claude token provenance: the user offered "this account was out of tokens since Sunday I think?" and an entire repo classification was built on it -- any review succeeding after that Sunday must belong to the other account. The hedge went unread. The user then posted their usage chart, showing continuous usage across the whole week and peaking the day after the supposed cutoff, which refuted the premise and voided the classification. The chart was one screenshot away the entire time.)
(2026-08-01, a six-PR session in this repo: four findings were read,
dispositioned, and reported to the user as established fact, each without the
one cheap check that would have confirmed it, and the user re-sent the same
link three times before one was run.
A fifth surfaced while this entry was being drafted, so five were checked in
the end.
Across those five the conclusion held in five of five and the particulars were
wrong in five of five -- a guard whose failure was broader than reported, a
script that was not the one running, a cited line number with nothing at it, a
hardcoded-value scope that named 2 of 6 sites, and
a two-command failure whose first command failed with an error the finding
never mentioned.
The shape then recurred while this entry was being written, and took two
further passes to settle.
The observation recorded in
review-verdict-pitfalls's fifth case was
handed over with a completion
time of 04:08:13Z, corrected here to 04:50:41Z, and then retracted in that
file as an invented particular, on the grounds that #1008 carried no
Copilot-attributable check run at all.
Re-measured 2026-08-03, the retraction is the step that was wrong: check run
91327863807 on that head is named copilot-pull-request-reviewer and reads
completed_at: 2026-08-01T04:50:41Z, conclusion: success.
So the first correction held, and so did the conclusion drawn from it, a green
reviewer check with no review behind it.
It was the verification of the verification that carried a false particular
forward.
That does not rescue the pattern so much as relocate it.
The retraction came from querying a surface that omits this check run ---
fully-clean's criterion 1 now names which surface, and
records that the reason for the omission is still unestablished --- and
nothing about the zero it returned announced that it was answering a narrower
question than the one asked.
A verification inherits the scope of whatever instrument it reaches for, so
"I checked and it is not there" stays a claim about the instrument until the
instrument's own coverage has been established.)
(2026-08-05/06, a ucdavis/bcs session driving several parallel agents: two
subagent particulars were relayed to the user as established fact without the
one query that settles each.
A subagent reported claude-review "failing repo-wide in Morrison-Lab/ai-config
with API Error: Usage credits required for 1M context", and that was published
to the user as a boxed FLAG.
gh run list -R Morrison-Lab/ai-config --workflow claude-review.yml --limit 60 --json conclusion,status returns 36 success, 21 cancelled, 3 in flight, and 0
failures, so the workflow was never failing at all.
The report's true neighbour is what made it survive: a review workflow was
failing repo-wide across the same window, but it was Antigravity Code Review,
4 failures, and its log gives a Google AI Studio 429 reading "Your project has
exceeded its monthly spending cap" -- a different workflow, a different vendor,
and a different error from the one reported.
So a spot check confirming that "a reviewer is down" would have confirmed the
wrong claim.
The second was flatter: an array job was described as "a 500-task array" across
several messages, inherited from an earlier framing.
data-raw/msm-vs-truth.sbatch reads --array=1-100%3.
Provenance of the two sides.
The Don't side came from the session itself rather than from a directive:
no user correction was issued, and the first claim was caught only when a later
subagent contradicted the first, forcing a retraction.
The Do side -- run the deriving query before relaying, and name it -- is
inferred, by carrying the reviewer-finding rule above onto the commissioned-report
case.
A search at the time found the general rule cited but never written:
memories/preferences.md appeals to "the standing verify agent reports with unfakeable asks rule", and unfakeable occurs nowhere else in the corpus, so
only that rule's commit-SHA instance had ever been recorded.)
(Lacaedemon/sparta #1281, 2026-08-16: an exploration subagent's report opened
a session with "_default_loadout() --- scripts/Battle.gd:829-836 (four
entries: Spearmen, Infantry, Archers, Cavalry)".
Two independent particulars, one sentence.
The entry count was checked directly and found to be five, with two
byte-identical Cavalry entries, a real derivation returning a real answer.
The docstring claim was never checked, and "five entries, not the four types its
docstring describes" was published in that PR's body, inventing a contradiction
between the code and a comment nobody had read.
scripts/Battle.gd:781 opens "The default battle loadout: spearmen, infantry,
archers, cavalry, cavalry", and git log -S dates that wording to #478, so the
docstring had never been wrong.
Both halves were one sed away, which rules out the reachable-half reading,
and they were separate claims rather than one claim restated, which rules out
the true-neighbour reading.
What carried the error was that they shared a sentence.
The review then confirmed the verified half in as many words, "the 'two
byte-identical Cavalry entries' claim is exactly what's in
Battle._default_loadout() today", which is true and silent on the docstring.
The clean verdict consequently read as corroborating the whole sentence.
Caught only when a later UMS pass re-derived the docstring claim while writing
the artifact-level entry, and reported the brief's own premise as false.)
(#1299, 2026-08-08: a timing relationship measured on that PR was read as showing that a verdict comment's timestamp can postdate commits the review never saw. The conclusion was stated to the user in chat as a flat finding, and put into a subagent brief minutes later with an explicit instruction to verify it, saying the ordering was the whole point.
The subagent verified it and the field was wrong.
created_at is 18:08:08Z and predates both commits, a60d967f at
18:10:48Z and d426bf83 at 18:11:44Z, so it is a sound anchor.
updated_at is 18:26:27Z, and GraphQL reports the comment isMinimized: true
with minimizedReason: "outdated" -- a later run collapsed it, which is what
moved that field.
So the claim is false of the field anyone would anchor on and true only of one
nobody does.
gh api repos/Morrison-Lab/ai-config/issues/comments/5227428537 \
--jq '{created_at, updated_at}'
gh api repos/Morrison-Lab/ai-config/pulls/1299/commits \
--jq '.[] | {sha: .sha[0:8], date: .commit.committer.date}'The brief's own hedge is what caught it, so this is also a case of
challenge-the-assignment's authoring-side rule
working rather than being skipped -- and it is the reason the asymmetry was
visible at all.
The delegated copy carried a detector; the copy the user acted on did not.)
(2026-08-02, this repo: a boxed RECOMMENDATION advised merging #1058 and
#1064 "whenever you like", calling them independent and both carrying clean
verdicts.
Both had already merged, #1064 at 2026-08-03T02:34:40Z and #1058 at
2026-08-03T02:34:49Z, roughly four minutes earlier, by a second account
rather than by that session.
gh pr view 1058 --json state returns MERGED and settles it in one call.
Because the recommendation carried no status word,
hooks/no-stale-pr-status.py could not fire on it either: its ASSERT list
is entirely state vocabulary and holds no imperative form, and its staleness
condition is anchored to a push of ours, which a third party's merge does
not produce.
Both halves are tracked in ai-config#1072.)
(2026-08-07, a Lacaedemon/sparta session: a Godot binary path failed, and the
session told the user that its memory note on that path was stale.
The note --- reference-godot-binary-path-windows.md, under this machine's
~/.claude/projects/C--Users-dougm-Documents-Github-sparta/memory/ --- was
correct.
It gives the Downloads path that the failure had just shown to be the right
one, and it documents the very trap the session had fallen into one command
earlier: that ls on the folder path "succeeds" and prints the two exe names
while running the binary fails.
It had never been opened.
The path had been inferred from where the repo lives, and that note's
MEMORY.md index line names the folder-shaped exe name, the _console
variant, and the C:/ form, but not the directory the note is about, so the
index alone could not have settled it either.
The file was accurate as written and needed no edit; only the claim about it
did.)
(UCD-SERG/lab-manual#452, 2026-08-04: every claim about the cluster was
established empirically --- a loopback ssh -X probe, ldd, capabilities(),
module avail, getent group sudo.
Every claim about the reader's own computer was written from memory, in the
same table, and the Linux row asserted "your desktop session is already an X
server", which is false on the many distributions now defaulting to Wayland.
Review caught it; nothing in the verified half could have.)
(2026-08-01, UCD-SERG/ucd-serg.github.io#89: a review workflow's
pull-requests permission was narrowed to read, justified by the argument
that "the action posts with its own app token, so the workflow token does not
need write".
The argument was wrong, and two Copilot reviews restated it without objection.
One query --- whether a claude-authored comment existed on any earlier PR ---
returned zero across the workflow's entire month of operation, which settled
both the mechanism and the fact that reviews had never once posted.
Nobody ran it until a fourth PR was opened to fix the consequence.)
(Lacaedemon/sparta#1222, merged 2026-08-07 as 320fe3b2: two regiments locked
in melee rotated about each other by 56 degrees over 700 ticks (56.14 headless
Windows, 58.0 Linux), and the rotation was attributed to Unit._press_into().
That function is six lines, and the operative one is
position += to.normalized() * move_speed * MELEE_PRESS_FRACTION * delta,
over var to: Vector2 = point - position, called from its one call site as
_press_into(enemy.position, delta).
That displacement lies along the line joining the two regiments, so it changes
the separation's length and never its bearing --- it cannot rotate the pair
however large it is, and reading those six lines would have said so for
nothing.
The one qualification is the two clampf lines that close the function, which
bound position.x and position.y against field_bounds independently and so
can truncate one component at a field edge; the measured pair was mid-field,
where they never fire.
Instead the candidate was instrumented, credited, published, and refuted in
review, and the corrected attribution puts _press_into at 0.002 degrees of
bearing rotation against SoldierBodies.couple's -59.163 of that run's -59.16
total.
The confirming evidence had been that the two bodies' contributions were
exactly anti-symmetric --- which is what a central pair looks like, and so was
the disproof.
Checked here against origin/main at 320fe3b2, and against the identity
dtheta = cross(r_hat, dr) / |r|, which returns exactly zero for a radial dr
and matches the exact bearing change to five decimals for a tangential one.)
(2026-08-05/06, ucdavis/bcs#587: cluster CPU efficiency was reported to the
user as "~35-40%, each task reserves 24 cores and uses a third", filed into the
issue, and then corrected in a comment to "~87%, the nodes are well utilized".
Both figures came from sinfo's CPU_LOAD, and the instrument was never
questioned in either direction.
A mechanism was also offered for the higher figure --- that co-resident tasks
interleave, one task's serial phase filling another's parallel phase --- which
the partition forbids: SelectTypeParameters = CR_CORE_MEMORY,
TaskPlugin = task/affinity, and OverSubscribe=NO, so cores are exclusive
and pinned.
CPU_LOAD is a value slurmd last pushed rather than a live reading.
Measured on node c2 by polling sinfo -h -n c2 -o %O against that node's own
/proc/loadavg every 5s: it held 21.07 across 49 consecutive samples, 245
seconds, while the live 1-minute load fell monotonically from 24.49 to 1.83,
then stepped to 12.22 and held while the live figure fell to 0.81 at the
last logged sample.
A separate 12-sample run caught the opposite error, 17.86 against a live
24.35-24.45.
That the errors run in both directions is what rules out treating it as a
biased-but-usable gauge, and it is why two samples of it minutes apart produced
contradictory conclusions with neither being a correction of the other.
The near-miss worth recording is the cross-check that would not have helped.
On a read taken moments after the poll stopped,
scontrol show node c2 reports CPULoad=12.22 at the same moment
sinfo -o %O reports 12.22 and /proc/loadavg reports 0.65, so the
obvious second command prints the same cache.
Only a different kind of source --- the file the daemon samples, rather than the
daemon's copy of it --- settled it.
Recorded for that cluster in ucdavis/bcs#592 / #593; the correction to
#587 had to reach both the issue body and its comment thread, because by then
the retracted figure and its retracted replacement were in different places on
the page.)
(2026-08-12, ucdavis/bcs#615: a PHI-count figure was published as a
correction when it was a second measurement.
.github/workflows/check-phi.yml there pins
Morrison-Lab/gha/.github/workflows/check-phi.yml@v2.
git rev-parse v2 in gha is e7291ccd7171e2f0ac8eb730707ca916795e737e,
which is PR #445's own merge commit, while origin/main is
695fbf56cf65d7779123e24782a40d80782386e1.
The two differ in the operator alternation of check-phi/check-phi.py's
_STUDY_ID_RE: v2 has
(?:\s*(?:<<-|<-|!=|==|=|:)\s*|\s+(?:eq|ne)\s+), and main adds
|\s+(?:not\s+)?in\s*\(\s* from gha#454.
Both revisions extracted and run whole-tree (PHI_BASE_REF empty) against
clean worktrees:
| detector | tree | allowlist | findings | files |
|---|---|---|---|---|
v2 |
bcs origin/main d638c05 |
absent | 93 | 19 |
main |
bcs origin/main d638c05 |
absent | 99 | 21 |
v2 |
bcs#615 head 3f529db |
real | 0 | 0 |
main |
bcs#615 head 3f529db |
real | 0 | 0 |
v2 |
bcs#615 head 3f529db |
empty | 92 | 19 |
main |
bcs#615 head 3f529db |
empty | 98 | 21 |
Diffing the two annotation streams shows the whole delta is SAS's membership
form: 6 findings, every one of them on a line matching (?i)\bin\s*\(, at 6
sites across 4 files.
The summary's file count moves 19 to 21 rather than to 23 because 2 of those 4
files were already flagged under v2 --- so "6 sites in 4 files" and "a
2-file delta" are two different quantities, both correct, which is the
labelling hazard algorithmatize-checks warns about arriving inside the
evidence for this one.
Neither figure retires the other.
93 is what that repository's CI reports today, because it pins @v2.
99 is what it will report once v2 slides past gha#454.
The round-4 comment nonetheless said "I earlier told the maintainer that
main carried 93 findings.
The derived figure is 99.
I had not run that measurement when I first stated it, and the number was
wrong."
Both halves are false: an earlier comment on the same PR had derived 93
explicitly, showing git rev-parse v2 first, and 93 remains correct for the
pinned revision.
The aggravating detail is where the qualifier survived.
That same comment's table was correctly captioned "Measured with gha
main's detector", and the Correction paragraph three lines below it dropped
the qualifier.
The honest caption and the misleading claim were in one comment, and the
quotable paragraph was the wrong one.
Retracted in a later comment on the same PR.
That retraction then misattributed the governing rule to fail-fast.md,
corrected in a follow-up once
git grep -n "A correction inherits its instrument" -- shared/ was actually
run --- which is the same read-versus-recall failure one artifact over.)
(Same session: writing a docstring that had to state precisely how a correction behaved across two study arms is what exposed the claim "relative error is identical across arms" as false, because the precision forced a computation that contradicted it. Tabulating the node types in a diagram is what exposed that four of them sat on three different scales. Against that, most of that hour's writing was post-hoc recaps: well organized, tabulated, and incapable of surfacing anything, since everything in them was settled before composition began. The user corrected roughly every three minutes throughout, several times on the same underlying failure, while the polished output continued.)
(2026-07-30, this task's own brief: long, complete, and carrying several false
claims about this corpus that survived precisely because writing it required
justifying nothing.
One --- "model on the two existing Stop hooks" --- would have had to earn its
place under a stripping pass, and one ls settles it.)
(2026-07-30, a ucdavis/bcs session: the five most confidently asserted claims
were all wrong, and each was one command from being settled.
A leaked credential was described as having gone into a public PR, when the
repository is private with three direct accounts.
This corpus was said to ship no hooks, from a grep against a checkout 27
commits behind.
A PR was reported green and conflict-free from a query returning 11 passing and
4 pending, taken before three of that PR's own later pushes.
A changelog count of ten was reported as nine, because the regex matched only
one of two link forms.
And a blocking Stop hook was called the right shape for a new rule, when it
would have suppressed error admissions.
The directives were "cai: use metacognition", "cai: think before you speak;
question yourself", and "cai: question your generative intuitions".)
(2026-08-07/08, this repo: a status report called
Morrison-Lab/ai-config#1278 blocked, on the grounds that it appends to
shared/workflow/fully-clean.md, "already 1304 lines against a 1200-line gate".
No part of that was checked.
scripts/check-memory-file-size.py defaults to --directory memories, so run
plain it prints "No memory file exceeds 1200 lines." and exits 0; issue #1236,
cited as the gate, describes it as "the advisory 1200-line gate" and had to pass
--directory shared/workflow explicitly to produce a number at all.
validate passes on that branch with the file at 1453 lines.
The line count was wrong in both directions too --- 1397 on main, 1453 on the
branch, 1279 when #1236 was filed --- so the figure matched nothing.
A blocker is a claim about a gate's current state, and one command settles it:
run the checker, then read the check's own result on the PR.
Drafting the correction produced the mirror error, which is the more useful
half.
Reading the size gate's test with a truncated grep ... | head -20 returned
only lines from its synthetic-fixture helper, which supported concluding that
the test never touches the real corpus and that issue #1221's "the next addition
will fail validate" was itself false.
It is not false.
scripts/test_check_memory_file_size.py asserts at module level that "this
repo's own memories/ is under the 1200-line default", validate.yml runs it,
and that is why memories/git.md sitting at 1199 lines has headroom of exactly
one.
The first claim came from never running the query; the second from running one
whose scope head had cut off --- the same failure with an instrument in front
of it, per "Illusions of knowing have an exact software form" in
metacognitive-monitoring.md.)
(Morrison-Lab/ai-config#1278, 2026-08-08, cost two rounds.
Round 1's review was reporting a different finding when it noted in passing that
"the Verdict:\s*(?:Clean|Approved|Ready)\b pattern is safe because it requires
immediate adjacency after Verdict:", with the evidence attached: "verified:
classify_verdict("Verdict: Not Ready") correctly returns '', not
'clean'".
Both halves are true.
The measurement is reproducible, and it varies the qualifier on one side only ---
Not precedes the phrase, and adjacency to a label does constrain what precedes
it.
The round-2 comment repeated the conclusion as though it covered trailing
qualifiers too, and a code exemption was written around it, if pat in BARE_CLEAN_PATTERNS, so the labelled pattern skipped the position, negation, and
conditional checks the same rounds had just built.
Round 3 reproduced Verdict: Ready for merge, but not until it addresses the following and Verdict: Ready for merge once the following items are addressed
as clean --- the one path in the function that had been declared safe in
writing, and, as the review put it, the one "surviving through the one code path
that was assumed safe without evidence".
The test suite reproduced the same scope error rather than catching it: the only
case touching that pattern was named "'Verdict: Ready' needs no guard (adjacency
already binds it)" and asserted the bare label with nothing following it.
Note the shape is the two-sided qualifier error from fail-fast.md one level up:
there a guard covered the before-side and missed the after-side, here a
premise did, and the premise then licensed skipping the guard entirely.)
(#1395 / #1407, 2026-08-12: two false mechanism claims
landed in scripts/test_ai_session.py during one PR.
The second is the one recorded above.
A helper needed a reliably dead PID, so it orphaned a child, killed it, and
polled kill -0 until that failed.
The loop did not terminate promptly, and a comment was written to justify the
design around that: PID 1 in this container "does not reap", so a killed orphan
"stays a zombie" permanently, and wait from a non-owning shell is "a no-op".
Both halves are false, and both were decidable by one probe.
Re-measured for this entry, 2026-08-12, uname -sr = Linux 6.18.5-fc-v20:
| probe | result |
|---|---|
ps -o comm= -p 1 |
process_api |
| immediately after the kill | stat=Z, ppid=1, kill -0 returns 0 |
wait <pid> from a shell that never owned it |
pid N is not a child of this shell, rc 127 |
poll kill -0 at 5 ms until it fails |
reaped after 225 polls, 1573 ms |
So PID 1 does reap, at roughly 1.6 to 2.0 seconds, and wait on a non-child
errors rather than doing nothing.
A companion entry, "kill -0 reports an unreaped zombie as alive", is proposed
in #1407 --- once merged, it lives in memories/claude-code.md and owns those
container facts and their volatility caveat.
The methodological point is that the symptom could not have told anyone which
mechanism was operating.
A poll loop that keeps seeing kill -0 succeed is exactly what "PID 1 never
reaps" predicts, and exactly what "PID 1 reaps asynchronously, about two seconds
from now" predicts too.
The true mechanism was the opposite in kind --- reaping happens, and the loop
was losing a millisecond-scale race, re-losing it on each retry because every
retry spawned a fresh PID --- and no amount of re-reading the comment, or
re-running the failing loop, would have separated the two.
The discriminating observation was the same one held longer.
The direction of the error is the part worth carrying: the immediate look supported the stronger claim, permanence, and the cheaper observation was the one that would have refuted it.)
(Morrison-Lab/ai-config#1281, 2026-08-07: a review cancelled with no verdict
was explained by concurrency: cancel-in-progress, correctly.
Asked to check, the session grepped the caller workflow,
.github/workflows/claude-review.yml, found no concurrency block, and
retracted the explanation to the user as something carried over from another
repo's setup without checking.
That caller is 68 lines and delegates to
Morrison-Lab/gha/.github/workflows/claude-code-review.yml@v2, which declares
the group at job level, line 328 of 1091, beneath a 25-line comment describing
this exact race.
The grep was sound and covered 68 lines of a call chain over 1150 lines long,
so it could not have returned a hit whether or not the claim was true.
memories/github-actions.md's "A caller with no concurrency: block can still
have its runs cancelled" had recorded the same fact two days earlier, from
PR #1224, and was not consulted.)
(#1238, 2026-08-07: a reviewer's gh pr view --json comments reported a comment's author_association as COLLABORATOR; this
session's own tool call reported MEMBER for the identical comment id.
Rather than run one more check, the memory being edited was corrected to
state both readings as an unresolved cross-surface disagreement -- which
was itself wrong, and became the review's next finding.
A third check, list_repository_collaborators, resolved it in one call: the
account held a direct collaborator grant, matching COLLABORATOR and
explaining the MEMBER reading as this session's own tool's outlier.
The "unresolved" framing cost a full review round it did not need to.)
Morrison-Lab/ai-config#1583, 2026-08-17.
A claude-review job completed its review and then failed at step 20, "Post
review comment", one second later.
Step 20's own log was not read.
A sibling step running one second earlier printed an env block showing
PR_NUMBER: empty, and that was reported as the likely cause --- a plausible
story, since a posting step with no PR number would indeed fail instantly.
Reading step 20's own log refuted it.
Its env showed PR_NUMBER: 1583, and the actual error was
HTTP 503: No server is currently available to service your request. (https://api.github.com/graphql) from gh pr comment, followed by
##[error]Process completed with exit code 1.
The empty variable belonged to a different step, which had succeeded with
outcome=success;conclusion=success;duration_ms=39.
Two details are what make it a case record rather than one bad guess.
The wrong diagnosis was more specific than the right one would have been at that moment, and specificity is what made it persuasive: it named a variable, a value, and a timestamp, all of them real.
And the two diagnoses made opposite predictions about the remedy.
An empty PR_NUMBER is a configuration defect, so a plain re-run would
reproduce it and the fix would be a dispatch or workflow-input change.
A 503 is transient, so a re-run should simply work.
A single rerun_failed_jobs recovered the run and the verdict posted, which
the 503 diagnosis predicted and the PR_NUMBER diagnosis ruled out.
ucdavis/bcs, 2026-08-19/20.
One session produced the same error five times, across five distinct claims
--- instances 2 and 3 arose within the same investigation, the rest in
unrelated domains --- each instance surviving self-review.
The recurrence is what makes it a case record: per
deterministic-tools's third-occurrence
bar, a third instance is the point at which the shape gets written down rather
than fixed one more time.
The fourth and fifth arrived while this entry was being written, which is
itself evidence about how easily the shape passes self-review.
Instance 1 --- a verified mechanism, an unverified instance.
Roxygen prose claimed that this repo's own symbol tracer beats
codetools::findGlobals(), because "a bare map() relies on the standalone
import, which data-raw/ does".
The mechanism half was measured: code was run confirming that findGlobals()
drops namespace-qualified call heads, so purrr::map() never reaches the
standalone map().
The instance half was not.
The one data-raw/ file carrying bare map() calls has library(purrr) above
them, so it does not rely on the standalone import at all.
A reviewer caught it.
The measurement establishes a fact about findGlobals().
The claim was about the contents of a directory.
Instance 2 --- a freshness check that settled the model, reported as
settling the data.
An analysis artifact was regenerated, and its freshness was verified by
checking that its coefficient terms matched the current model specification ---
age_monthly present, the obsolete age2 and age75 absent.
That check is sound, and it proves the artifact came from the current
model.
The numbers were then reported as "verified".
The check says nothing about which population was fed in, and the
population was the live question.
One measurement, two axes, and only one of them measured.
Instance 3 --- a measured difference, read as a measured direction. Following on from instance 2, two candidate data extracts were compared, and the comparison was run correctly: they are different populations, differing in sites and by roughly 410,000 participants, with neither a subset of the other. From that, the session concluded the analysis had run on the wrong extract and publicly retracted the numbers. The comparison establishes only that the two differ. Which one is current is a separate fact the session did not hold, and the maintainer confirmed the extract actually used was the correct one --- the documentation relied on for the retraction was the stale half.
Instance 3 is the one that fixes the shape as an inference error rather than as optimism. Here the overreach ran toward alarm: it retracted a true result, in public, on the strength of a sound measurement of something else. A rule watching for over-claiming would have passed it, and the act of retracting made it feel more careful than the claim it replaced.
Instance 4 --- a complete enumeration of branches, reported as an
enumeration of refs.
A commit carrying a leaked credential (5da971a1) was squash-merged and its
PR closed.
git branch -r --contains 5da971a1 returned nothing, and that was reported to
the maintainer as the commit being "no longer reachable from any remote ref",
with the exposure closed as far as git could close it.
check-secrets flagged the same six findings on the next PR.
This is the sharpest of the five, because the measurement has no defect at all.
git branch -r --contains correctly enumerates the branches containing a
commit, and it returned the right answer for that question.
The claim was about refs.
Branches are a proper subset of refs, and the excluded subset is the one that
decides this particular question: GitHub retains refs/pull/<N>/head
permanently, and closing a PR or deleting its branch does not remove it.
The default fetch refspec is +refs/heads/*:refs/remotes/origin/*, so
refs/pull/* is never fetched and git branch -r cannot see it whatever the
repository state.
The other instances admit an argument that the measurement was incomplete.
Here it was complete and correct for its own scope, and the scope was silently
widened by one word in the sentence that reported it.
The consequence was operational rather than only epistemic.
The conclusion licensed a recommendation against adding .gitleaksignore
entries, on the reasoning that squashing would clear the finding.
It does not, so the repository carries a permanently red security check until
the allowlist lands --- the opposite of what was advised.
Writing the population into the sentence is the whole fix, and it costs one word. "No branch contains it" and "no ref contains it" differ by one word, and the first is what the command established.
Instance 5 --- an accurate pass and fail count, reported as the suite passing. A default path was changed, the affected test files were run, and "43 pass, 0 fail" was reported as evidence the change was safe. A reviewer then found a test the change had broken.
The count was accurate.
The skip count was never read: 15 tests were skipped, and the skipped set
contained exactly the broken test.
It skipped because skip_if_not_installed("arrow") fired, and arrow was
absent because the verification command carried R_PROFILE_USER=/dev/null ---
a habit picked up as a workaround for an unrelated renv startup failure.
Measured both ways in ucdavis/bcs on 2026-08-20:
R_PROFILE_USER=/dev/null : requireNamespace("arrow") -> FALSE
renv active : requireNamespace("arrow") -> TRUE
Re-running with renv active, after fixing the test, gave 40 pass, 0 fail, 0 skip. The two runs' totals are not comparable --- 43 passed plus 15 skipped is 58, against 40 --- and this record does not establish what changed between them, so the load-bearing figure is the 0 skip rather than either pass count. Reporting them as comparable would be this section's own error committed inside its own case record. A suite in which roughly a quarter of the tests did not execute (15 of 58) has not reported that they pass.
The second lesson is in why the skip happened.
R_PROFILE_USER=/dev/null is a documented workaround in that project, and its
side effect is invisible at the call site.
It changes which packages are available, hence which tests run, while changing
nothing about the number that gets reported.
That generalizes past R.
Any flag that skips environment setup --- a --no-config, a bare interpreter,
a container built without the optional extras --- can shrink what is being
measured without shrinking the figure reported.
The shrunken run is usually the faster one, so the habit is self-reinforcing.
UCD-SERG/serocalculator#668, 2026-09-01.
A fix removed an RNG-kind leak from a simulation function, and part of
reviewing the fix was predicting which existing snapshot tests would change
once it landed --- a real cost, since a changed snapshot needs re-baselining
and a reviewer's time to confirm the new values are still correct.
The reasoning traced, correctly, which branch of RNGseq_seed() a call
takes depending on whether a .Random.seed already exists in the session.
That is the mechanism, and it was verified by reading the function.
The claim built on top of it was that specific existing test calls would hit
the fixed branch and their snapshots would change.
Nobody checked what those calls actually ran under.
They used the package's multi-core default for num_cores, which on the
test runner's platform forks the R process for %dopar% rather than
building a cluster --- so the leak the PR fixed, which the mechanism
analysis correctly traced, never reached those particular calls at all.
The prediction was withdrawn once grep-ing the test file for its
num_cores argument showed the actual value.
The mechanism was verified.
The instance was asserted.
Those are the two different propositions
"A sound measurement does not license the claim standing next to it"
names: a claim about how RNGseq_seed() branches, and a claim about which
recorded test calls take that branch, standing in one paragraph as though
verifying the first settled the second.
The remedy that section gives is the one that closed it here too --- derive
the population (grep the calls' actual num_cores) rather than reasoning
from the mechanism to the instance.
ai-config#3023, 2026-09-03.
A review of scripts/lib/shellcmd.py found that _heredoc_free scanned only the first heredoc opener on a line, so cat <<A > f1 && cat <<B > f2 left the second body as live text and a git commit && git push written inside it was refused.
The fix was easy. The commit message was wrong: it called the defect "a regression of this PR's own opener-line fix", because the previous commit had reworked exactly that function and keeping the opener line's remainder was exactly the kind of change that could plausibly have introduced it. No check was run. The claim arrived as an admission and read as candour, which is what stopped it being read as a claim at all.
The reviewer said it predated the fix.
Settling it cost one command: git show 7b54d28:scripts/lib/shellcmd.py, then exercising that copy on the same input.
The old version returns a blanked opener followed by both git commands as live text --- the same defect.
The routes differ, which is why it was believable: the old code dropped the opener-line remainder and lost the second opener along with it, the new code kept the remainder and never rescanned it.
Same observable failure, two mechanisms, and neither of them the new commit's doing.
Two things generalize. The counterfactual is nearly free whenever the prior version is a commit away, so there is no cost argument for skipping it. And the reviewer's own attribution was equally a cause claim: agreeing with it because a reviewer said so would have produced the right answer by the wrong route, and the same command settles both.
The section written from this case opened with the same error, one level up.
Its first draft began "The two sections above govern a cause read off the wrong artifact and a justification written after the decision", which is a claim about the fragment's own structure --- and it was recalled rather than read.
The section directly above is "A story that fits the evidence is not a finding", whose discriminating-experiment remedy the new section's counterfactual instantiates, so the entry presented itself as a fourth independent rule when it is a specialization of its immediate neighbour.
Settling it cost one grep -n '^## '.
That is the scope claim-type from the top of this fragment, committed inside a section about the cause claim-type, by an author who had just been corrected for an unchecked cause claim. Which is the transferable part: knowing a rule, and having just been burned by it, does not arm the check. Only asking what kind of claim a sentence is arms it, and "the two sections above" does not feel like a claim at all --- it feels like navigation.
Morrison-Lab/ai-config and ucdavis/rampp, 2026-09-02/04.
Five instances, four of them spanning "A sound measurement does not license the claim standing next to it" and verify-the-right-artifact and one falling under neither, and worth recording as a set rather than as five bullets --- not because they share a taxonomy, since they do not, but because what they share is invisible in any one of them.
They do not fall under one rule, and saying so is the first thing this record has to get right.
Instances 1 and 2 are substitutions: each artifact was real and was not the one the claim was about, and both are filed that way by their own sources --- ai-config#3039 says verify-the-right-artifact "names this exact substitution", and keep-checkouts-fresh says the same of instance 2 by name.
Instances 3 and 4 have no substitution in them: the artifact is the right one, read correctly, and the sentence after the reading answers a question it does not address.
Instance 5 is neither, and is the plainest of the five --- a field read wrongly.
What they share is a felt sense, not a mechanism: in each one the erroneous step felt like reading the evidence rather than adding to it, which is why each survived self-review. That is the transferable observation, and it is also the reason the taxonomy question is worth keeping separate from it. Knowing which of the two rules governs an instance is what tells you where to look; knowing that the failure feels like reading is what tells you to look at all.
Instance 1 --- a merged PR's check-run names, read as what the default branch emits.
The names were real and were accurate for that pull request's head.
A ruleset's required_status_checks matches what the default branch emits on every future pull request, which is a different set once a workflow has been renamed.
Wrong contexts were written to a live ruleset, where a string no workflow emits sits as Expected forever and would block every merge in that repository until removed, with nothing red to point at.
That consequence is how #3039 and the guard's docstring both state it --- derived from the matching rule, not from an observed blocked merge.
Recorded as ai-config#3039 and guarded by hooks/no-underived-required-check.py (#3137), whose docstring carries the full account;
the entry here is the inferential shape, not the guard.
Instance 2 --- the newest cache directory, read as the copy that runs.
A real directory carrying a real stale constant, and not the copy the later capture named.
mistake-patterns's Pattern 43 Fix section already rules that proxy out by name, so this is an occurrence of a recorded Fix step being skipped rather than of a new mechanism.
Instance 3 --- a sample whose window excluded its own target.
Twenty distinct hook command lines captured over 240 seconds, all from $HOME/.claude/hooks/ and none from a plugin root, read as showing the plugin path inert.
The sample is sound;
its population is the complement of what it was aimed at, because the hook of interest could not fire during the window.
This is the one metacognitive-monitoring's new sub-case is written from, and the most reusable of the five: a sample drawn when the event of interest is impossible looks identical to a sample that found nothing.
Instance 4 --- a resolved path, extended into a lifetime claim and a remedy.
A ps capture correctly resolved the path of the copy that fired during that capture.
That was extended to "a per-session snapshot, frozen for the life of the session" and to a "restart the session" remedy, neither of which the capture touches --- a capture names a path at an instant and says nothing about how long that path stays selected.
Both were withdrawn.
The remedy half is the sharper one, since "An action you recommend is a claim about state" governs it and a recommendation does not present itself as an assertion.
Instance 5 --- two blank validate: entries, read as passing.
statusCheckRollup reported them with no conclusion because they were in progress.
gh-cli already states the rule this violates --- gating code must require CheckRun.status === 'COMPLETED' and a terminal StatusContext.state, treating anything else as still in progress --- so this instance is listed for the count rather than because anything is missing from the corpus.
A sixth arrived while this record was being written, in the write-up itself, and it is the sharpest of them.
Drafting the over-warn entry this change adds to fail-fast, this session ran gh label list -R Morrison-Lab/ai-config, saw no-ai-review returned, and wrote that ai-config#1709's reopening comment had asserted an absent label that was in fact present --- convicting that comment of the very error this record catalogs.
The label's createdAt is 47 minutes after that comment, and the comment is what caused it to be created, having proposed exactly that.
The gh query was correct, current, and about a different moment than the claim it was used to settle.
So the reading was a substitution along the time axis rather than the artifact axis: the repository now stood in for the repository then, which is a distinction no command in the query names.
The fourth adversarial round on this change caught it.
The transferable step is cheap: when a query is used to judge a dated claim, read the artifact's own timestamp --- createdAt on a label, created_at on a comment --- rather than only its present value.
Tracked as ai-config#3271.
ucdavis/rampp #154, #165, #166, #167, #168, 2026-09-09.
A commit (f6b4475) downgraded roxygen cross-reference links [fn()] to plain code spans `fn()` across 18 R/ files, to silence "no visible Rd target" warnings that @noRd was producing. (The review under scrutiny said "~13 files";
18 is the derived figure, from git show f6b4475 -- 'R/*.R' filtered for removed lines carrying a [fn()] link.
Repeating the review's own uncounted number would have reproduced the defect this entry is about.)
The correct fix was to drop @noRd so the topics gained .Rd pages;
the commit instead deleted the markup the check complained about.
Only one reviewer actually read that diff.
Copilot had been requested, and every Copilot review on the PR from 2026-09-04T17:37Z onward --- f6b4475 landed at 23:26Z --- carried the quota-skip stub body, "Copilot was unable to review this pull request because the user who requested the review has reached their quota limit", which self-review-fallback already names as a non-verdict.
So the PR looked doubly reviewed while one reviewer had read nothing;
counting review-shaped activity is not counting reviews.
The Claude review did read it, did not flag it as a regression, and actively endorsed it:
"Doc-link fixes: complete [...] I checked the remaining
[func()]-style links left inR/and confirmed every one now points at an exported, documented function [...] so this fix is complete rather than partial."
Two distinct failures follow, and they belong to two different reviews. Attributing both to the first review took three adversarial rounds to dislodge, which is the part of this case worth keeping.
The first review's failure is the endorsement, and only the endorsement.
The scope claim in that quoted sentence looks like the classic underived sweep, and the first draft of this entry built its whole argument on it.
Three rounds of adversarial review took that argument apart in stages: first that all three of the checker's findings were evidence, then that one of them was, then that none of them was.
Running the checker at f6b4475 itself settles it --- Scanned 123 R file(s) against 63 documented topic(s). No roxygen cross-reference downgrades found. man/strip_rv.Rd and man/build_failure_survival.Rd are both first created by bd987b9, four days later, so at f6b4475 neither name was a documented topic and neither span was detectable.
f6b4475 never touched R/find_failures_not_enrolled.R either;
4a7bb49d had downgraded that link six hours earlier, before even the base of the diff range the review named.
So the review's "every one" was, as far as any instrument can tell, true of the population that existed when it was made.
That is worth stating plainly because the instinct runs the other way. An underived universal claim is usually wrong, the vocabulary for saying so was ready, and a checker producing three hits looked like the measurement that closed it. The hits were real and were measuring a later tree. Being right that a claim is underived is not the same as having a counterexample, and the second is what an enumeration argument needs.
The scope-claim failure belongs entirely to the second review, where it is well supported.
Six commits later, bd987b9 dropped @noRd, generated the .Rd pages, and restored the links.
Its review asserted "restores the [func()] markdown links" on the basis of an explicitly stated sample --- "Spot-checked a few of the restored pages" --- and missed the leftover in R/find_failures_not_enrolled.R.
At that commit the checker finds three, so a derived sweep was available and a sample was used instead.
A human caught one of the three by hand.
This is the worked instance of "Search for the artifact instead of arguing about whether it would exist"'s "sharpest instance" bullet, and an instance of the existing rule rather than a new one.
The checker also measures a different property from the one either review asserted. Both reviews claimed something about links that exist --- that they resolve, that they were restored. The checker reports code spans that could be links and never inspects an existing link at all. Keeping that straight is what makes the second review's case hold and the first one's collapse.
The genuinely new observation is that the reviewer was not blind to the change -- the review read the diff correctly and endorsed the wrong fix anyway.
dont-incur-technical-debt already carried reviewer-facing bullets, so the gap was narrower than a missing audience: none of them covered a reviewer correctly diagnosing why a change was made (the warning) and then certifying the change that merely stops the symptom from being visible, rather than asking whether the underlying condition (@noRd on a topic with real, working links) was itself the defect.
The review did the hard part --- it understood the mechanism --- and then treated "the warning is gone" as evidence the fix was right, instead of asking what produced the warning and whether removing it was the correct response.
dont-incur-technical-debt.md's "In review" section now carries a bullet for this case: flag a fix that makes a check pass by removing the input the check was flagging, when the diagnosis shows the input was correct and the check's target (the missing .Rd page) was the actual defect.
The domain-specific rule, however, was already written -- and that is the more useful half of this case.
reuse-docs-and-args states the whole thing: use @keywords internal, never @noRd, because @noRd suppresses the .Rd page a [helper()] link needs;
and then, naming the exact remedy the commit chose,
The fix is not to strip the doc-reuse syntax (retyping the
@paramdescriptions, spelling out the...forwards, or downgrading the link to plain code font) [...] Use@keywords internalon the helper and keep the@inheritDotParams/@inheritParams/[helper()]references intact.
That passage is authoring guidance, and the same fragment's separate "In review" section turns it on reviewers too, listing "an internal function marked @noRd instead of @keywords internal" among the findings to flag.
f6b4475 left every @noRd in place and deleted the links instead, so it was flaggable on that bullet as written.
The repo eventually arrived at the fragment's own remedy on its own: bd987b9 dropped @noRd and restored the links, six commits and four days after f6b4475 (2026-09-04T16:26 PDT to 2026-09-08T23:29 PDT).
The rule was not loaded, and finding that out is the actionable half.
The first draft of this entry asserted the reviewing workflow had this corpus installed as a plugin, so that a loaded rule had failed to fire.
The adversarial review checked, and ucdavis/rampp's claude-code-review.yml installs plugin_marketplaces: 'https://github.com/anthropics/claude-code.git' with plugins: 'code-review@claude-code-plugins' --- a different plugin.
git log --all -p over that repo's workflow files returns no match for ai-config at any point in its history, and as of 2026-09-09 the repo's main carried no CLAUDE.md either. (That last clause is already moving: the install gap was filed as #167, and a claude[bot] session opened #168 adding a CLAUDE.md three minutes later, which is why the clause is dated rather than stated flat --- see timestamp-volatile-claims.)
So the review had no access to the rule that named its exact case, and the gap is an install rather than a lapse.
Two things follow, and the second is the general one.
A lab rule only governs the repos that load it, so a repo running an AI review without this corpus installed gets a reviewer with none of the lab's standards --- which looks identical from the outside to one that has them and ignored them. Check what a review workflow actually installs before concluding anything about why it missed something.
And deterministic-tools still decides the deliverable, for a reason the install gap does not touch.
Even installed, a written rule is only as good as the reader noticing that this moment is the one it governs, and a de-linking diff does not announce itself --- it looks like a warning being fixed.
A checker that maps \alias{} to declaring file and reports the population cannot fail to notice, and at bd987b9 it reports all three where a spot-check found none and a human found one.
- Do: enumerate the population a review claim quantifies over (
grep/an instrument over every file, not the files the diff already named) before writing "every one" or "all of them". - Do: ask, once a review has correctly named why a diff exists, whether the diff addresses that cause or only removes the thing that detected it -- a warning silenced and a defect fixed are not the same outcome even when both diffs are one line.
- Do: run the instrument against the commit the claim was made at, not against the branch head, before offering its output as a counterexample.
- Don't: accept a stated sample --- "spot-checked a few" --- as support for a universal claim when the population was derivable at that commit.
- Don't: read a correct mechanism diagnosis in a review as license to skip asking whether the fix under review acts on that mechanism or merely hides its symptom.
- Don't: treat "this claim was underived" as equivalent to "this claim was false" --- the first is about method and is usually right, the second needs an artifact that existed at the time.
- Don't: author a new rule for a case without first searching for an existing one --- and check whether the existing rule was even loaded, since an uninstalled rule wants an install and a bypassed one wants an instrument.
(#3707, commit 1cfcd075, 2026-09-17, addressing a review round against d256d23: a guard function, _agent_subtypes, was widened to read two more dict keys (agent, persona), and the comment introducing the widening gave its reason as a scope claim about a sibling function in the same file: "agent and persona are here because _is_reviewer_record below already treats them as persona-naming keys, and two predicates in one file disagreeing about what names a persona is how a dispatch gets seen by one and not the other."
That sentence states a population --- the keys the sibling predicate reads --- and a target: this function should read the same ones.
Nobody derived either set.
_is_reviewer_record reads ten distinct keys across its two passes: attributionAgent, agent_type, subagent_type, subagentType, TypeName, Role, agent, name and persona in both, plus a lowercase role read only from a nested message dict.
_agent_subtypes before the widening read six of those;
the widening added two more, landing at eight and leaving two uncovered (role, attributionAgent) that the justifying comment's own wording already promised to close.
The fix read as complete because the comment supplied the reason and the diff visibly acted on it --- two keys were added, disagreement was reduced, and a reader checking the diff against the stated reason finds them consistent.
Consistent with the reason is not the same as satisfying it: the reason was a claim about two full sets matching, and "some progress toward matching" confirms a directional claim while leaving a quantified one unchecked.
A later review round derived both sets and found the gap by counting, not by rereading the comment.
The record of that gap was itself under-derived on the first pass.
A companion case describing this incident first named the miss as "two keys a sibling predicate read that this one did not" --- the two the fixture had happened to surface (see fixtures-are-not-evidence.cases.md).
Deriving both key-lists directly (rather than reading the number off the test fixture that had exposed part of it) showed the true gap was four, not two.
A count copied from evidence that was real but partial repeats exactly the error the parity comment made one level up: it reports what a nearby artifact showed, not what the population contains.
The resolution was not "close the gap completely," and that half of the fix is the more useful fact to keep.
role was added, because it is the same key as the already-read Role in a different casing --- reading one and not the other is the identical split-predicate defect one level in, and it is safe on its own terms besides (a message's role can only hold values like user/assistant, which the sibling predicate itself filters).
attributionAgent was deliberately left out, with the comment now saying why: it names who authored a transcript record, while _agent_subtypes reads a tool's input --- a different question, so copying the key across would be structural fit standing in for a transferred purpose (see check-purpose-before-reusing).
Parity with the sibling function's principle ("a name key means the same thing to both predicates") is not parity with its key list: the correct target set was never "every key the sibling reads," and deriving the list without also asking whether each key's purpose transfers would have produced the wrong fix by symmetry.)
- Do: when a fix's own comment states a parity or consistency claim between two predicates, derive both key/case sets and diff them, rather than confirming the diff moves toward the stated reason.
- Do: when the gap between two derived sets was found via a fixture or other partial evidence, re-derive it directly before recording its size --- the fixture's own coverage is a fact about the fixture, not the gap.
- Do: ask separately, for each item the diff would need to add for full parity, whether its purpose transfers to the new site --- a name that recurs in both structures can still mean two different things.
- Don't: read "the diff added items toward the stated reason" as "the diff satisfies the stated reason" --- a directional claim and a quantified one need different checks.
- Don't: treat full parity with a sibling's key list as the correct fix by default; parity is owed to the principle the comment states, and a key whose purpose does not transfer is correctly excluded even when doing so leaves the sets unequal.
(#3707 / #3746, commit 1cfcd075, 2026-09-17: a guard excludes certain tool names from being treated as reviewer dispatches on a flat, call-id-less transcript shape (OpenCode/OMO), since that shape never populates the field the exclusion's ordinary safety net depends on.
The comment defending the exclusion read: "That costs no real capability: a genuine OMO dispatch-then-retrieve returns no verdict on origin/main either."
The sentence's subject --- "a genuine OMO [reviewer report]" --- is not one shape.
A dispatch-then-retrieve (two paired records, a dispatch call and a separate retrieval call under an excluded tool name) is one;
a single flat record under an excluded tool name that both dispatches the reviewer and carries its report in its own paired result is a second, and the comment's claim was checked against only the first.
The second shape does lose authorization under the exclusion where it would not on origin/main, which a new regression case (case 18) pins directly: denied here, admitted on the unpatched branch.
The claim was true of the instance it was written against and false of the instance it never named, and both instances are equally "a genuine OMO [report]" under the sentence's own wording.
This is the "costs nothing" shape of the "Do: scope the claim to what was measured" bullet under "Search for the artifact instead of arguing about whether it would exist": a no-loss claim quantifies over every way the excluded population can arise, so it needs that population enumerated --- every shape the transcript format allows a reviewer dispatch to take --- not one representative shape checked and generalized from. Writing the claim felt like reporting a measurement, because the checked shape genuinely does cost nothing; the population it was silently generalized to was never listed.)
- Do: before writing "this costs nothing" or "this loses no capability," enumerate every shape the excluded input can take, not only the one a test or a mental model already covers.
- Do: add the regression case for the unchecked shape in the same round that names the gap, so the population claim becomes checkable rather than merely narrowed in prose.
- Don't: generalize a no-loss claim from a single checked shape to "a genuine [format]" when the format admits more than one paired-record layout --- name the shape actually measured instead.
- Don't: treat a claim as safe because it is true of the case that prompted writing it; a no-loss claim's failure mode is the case nobody thought to name.
"All three whitespace runs are bounded now" was true of the source; "and the tests pin it" was checked by rereading, not by counting
(Morrison-Lab/ai-config PR #3826, commits 0388cdfa and f05793e8, 2026-09-21.
A regex in hooks/no-unread-issue-claim.py had three separate \s* runs;
an earlier round bounded two of them and left the third unbounded, so the pattern's longest possible match was still unbounded end to end.
0388cdfa bounded the third run too and stated, correctly, that the source now had all three bounded --- and added a second claim in the same commit message: "Four cases now pin what the comment claims --- a prefix at the full 38 characters, the bound at exactly 8, one character past it at 9, and a padded run inside pull request that only a bounded inner run rejects."
The source claim was Scope over the regex's own three quantifiers and was true;
a differential read of the pattern confirms all three are bounded.
The test claim was also Scope, over the same three quantifiers, this time asking whether each has a case that fails if that specific bound is removed --- and it was false.
f05793e8, one commit later: "The cases pinned ONE.
Widening either of the other two by a single character, or reverting the trailing one to unbounded, left 72 of 72 green."
Two of the three runs had no case that depended on their own bound at all;
the suite was green regardless of what those two runs allowed.
0388cdfa's own commit message states the general form of the mistake it was about to make, one clause before making it: "the whitespace is bounded now" is not a claim you can check by rereading the sentence you just wrote.
It is a claim you check by counting the quantifiers.
That sentence is correct about the source claim it was defending and was never applied to the test claim two sentences later in the same message --- the general rule was stated and a different, structurally identical instance of the exact violation it names shipped in the same commit.
Recognizing a claim as Scope ("all N of X now do Y") is not, by itself, the check. The check is deriving N and confirming each member independently --- here, one assertion per whitespace run, not a shared prefix-length case that happens to move if any of the three shrinks. A single passing case that is consistent with all three bounds holding is not evidence that all three are independently pinned, the same gap "A parity comment justified a fix that only partly achieved the parity it named" above names for a different claim shape: a directional or aggregate check confirms movement toward a claim, not the claim's exact scope.)
- Do: for a claim of the shape "all N of X now do Y," derive N (count the quantifiers, list the branches, enumerate the members) and confirm each one has its own independent check, rather than confirming the surrounding sentence reads correctly.
- Do: treat "I already stated the general rule this violates, earlier in the same message" as no defense --- stating the rule and checking against it are different acts, and the first does not perform the second.
- Don't: accept a single case that is consistent with several claimed properties as pinning all of them --- widen or revert each property independently and confirm the suite catches it.
- Don't: read a source-level Scope claim ("all three runs are bounded") as carrying a test-level Scope claim ("and all three are independently tested") for free --- they are two different populations that happen to share a sentence.