Skip to content

v1.20.0: the audit blocked the shape the skill itself prescribes - #16

Merged
wan-huiyan merged 1 commit into
mainfrom
feat/severity-exemption-and-chip-ordering
Aug 7, 2026
Merged

v1.20.0: the audit blocked the shape the skill itself prescribes#16
wan-huiyan merged 1 commit into
mainfrom
feat/severity-exemption-and-chip-ordering

Conversation

@wan-huiyan

Copy link
Copy Markdown
Owner

Found by running this skill end to end on a session that had a review round.

The defect is not the false positive — it is what the false positive teaches

label_audit.py flagged §7's P0/P1/P2/P3 severity column: the exact table the skill's own handoff template asks for. So every handoff carrying a review round hits a false positive on its first run, and the documented response is the frontmatter escape hatch.

A skip waives the ENTIRE file. So a document reaching for the hatch to clear one nuisance table also silently waives the real code legend eight sections further down — which is the failure the audit exists to prevent. A false positive on a prescribed shape does not cost 30 seconds; it trains authors to waive files by reflex.

The fix

Severity grades are exempt in the scanner. They are the author's own assignment: there is no external system to fabricate against, the row's description is the claim, and the disposition column already cites where it was fixed.

Covers P0P9, S0S9, SEV0SEV9, and CRITICAL / BLOCKER / HIGH / MEDIUM / LOW / INFO / MAJOR / MINOR / TRIVIAL / NIT.

Real legends still block, and one fixture proves both halves

tests/fixtures/label_audit_severity.md carries a P-number table, a HIGH/MEDIUM/LOW table and a Salesforce code table in the same file. The test asserts the seven severity findings are silent while Accepted Fully still blocks.

Confirmed to go red with the exemption removed:

✖ exempts self-assigned severity grades but still blocks real code legends
  AssertionError: severity-graded row must be exempt: /ungated funnel/
ℹ pass 14
ℹ fail 1

…then restored, 77 pass / 0 fail.

Verified against the case that prompted it: the session's own handoff passes CLEAN with its label-audit-skipped: frontmatter deleted, so it no longer needs the hatch at all.

Step 14 also gains the terminating chip's ordering

The last chip in a chain must name its own pull request, which tempts you to write the number before the PR exists. It will be wrong: on 2026-08-07 a session wrote 846 and GitHub allocated 849, three others having opened in between.

A guessed chip is the worst kind of wrong here because it is plausible — it points at a real, recently-merged PR belonging to somebody else's work, so nothing looks broken and the validator passes. Create, read the number it prints, then --amend.

Process note

Step 24e's cache-vs-source diff was run first, as it requires. The only difference was this change; no cache-only content needed rescuing.

npm test: 79 tests, 77 pass, 0 fail.

🤖 Generated with Claude Code

Found by running this skill end to end on a session that had a review round.

**`label_audit.py` flagged §7's `P0`/`P1`/`P2`/`P3` severity column** — the exact
table the skill's own handoff template asks for. So every handoff carrying a
review round hits a false positive on its first run, and the documented response
is the frontmatter escape hatch.

**That is the actual defect.** A skip waives the ENTIRE file, so a document
reaching for it to clear one nuisance table also silently waives the real code
legend eight sections further down — which is the failure the audit exists to
prevent. A false positive on a prescribed shape does not cost 30 seconds; it
trains authors to waive files by reflex.

Severity grades are now exempt in the scanner. They are the author's own
assignment: there is no external system to fabricate against, the row's
description IS the claim, and the disposition column already cites where it was
fixed. Covers `P0`-`P9`, `S0`-`S9`, `SEV0`-`SEV9`, and
CRITICAL/BLOCKER/HIGH/MEDIUM/LOW/INFO/MAJOR/MINOR/TRIVIAL/NIT.

**Real legends still block, and the fixture proves both halves in one file** —
`label_audit_severity.md` carries a P-number table, a HIGH/MEDIUM/LOW table and
a Salesforce code table, and the test asserts the seven severity findings are
silent while `Accepted Fully` still blocks. **Confirmed to go red with the
exemption removed** (`✖ exempts self-assigned severity grades…`), then restored.

Verified against the real case that prompted it: the session's own handoff
passes CLEAN with its `label-audit-skipped:` frontmatter deleted, so the file no
longer needs the hatch at all.

**Step 14 also gains the terminating chip's ordering.** The last chip in a
chain must name its own pull request, which tempts you to write the number
before the PR exists. It will be wrong: on 2026-08-07 a session wrote `846` and
GitHub allocated `849`, three others having opened in between. A guessed chip is
the worst kind of wrong here because it is *plausible* — it points at a real,
recently-merged pull request belonging to somebody else's work, so nothing looks
broken and the validator passes. Create, read the number it prints, then amend.

Step 24e's cache-vs-source diff was run first, as it requires: the only
difference was this change, and no cache-only content needed rescuing.

npm test: 79 tests, 77 pass, 0 fail.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@wan-huiyan
wan-huiyan merged commit d7283ca into main Aug 7, 2026
3 checks passed
@wan-huiyan
wan-huiyan deleted the feat/severity-exemption-and-chip-ordering branch August 7, 2026 09:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant