v1.20.0: the audit blocked the shape the skill itself prescribes - #16
Merged
Merged
Conversation
Found by running this skill end to end on a session that had a review round. **`label_audit.py` flagged §7's `P0`/`P1`/`P2`/`P3` severity column** — the exact table the skill's own handoff template asks for. So every handoff carrying a review round hits a false positive on its first run, and the documented response is the frontmatter escape hatch. **That is the actual defect.** A skip waives the ENTIRE file, so a document reaching for it to clear one nuisance table also silently waives the real code legend eight sections further down — which is the failure the audit exists to prevent. A false positive on a prescribed shape does not cost 30 seconds; it trains authors to waive files by reflex. Severity grades are now exempt in the scanner. They are the author's own assignment: there is no external system to fabricate against, the row's description IS the claim, and the disposition column already cites where it was fixed. Covers `P0`-`P9`, `S0`-`S9`, `SEV0`-`SEV9`, and CRITICAL/BLOCKER/HIGH/MEDIUM/LOW/INFO/MAJOR/MINOR/TRIVIAL/NIT. **Real legends still block, and the fixture proves both halves in one file** — `label_audit_severity.md` carries a P-number table, a HIGH/MEDIUM/LOW table and a Salesforce code table, and the test asserts the seven severity findings are silent while `Accepted Fully` still blocks. **Confirmed to go red with the exemption removed** (`✖ exempts self-assigned severity grades…`), then restored. Verified against the real case that prompted it: the session's own handoff passes CLEAN with its `label-audit-skipped:` frontmatter deleted, so the file no longer needs the hatch at all. **Step 14 also gains the terminating chip's ordering.** The last chip in a chain must name its own pull request, which tempts you to write the number before the PR exists. It will be wrong: on 2026-08-07 a session wrote `846` and GitHub allocated `849`, three others having opened in between. A guessed chip is the worst kind of wrong here because it is *plausible* — it points at a real, recently-merged pull request belonging to somebody else's work, so nothing looks broken and the validator passes. Create, read the number it prints, then amend. Step 24e's cache-vs-source diff was run first, as it requires: the only difference was this change, and no cache-only content needed rescuing. npm test: 79 tests, 77 pass, 0 fail. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Found by running this skill end to end on a session that had a review round.
The defect is not the false positive — it is what the false positive teaches
label_audit.pyflagged §7'sP0/P1/P2/P3severity column: the exact table the skill's own handoff template asks for. So every handoff carrying a review round hits a false positive on its first run, and the documented response is the frontmatter escape hatch.A skip waives the ENTIRE file. So a document reaching for the hatch to clear one nuisance table also silently waives the real code legend eight sections further down — which is the failure the audit exists to prevent. A false positive on a prescribed shape does not cost 30 seconds; it trains authors to waive files by reflex.
The fix
Severity grades are exempt in the scanner. They are the author's own assignment: there is no external system to fabricate against, the row's description is the claim, and the disposition column already cites where it was fixed.
Covers
P0–P9,S0–S9,SEV0–SEV9, and CRITICAL / BLOCKER / HIGH / MEDIUM / LOW / INFO / MAJOR / MINOR / TRIVIAL / NIT.Real legends still block, and one fixture proves both halves
tests/fixtures/label_audit_severity.mdcarries a P-number table, a HIGH/MEDIUM/LOW table and a Salesforce code table in the same file. The test asserts the seven severity findings are silent whileAccepted Fullystill blocks.Confirmed to go red with the exemption removed:
…then restored, 77 pass / 0 fail.
Verified against the case that prompted it: the session's own handoff passes CLEAN with its
label-audit-skipped:frontmatter deleted, so it no longer needs the hatch at all.Step 14 also gains the terminating chip's ordering
The last chip in a chain must name its own pull request, which tempts you to write the number before the PR exists. It will be wrong: on 2026-08-07 a session wrote
846and GitHub allocated 849, three others having opened in between.A guessed chip is the worst kind of wrong here because it is plausible — it points at a real, recently-merged PR belonging to somebody else's work, so nothing looks broken and the validator passes. Create, read the number it prints, then
--amend.Process note
Step 24e's cache-vs-source diff was run first, as it requires. The only difference was this change; no cache-only content needed rescuing.
npm test: 79 tests, 77 pass, 0 fail.🤖 Generated with Claude Code