fix(check-pr-fully-clean): stop scoring non-blocking as a finding - #3497
Conversation
…st "non-" The #2419 fix only exempted the non-/non compounds from the Block(?:ed|ing)? alternative, so "not blocking" and "no blocking findings remain" -- the same statement with a different negator immediately before the word -- still matched. Add (?<!not\s)(?<!no\s) lookbehinds alongside the existing non-/non\s ones. The generic NOT_CLEAN_NEGATION_PREFIX guard used by classify_verdict and _unresolved_finding_pattern had the mirror image of the same gap: its negator-to-phrase separator was a bare \s, so a hyphenated negation ("No-changes requested.") was not exempted even though the spaced form already was. Widen it to [\s-]. The existing hyphen-specific check on the "changes requested" pattern is widened the same way, though it turns out to be redundant with the NOT_CLEAN_NEGATION_PREFIX fix -- kept for consistency with the issue's sketch. Rejected and Unapproved were checked for the same treatment and left unguarded: unlike "non-blocking", there is no established review idiom "non-rejected" or "not unapproved" anywhere in this corpus's own prose or test fixtures, so a speculative guard there would only risk swallowing a genuine finding for no real benefit. Both directions are pinned with mutation-checked tests: the four new/positive exemption checks fail if the lookbehinds or NOT_CLEAN_NEGATION_PREFIX widening are reverted, and the "not a nit -- it is blocking" / "no doubt this is blocking" checks fail against a blanket negation-window implementation (confirmed against a standalone simulation of that anti-pattern) while passing against this fixed-width one. Closes #3487 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ctives Round-2 finding from the adversarial-reviewer dispatched against d287f21: substituting [\s-] for every \s in NOT_CLEAN_NEGATION_PREFIX (and the parallel changes-requested guard) let the hyphen stand in for a space inside the filler-word separator too, not just between the negator and the target phrase. That swallowed real not-clean statements whose negator word happens to open an unrelated hyphenated compound adjective -- "Not-negligible changes requested.", "No-nonsense changes requested here.", "Not-yet-addressed changes requested below.", and "Never-resolved changes requested." all silently classified as no-verdict instead of not-clean. Replace the blanket substitution with a dedicated hyphen alternative that only fires in the one position the fix actually needs: immediately after the negator, with no intervening filler word ("no-changes", "non-blocking"). Any filler word must still be space-separated, exactly as before #3487. Confirmed the four phrases above stay not-clean, added them as regression tests, and confirmed (by reverting to the blanket form) that those tests fail against it. Full suite: 797 passed, 0 failed. Live repro still exits 0: python3 scripts/check-pr-fully-clean.py 3468 -R Morrison-Lab/ai-config Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
Claimed by a Claude Code CLI session — driving this PR to a clean review and green CI. Found while measuring a different PR: the instrument reported #3468 not-clean for three consecutive turns, and I chased that verdict rather than asking why a PR whose every review says "Ready for merge" was scoring as having findings. Posted by Claude Code (AI agent) --- not written by a human. |
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
…line Review of #3497 found the hyphen branch written as `(?:no|not|nothing|none|never)(?:-|\s+...)`, which exempted `not-blocking`, `no-blocking`, `never-blocking` and `not-rejected` through the FULL `classify_verdict` / `_unresolved_finding_pattern` pipeline. That undid one layer up the narrowness `_BARE_REJECTION`'s own lookbehinds were written to preserve, and it undid it in the dangerous direction -- a swallowed not-clean. It passed 806 tests because the new cases asserted against `_BARE_REJECTION` directly, and the file says why: for the SPACE forms the bare level is the right place, since the generic guard already covers them. That reasoning does not carry to the hyphen forms, whose entire failure mode lives in the generic guard. So the hyphen cases are now asserted through the pipeline. `not`, `never`, `nothing` and `none` lose the hyphen. `not-negligible` and `not-a-nit` are compound adjectives whose hyphen does not negate what follows. `no-` keeps it, which is a partial rebuttal of the finding rather than a straight fix. The guard sees only the text BEFORE the matched phrase, so it cannot exempt `no-changes requested` (a real usage, #2369) while flagging `no-blocking` -- the two differ only in what follows. Granted `no-`, `no-blocking` reads clean, which is the right answer anyway: it means what `non-blocking` means. Both halves are now pinned by tests so the tradeoff cannot be silently reversed. Mutation-checked against the restored 806/806: the over-broad hyphen branch restored -> 800/806, 6 failing `python3 scripts/check-pr-fully-clean.py 3468 -R Morrison-Lab/ai-config` exits 0. Refs #3487 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
Pausing here. State, so nothing is lost. The review's finding stands, and my rebuttal was wrongI argued the guard could not flag And it found a second, worse one
All five classify Why 806 tests did not catch itThe new cases test Next step, for whoever picks this upScope the hyphen exemption to the pattern it was written for, using the
The pattern worth namingThree rounds on this PR, and each fix reintroduced the same defect class one layer out: Posted by Claude Code (AI agent) --- not written by a human. |
The review was right twice, and I withdraw the rebuttal. `NOT_CLEAN_NEGATION_PREFIX` is consulted for EVERY pattern, so granting it `non-`/`no-` silently swallowed `non-rejected`, `no-rejected`, `non-unapproved`, `non-impasse` and `non-deadlock` -- all `not-clean` on `main`, all reading clean on this branch. That is a regression this branch introduced, in the dangerous direction the whole chain exists to close, and it falsified this file's own claim that `Rejected`/`Unapproved` were left unguarded: true of the lookbehinds, false of the shipped behaviour. I had also argued the guard could not flag `no-blocking` while exempting `no-changes requested`, because it sees only the text before the match. The loop carries `pat` and already branches on it three times, so that was a property of the regex I wrote rather than of the pipeline. The fix needs neither the shared guard nor an enumeration in the loop -- this file already lost twice to enumerating which patterns need handling. `no-changes requested` is one phrase, so the hyphen goes on that phrase: `(?<!no-)Changes\s+requested`, in `_BARE_REJECTION` and in both pattern lists. The generic guard returns to space-only, and every hyphenated negator before any not-clean phrase stays flagged. `non-blocking` -- the case #3487 is about -- needs none of this. It was already covered by `_BARE_REJECTION`'s own `(?<!non-)` lookbehind; the hyphen in the shared guard was never load-bearing for it. Mutation-checked against the restored 811/811: generic hyphen exemption re-added -> 805/811, 6 failing `(?<!no-)` dropped from _BARE_REJECTION -> 809/811, 2 failing `(?<!no-)` dropped from both pattern lists -> 808/811, 3 failing `python3 scripts/check-pr-fully-clean.py 3468 -R Morrison-Lab/ai-config` exits 0. Refs #3487 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
… comment Three findings from the review of 6065247, all accepted. **The lookbehinds had no word boundary.** `(?<!no-)` and `(?<!no\s)` fire on any longer word ending in those letters, so "Volcano-changes requested.", "Casino-changes requested." and "casino blocking issue remains" all classified as no-verdict instead of not-clean -- a swallowed not-clean, from a guard added to stop swallowing not-cleans. Every lookbehind is now `\b`-anchored. **The `if pat == r"changes\s+requested\b":` branch became unreachable** when the previous commit put `(?<!no-)` into both list entries: the string equality can no longer hold. Its own comment called it redundant, which it was one commit earlier; it is dead now, and deleting it changes no test. **The comment block above `NOT_CLEAN_NEGATION_PREFIX` still described the mechanism the previous commit removed**, asserting that `no-` keeps the hyphen and `no-blocking` reads clean -- contradicted by the shipped regex and by the tests beside it. Rewritten to say what the code now does and why the hyphen lives on the phrase instead of the shared guard. Mutation-checked against the restored 817/817: `\b` dropped from the Block lookbehinds -> 814/817, 3 failing `\b` dropped from the `no-` guards -> 814/817, 3 failing `python3 scripts/check-pr-fully-clean.py 3468 -R Morrison-Lab/ai-config` exits 0. Refs #3487 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
Claude finished review — View run SummaryThis round is a direct, targeted fix for the exact regression I flagged in the prior review round (commit The fix removes the hyphen alternative from the shared I independently re-derived and ran the regexes rather than trusting the commentary or the new tests, per this repo's own All match expectations. I also confirmed the full local test suite passes: I found no other issues in this round's diff — no new patterns introduced, no scope creep, and the redundant code removal was correctly scoped (only the dead FindingsNone. The prior round's finding ( VerdictReady for merge — the previously-flagged regression is fixed and independently verified; no new issues found. Structured Review Data (JSON){
"schema_version": "1.1",
"reviewer": "claude",
"commit_sha": "348393ffcb0aa9c44f7a523a952853d453a98581",
"verdict": "CLEAN",
"findings": [],
"detailed_assessment": "The NOT_CLEAN_NEGATION_PREFIX hyphen-branch regression flagged in the prior round (commit 50885dc1) is fixed by scoping the hyphen exemption onto the specific Changes-requested phrase in scripts/check-pr-fully-clean.py rather than the shared negation-prefix guard, verified independently via classify_verdict on not-blocking, no-blocking, never-blocking, and not-rejected all correctly returning not-clean.",
"holistic_assessment": "The fix closes the exact test-coverage gap that let the regression pass 797 tests last round by adding pipeline-level tests through classify_verdict and _unresolved_finding_pattern, and the full local suite of 843 tests passes with CI showing no failed checks."
}Reviewed commit: 163a963 |
check-pr-fully-clean.pyscored the stringnon-blockingas a finding.\bblocking\bmatches insidenon-blocking— the hyphen ends the preceding word, so the word boundary sits right beforeblocking. The only negation guard in the file was scoped tochanges\s+requestedand tested\bno\s+…, which requires a space, so a hyphen was never going to match anyway.Why it mattered
"Non-blocking" is the commonest way a reviewer marks a finding as explicitly not blocking, so the instrument reported NOT CLEAN precisely when a reviewer had gone out of their way to say the opposite.
That is the instrument committing the error it exists to prevent. The whole argument for
check-pr-fully-clean.pyover a hand-rolled phrase search — the one its own guard hook makes when it refuses agrepfor### Verdict— is that phrase searches misread in both directions.Live before this change, on #3468:
All nine matches sat inside sentences like "Ready for merge. One non-blocking suggestion … does not block this PR's own content". After this change the same command exits 0.
The change
_BARE_REJECTION'sBlock(?:ed|ing)?lookbehinds now covernot\sandno\sas well asnon-andnon\s.NOT_CLEAN_NEGATION_PREFIXaccepts a hyphen after the negator through a dedicated alternation that fires only immediately after it — deliberately not a blanket[\s-]substitution._unresolved_finding_pattern'schanges requestedbranch is redundant with the above and kept for parity with the issue's sketch; its comment says so, and reverting it fails no test.RejectedandUnapprovedare left unguarded. A grep fornon-rejected/not rejected/no rejectedand theUnapprovedequivalents found no genuine idiom anywhere in the repo, so a speculative guard there would only risk swallowing a real finding for no benefit.Why the blanket substitution was wrong
The obvious
[\s-]version was written first, and review caught it swallowing real findings: "Not-negligible changes requested." classified clean. That is the dangerous direction — a false clean — which is why the fix narrows to hyphen-immediately-after-negator rather than treating hyphens and spaces alike everywhere in the separator.Verification
797 tests pass. 20 are new, each mutation-checked:
not\s/no\slookbehinds reverted\s[\s-]reintroducedchanges requestedguard revertedDangerous-direction controls, all still correctly flagged as findings: "this is not a nit — it is blocking", "no doubt this is blocking", "not-negligible changes requested", "non-trivial changes requested", "nothing here is optional: blocking". Cross-clause too: "Non-blocking suggestion. Changes requested regardless." stays not-clean, since the hyphen-adjacency window does not leak across an independent clause.
Closes #3487
🤖 Generated with Claude Code