Make the mutation gate honest on any runner, and move to the current stack - #37
Conversation
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y7QdHVqb851SbSZCeitqg4
A Survived verdict is only sound if a covering test actually executed. @stryker-mutator/vitest-runner reports every covered mutant as Survived on vitest 5 (stryker-js#6210) because its per-test filter matches nothing, so each mutant run executes nothing and the runner returns an unexamined default. unrunSurvivedMutants recognises that shape from the report alone. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y7QdHVqb851SbSZCeitqg4
Replaces a storm of false stryker/survived with one guardrails/analyzer-failed naming the count. Fails closed: dropping the mutants and reporting nothing would be a fail-open on the analyzer this pack most depends on. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y7QdHVqb851SbSZCeitqg4
The banner guard proves we read stryker's output correctly. Nothing proved the runner still detects anything, which is the gap stryker-js#6210 walked through: on vitest 5 every covered mutant comes back Survived, so the gate manufactures violations. Verified both ways -- passes on vitest 4, fails on vitest 5 with 14 unrun survivors. The fixture is built in a temp dir rather than checked in, because a fixture inside the repo is copied into stryker's sandbox during the repo's own mutation pass and the nested run cannot resolve a runner plugin from there. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y7QdHVqb851SbSZCeitqg4
unicorn 74 renamed prevent-abbreviations -> name-replacements (same allowList option) and added ~15 rules. Every finding in guardrails-core/src is fixed in the code, not silenced. Two rules are relaxed for TEST FILES ONLY, each with its reason in eslint.config.js: no-top-level-assignment-in-function (flags vitest's own `let x; beforeEach(() => x = ...)` idiom) and max-nested-calls (raised to 4, the depth of `expect(await run(build(arg)))`). Tool-upgrade review per CLAUDE.md: loose-rules.ts needed no change (the new unicorn rules are style, not the loose classes -- test-integrity, architecture, mutation, dead-code); audit.ts needed no change (no suppression syntax moved). Three findings surfaced by running the upgraded gate on ourselves: - ESLint 10 resolves config files per directory, so it LOADED the eslint configs inside .claude/worktrees/ and failed on a rule id renamed by this upgrade. Same untracked-but-not-ignored class as the nested-worktree finding already in plan.md; both worktrees and .stryker-tmp are now ignored. - The drift guards cannot run inside our own mutation run -- eslint 10's project service uses worker threads that die in a stryker worker, and the new runner guard nests a vitest in a vitest. Both now stand down there, sharing one documented rule; they still run everywhere else. - lib was ES2023 while engines.node is >=24, so Iterator#toArray was unavailable. Raised to ES2024 + ESNext.Iterator, matching the declared runtime. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y7QdHVqb851SbSZCeitqg4
tsup 8.5.1's dts pass cannot parse TypeScript 6 -- `Error: error occurred in dts build` -- and it is the latest release, so `dts: false` and a `tsc -p tsconfig.build.json` step after it (tsup cleans dist/, so order matters). The type output becomes one .d.ts per module instead of a bundle; package-exports now asserts the exports map's `types` target actually exists, and the tarball smoke test passes, which is what proves the published shape. TypeScript 7 stays out of reach: typescript-eslint has no v9 and 8.69 declares typescript >=4.8.4 <6.1.0. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y7QdHVqb851SbSZCeitqg4
- .gitignore now covers reports/stryker-incremental.json. Stryker's incrementalFile default is not under reports/mutation/, so a greenfield repo's first `git add -A` committed a mutation-result cache. Not a gate fail-open -- runStryker deletes that file before every run. - The skipped-analyzer warning now names `guardrails init --apply`. init seeds a starter config only for an analyzer the repo already declares, so a bare greenfield init writes none of them, and installing the tools afterwards meets analyzer-failed with upstream's --init advice attached, which writes a different config than the seed. The gating stays; this is the missing pointer. - GIT_WRITE learned git's global options, so `git -c core.hooksPath=/dev/null commit` is gated. Unlike `git -C <path>` (an accepted miss -- the hooks still run), that one defeats the hook floor, and it is the exact bypass the scaffolded AGENTS.md forbids by name. Prose still returns in 0.03s. - adopting-guardrails now pins vitest with stryker-js#6210 as the reason, and the worked-example table carries the runner rows. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y7QdHVqb851SbSZCeitqg4
Also DRYs stryker-adapter's report parsing into one `parseReport`. The new unrunSurvivedMutants had copied the try/catch that carries a sanctioned equivalent-mutant directive, which sanctions-check correctly caught as the declared count drifting from 1 to 2. Sharing the parse is the fix; widening the exemption would have bought no new behaviour. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y7QdHVqb851SbSZCeitqg4
|
Claude finished @ddteeter's task in 4m 25s —— View job Review: PR #37 — mutation gate honesty + stack upgrade
Reviewed the diff directly ( Core fix:
|
From PR review: if a runner populated coveredBy but never testsCompleted, the guard would reclassify its genuine survivors as analyzer-failed. Measured on the command runner -- the one init seeds by default -- with a vacuous test: 11 genuine survivors, every one coveredBy: []. A runner that cannot attribute tests to mutants reports no covering list at all, so it exits at the covering === 0 check. Zero misfires. Recorded on the test that covers it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y7QdHVqb851SbSZCeitqg4
|
Checked the It cannot misfire, and the reason is the other half of the condition. A runner without per-test data does not report a partial record — it reports no covering list at all. Measured on the Every genuine survivor comes back So the Pinned in 4dbefd8 on the test that already covered the shape, with the measurement, so the reasoning survives the next runner bump. |
|
Claude finished @ddteeter's task in 6m 11s —— View job Review: PR #37 (re-review after
|
Confirms in a real consumer repo what the unit tests assert: the exit criterion is reachable in one pass, and on vitest 5 the adopter gets one honest analyzer-failed instead of 17 false survivors handed to the fixer. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y7QdHVqb851SbSZCeitqg4
|
Claude finished @ddteeter's task in 2m 4s —— View job Review: PR #37 (re-review after new push)
The new commit since the last automated pass is Outstanding:
|
Caught in PR review. The option alternation could tile a run of flags two ways (`-c -c` as flag-plus-value or as two flags), the (a|aa)* shape: Fibonacci(k) partitions a failing match must exhaust. Measured 5.69s at 40 tokens and 51s at 64 on the Bash PreToolUse hook's own input -- a long enough command would have hung the session. Requiring a -c value to start with a non-dash leaves one parse: 0.02ms at n=1000, every gating case unchanged. Worth recording why lint missed it: splitting the literal into String.raw fragments to satisfy sonarjs/regex-complexity also took it out of static analysis. Pinned by an adversarial timing test instead. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y7QdHVqb851SbSZCeitqg4
|
You were right, and I was wrong to ship it. Measured it before fixing:
Exactly the Fibonacci tiling you described — Fixed in 43ca11a by requiring a The part worth keeping from this. This repo's own Thanks — this was the one thing in the PR that could have made the gate worse than not having it. |
|
Claude finished @ddteeter's task in 2m 21s —— View job Review: PR #37 (re-review after
|
Raised in review as the one place the linearity fix could have traded time for a false negative: `git -c http.extraHeader=-x commit` is now read as two flags rather than flag-plus-value. Different internal parse, same verdict. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y7QdHVqb851SbSZCeitqg4
|
Claude finished @ddteeter's task in 3m 5s —— View job Review: PR #37 (re-review after
|
The finding
A fourth greenfield adoption, run from the packed tarball with the analyzer set at the versions npm resolves today, found that the mutation gate is broken on the current stack — and broken in the worst direction. Same code, same tests, one variable:
Cause is upstream and known: stryker-js#6210 — vitest 5 changed
testNamePatternto join the name chain with' > ', the runner still uses a space join, so its per-test filter selects nothing and each mutant run executes zero tests. Confirmed with a probe: the suite ran once (the dry run) and never again. No configuration works around it — all six combinations ofcoverageAnalysis×vitest.relatedreport 12/12 Survived.This does not miss violations, it manufactures them. In an agent-driven repo the fixer is handed twelve unkillable mutants, the ladder exhausts, and the only exits it can reach — a sanctioned suppression, or
analyzers.stryker: "off"— are both forbidden by the scaffoldedAGENTS.md. The guardrail teaches the agent to disable the guardrail.What this does about it
Stops it lying. The report carries an exactly discriminating signal:
coveredBytestsCompleted1— all 90— all 12A
Survivedverdict with covering tests and zero executions is not evidence of survival; it is a run that never happened.runStrykernow raises oneguardrails/analyzer-failednaming the count instead of a storm of falsestryker/survived. Upstream proposes the same invariant for itself in #6146 (unmerged since July), and that runner has three more open false-survivor bugs — so the guard stays after #6210 is fixed.Adds the guard that was missing.
test/drift/stryker-runner.test.tsruns real stryker through the vitest runner over code whose tests kill every mutant, and asserts kills come back. Verified both ways: passes on vitest 4, fails on vitest 5 with 14 unrun survivors. It is the only test in the suite that would have failed on avitest@5bump.Closes the dogfooding gap behind all of it. This repo was validating a stack no adopter gets — vitest 4.1 / stryker 9.6 / eslint 9.39 / TypeScript 5.9, against a greenfield's 5.0 / 10.0 / 10.10 / 6.0. Everything but vitest is now current. TypeScript 6 needed tsup's dts pass replaced with
tsc --emitDeclarationOnly, since tsup 8.5.1 (the latest) cannot parse it.vitestis pinned to^4with the reason in the file; unpinning is tracked in #35. This PR does not close #35 — it pins, it does not unpin.Also fixed, from the same adoption
git -c core.hooksPath=/dev/null commitwalked past the Bash gate. Unlike the acceptedgit -C <path>miss (the hooks still run), this one defeats the hook floor — and it is the exact bypassAGENTS.mdforbids by name. Prose still returns in 0.03s.init --applyseeds no analyzer configs (the gating is deliberate), and installing the tools afterwards meetsanalyzer-failedwith upstream's--initadvice attached, which writes a different config than the seed. The skipped-analyzer warning now namesguardrails init --applyas the second half of the fix.reports/stryker-incremental.jsonwas committed by the firstgit add -A. Not a gate fail-open —runStrykerdeletes it before every run..claude/worktrees/and failed on a rule id this upgrade renamed. Same untracked-but-not-ignored class as the existing nested-worktree finding, new surface.#!entry points were mutated as production code —isTestFilenow covers test directories, and a shebang file is excluded from mutation only (eslint and tsc still check it), exactly asisConfigFilealready is.Every unicorn 74 finding in
guardrails-core/srcis fixed in the code. Two rules are relaxed for test files only, each with its reason ineslint.config.js.Verification
npm run build,npm test(1106 passing),npm run lint,guardrails verify(clean),npm run test:coverage(97.9% lines),npm run check:graph,sanctions-check, andscripts/smoke-tarball.mjs— all green. No new sanctioned suppression: where the new code would have duplicated one, the parse was shared instead.Spec:
docs/superpowers/specs/2026-09-05-mutation-integrity-and-stack-upgrade-design.mdPlan:
docs/superpowers/plans/2026-09-05-mutation-integrity-and-stack-upgrade.mdFindings:
plan.md→ "Findings: a fourth greenfield adoption"🤖 Generated with Claude Code
https://claude.ai/code/session_01Y7QdHVqb851SbSZCeitqg4