Skip to content

feat(test-deletion-guardrail): block unjustified test removal in tdd-cycle + verify - #13

Merged
MountainUnicorn merged 3 commits into
mainfrom
feat/test-deletion-guardrail
Apr 23, 2026
Merged

feat(test-deletion-guardrail): block unjustified test removal in tdd-cycle + verify#13
MountainUnicorn merged 3 commits into
mainfrom
feat/test-deletion-guardrail

Conversation

@MountainUnicorn

Copy link
Copy Markdown
Owner

Summary

Implements specs/test-deletion-guardrail.md (v0.9.0, M3 pre-GA hardening). Defends ADD's signature TDD claim against the Kent Beck / TDAD-paper failure mode: coding agents prefer to delete the failing test rather than satisfy it, which silently corrupts the RED → GREEN separation.

  • scripts/check-test-count.py — stdlib-only snapshot / compare / gate CLI. One-shot --baseline <ref> form is the entry point invoked by /add:verify Gate 3.5.
  • core/lib/impact-hint.sh — pure shell + jq + grep "files-likely-affected" hint fed to /add:implementer at start of GREEN.
  • core/knowledge/test-discovery-patterns.json — regex catalog covering Python, TS/JS, Go, Ruby, Rust (extensible per project).
  • New Gate 3.5 "Test Surface Integrity" in /add:verify, between Gate 3 and Gate 4.
  • core/rules/tdd-enforcement.md gains the "Test-Deletion Invariant" section (AC-028).
  • scripts/compile.py now ships core/lib/ + core/knowledge/ into both runtimes.

Acceptance Criteria

  • AC-001..AC-006 — RED snapshot + fail-on-empty + commit convention
  • AC-007..AC-013 — GREEN re-snapshot + comparison (added/removed/renamed/replaced) + normalized body hashing
  • AC-014..AC-017 — Gate 3.5 placement, missing-snapshot error, structured summary
  • AC-018..AC-024 — Files-likely-affected hint (diff + import resolution + spec path scan + anti-pattern learnings lookup + empty-diff fallback)
  • AC-025..AC-027 — Telemetry JSONL (deferred to companion telemetry-jsonl spec; integration contract documented in this PR)
  • AC-028 — Rule invariant

Justification markers accepted by the gate:

  1. Commit trailer in range base..HEAD: [ADD-TEST-DELETE: <reason>]
  2. Per-cycle override file: .add/cycles/cycle-{N}/overrides.json with kind: "test-rewrite"

Test plan

  • 15 fixture tests in tests/test-deletion-guardrail/ covering the gate matrix: count-same, count-increased, count-decreased (no override → FAIL), count-decreased-with-approval, empty-before (bootstrap), rename, replacement-without-approval, replacement-with-approval, missing-green, compare-json-shape, rename-detection, compare-summary-format, discovery-python-basic, empty-red-fails, impact-hint-basic
  • python3 scripts/compile.py --check passes (no drift)
  • python3 scripts/validate-frontmatter.py passes
  • bash tests/hooks/test-filter-learnings.sh passes (regression check)
  • Dog-food: python3 scripts/check-test-count.py --baseline origin/main returns PASS (net test count increased from 0 to 0 in harness terms; no removals)

Part of

M3 pre-GA hardening (Cycle 3). Spec: specs/test-deletion-guardrail.md. Complements telemetry-jsonl (Cycle 3) which owns the JSONL emitter referenced by AC-025..AC-027.

Files Touched

Path Change
core/skills/tdd-cycle/SKILL.md §RED snapshot, §GREEN snapshot, §--allow-test-rewrite, Gate 3.5 reference
core/skills/verify/SKILL.md New Gate 3.5 between Gate 3 and Gate 4
core/skills/test-writer/SKILL.md Step 4b: capture + commit RED snapshot
core/skills/implementer/SKILL.md Pre-flight 1b (impact hint), Step 5b (no-delete directive), Step 7 (GREEN snapshot)
core/rules/tdd-enforcement.md New "Test-Deletion Invariant" section
core/knowledge/test-discovery-patterns.json New language regex catalog
core/lib/impact-hint.sh New shell helper
scripts/check-test-count.py New snapshot/compare/gate CLI
scripts/compile.py Copy core/lib/ + core/knowledge/ into both runtimes
tests/test-deletion-guardrail/ New fixture test suite + runner
docs/plans/test-deletion-guardrail-plan.md Implementation plan + AC coverage matrix
CHANGELOG.md Entry under [Unreleased]

🤖 Generated with Claude Code

MountainUnicorn and others added 3 commits April 23, 2026 05:40
Implements the comparison + gate portion of specs/test-deletion-guardrail.md.
scripts/check-test-count.py takes RED/GREEN test-surface snapshots, classifies
deltas as added/removed/renamed/replaced, and exits non-zero when tests were
removed without an [ADD-TEST-DELETE: ...] commit trailer or an
overrides.json record. One-shot --baseline <ref> form is the entry point
invoked by /add:verify Gate 3.5.

Ships:
- scripts/check-test-count.py (stdlib-only; snapshot / compare / gate subcmds)
- core/lib/impact-hint.sh (pure shell + jq + grep; AC-018..AC-024)
- core/knowledge/test-discovery-patterns.json (Python/TS/JS/Go/Ruby/Rust catalog)
- tests/test-deletion-guardrail/ — 15 fixture tests covering the gate matrix
- scripts/compile.py now copies core/lib/ and core/knowledge/ into both runtimes

Acceptance criteria covered: AC-001..AC-024, AC-028. AC-025..AC-027 (telemetry)
are deferred to the companion telemetry-jsonl spec.

Part of M3 pre-GA hardening. Spec: specs/test-deletion-guardrail.md.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds snapshot + gate hooks to the TDD sub-agent chain so the guardrail
shipped in the previous commit actually runs during a /add:tdd-cycle:

- /add:test-writer: capture RED snapshot + commit with standard message
  (AC-005); fail if zero tests added (AC-006)
- /add:tdd-cycle: emit files-likely-affected hint before GREEN; capture
  GREEN snapshot after GREEN; new --allow-test-rewrite flag for approved
  rewrites
- /add:implementer: consume hint in pre-flight (AC-022); explicit
  NEVER-delete directive (AC-028); capture GREEN snapshot (AC-007)
- /add:verify: new Gate 3.5 "Test Surface Integrity" between Gate 3 and
  Gate 4; skips cleanly when no cycle dir exists

Part of M3 pre-GA hardening. Spec: specs/test-deletion-guardrail.md.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds the Test-Deletion Invariant section to the auto-loaded TDD rule (AC-028):
"Tests added during RED MUST exist (passing) at the end of GREEN. Test deletion
during the cycle is forbidden without --allow-test-rewrite and explicit human
approval. Test renames are permitted; test replacements require approval."

Documents both accepted justification-marker formats:
  1. Commit trailer: [ADD-TEST-DELETE: <AC-id or reason>]
  2. Per-cycle file:  .add/cycles/cycle-{N}/overrides.json with kind:test-rewrite

Both land in telemetry for retro review. Cross-references the enforcement
points in /add:test-writer, /add:implementer, and /add:verify Gate 3.5.

Part of M3 pre-GA hardening. Spec: specs/test-deletion-guardrail.md.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@MountainUnicorn
MountainUnicorn force-pushed the feat/test-deletion-guardrail branch from 3db2b50 to 6fce85b Compare April 23, 2026 09:40
@MountainUnicorn
MountainUnicorn merged commit acac672 into main Apr 23, 2026
3 checks passed
@MountainUnicorn
MountainUnicorn deleted the feat/test-deletion-guardrail branch April 23, 2026 09:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant