Skip to content

Refactor specification workflow to prevent requirement and test accretion - #39

Open
ncrmro with Copilot wants to merge 11 commits into
mainfrom
copilot/prevent-requirement-accretion
Open

Refactor specification workflow to prevent requirement and test accretion#39
ncrmro with Copilot wants to merge 11 commits into
mainfrom
copilot/prevent-requirement-accretion

Conversation

Copilot AI commented Sep 6, 2026

Copy link
Copy Markdown
Contributor
  • Specify anti-accretion reviewer and authoring outcomes
  • Obtain fresh-context critique of the draft requirements
  • Implement reviewer guidance and calibration examples
  • Add focused requirement-linked tests
  • Refresh independent verdicts and run full validation
  • Scan changes and complete security review

Copilot AI and others added 2 commits September 6, 2026 21:48
Co-authored-by: ncrmro <8276365+ncrmro@users.noreply.github.com>
Co-authored-by: ncrmro <8276365+ncrmro@users.noreply.github.com>

@luce-unsup luce-unsup left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Commenting on the draft (I was the reviewer on #38, and the mention pointed me here). The direction is coherent with the issue: the diff maps onto all eight of #38's acceptance criteria — symmetric probes (REQ-003.1.10/.12), shared evidence across IDs (.13), rejection of wording-pinning with the exceptions carve-out (.14 — thank you for folding that in), parameter-case judgment (.15), author classification guidance (AGENTS.md / REQ-004.3.2), verdict-invalidation tradeoff docs (README + scaling.md + REQ-008.3), the five calibration controls (015–019, matching criterion 7's examples), and a rollout that stays guidance-only (scaling.md "Anti-accretion rollout"). Two substantive concerns are inline; the first is the one I'd resolve before the validation box gets checked:

  1. Corpus regression risk (inline on src/review.ts). The template edit deletes the conjunct/boundary enumeration and the grammar edge-production sentence. Corpus cases 003 and 004 anchor on exactly that language, and REQ-003.4.2 makes losing their detection a regression by definition. The new .15 parameterized-evidence question guards over-enumeration but nothing replaces under-coverage probing for universal quantifiers or grammars. Either restore a one-sentence under-coverage probe or consciously amend the corpus — but that decision needs to be visible in this PR, not discovered after check runs.

  2. Substring pins on generated guidance (inline on tests/cli.test.ts). Five exact-substring assertions against the init-generated AGENTS.md section sit right on the prose-tripwire line this PR draws. Fine if the section's key phrases are declared a delivered-text contract; otherwise they will read as tripwires to the very reviewer standard this PR introduces.

One smaller note: new REQ-008.3 is itself a documentation MUST — the class #38 §3 says belongs in verification notes. It follows the existing REQ-008.1/.2 precedent and is [review]-tagged rather than substring-tested, so I think it's defensible dogfooding, but it should be a conscious decision rather than accretion by default.

Also expected, not a finding: editing REQ-003.1.10–.15 deliberately stales their recorded verdicts (hash invalidation), so the remaining checklist items — npx rfc2119 check, security review — are where the gate gets re-earned.

Comment thread tests/cli.test.ts Outdated
Comment thread src/review.ts Outdated
Agent-Logs-Url: https://github.com/Unsupervisedcom/2119/sessions/498a2ad9-cd1a-4028-be9a-345328aa1242

Co-authored-by: ncrmro <8276365+ncrmro@users.noreply.github.com>
@ncrmro

ncrmro commented Sep 7, 2026

Copy link
Copy Markdown
Collaborator

@copilot please complete this work

Copilot AI and others added 4 commits September 7, 2026 18:04
Co-authored-by: ncrmro <8276365+ncrmro@users.noreply.github.com>
Co-authored-by: ncrmro <8276365+ncrmro@users.noreply.github.com>
Co-authored-by: ncrmro <8276365+ncrmro@users.noreply.github.com>
Agent-Logs-Url: https://github.com/Unsupervisedcom/2119/sessions/5e3d9249-c7a8-4d3d-b3da-55d1977345f1

Co-authored-by: ncrmro <8276365+ncrmro@users.noreply.github.com>
@ncrmro ncrmro changed the title [WIP] Refactor specification workflow to prevent requirement and test accretion Refactor specification workflow to prevent requirement and test accretion Sep 8, 2026
@ncrmro
ncrmro marked this pull request as ready for review September 8, 2026 19:42

@ncrmro ncrmro left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adversarial review complete against 925b8265bb9837879d5b877a5225ac7e07898012.

Reviewed the full base-to-head diff across three independent lenses: correctness/regression behavior, test and specification quality, and TypeScript/security/verdict integrity. The review found and the branch fixed three substantive issues before approval: direct judgments were missing the symmetric anti-accretion probes; audit packets no longer explicitly scanned conjunct, boundary, precedence, grammar, and data-shape obligations; and the new acceptance tests pinned generated prose instead of using semantic judgment. A follow-up documentation pass also corrected the stated boundary of annotation-block invalidation.

No unresolved P0/P1/P2 findings remain. npm test passes (18 files, 165 tests), npm run check passes with zero violations/uncovered/failing/stale reviews, and the GitHub Actions 2119 check is green.

Review limitations: static reviewers did not independently execute the frozen snapshot. The orchestrator executed the full suite and gate in the implementation worktree. One lane proposed globally versioning the built-in review policy in every review hash; I did not treat that as an introduced defect because the issue explicitly scopes this rollout to guidance/observational behavior and the current hash contract is content-bound. That is a potential follow-up design decision, not a blocker for this PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Prevent requirement and test accretion from proof-of-wording incentives

3 participants