Consolidate RFC 2119 requirements and low-value tests without weakening coverage

[ncrmro]

Context

An audit of Panopticon's RFC 2119 specifications and test suite found substantial real behavioral coverage alongside requirement and test accretion driven by duplicated contracts, artificial clause splitting, documentary/test-process MUSTs, implementation prescriptions, and exhaustive matrices.

Upstream workflow issue: Unsupervisedcom/2119#38

The current snapshot contains approximately:

• 64 specification files;
• 761 active normative requirements;
• 3,277 collected test executions, compared with 683 in the original Panopticon repository;
• 2,115 unique collected test bodies, compared with 627 originally.

The audit estimates that the requirements can be consolidated to roughly 400 canonical normative contracts without removing an external interface or security invariant. It estimates that 500–1,050 collected executions and 125–380 unique bodies may then be removable or collapsible. These are review ranges, not quotas.

Goal

Clean up the requirements first, then consolidate low-value tests while preserving happy paths, plausible production edge cases, security invariants, agent-facing instruction behavior, compatibility, and historical regression coverage.

This issue MUST NOT be implemented as a bulk test deletion or as an attempt to hit a preferred count.

Main findings

Artificial splitting and duplication

• Turn and blocked-marker lifecycle behavior is repeated across REQ-008, REQ-010, REQ-016, REQ-026, and REQ-045.
• Authentication failure shape, route classes, credential channels, bootstrap credentials, secret non-disclosure, and snapshot lifetime are repeated across REQ-035, REQ-044, REQ-045, REQ-047, REQ-048, REQ-049, REQ-050, and REQ-054.
• REQ-002 divides one delivered review-skill contract into many sentence-level requirements.
• REQ-011 divides one status-precedence table into clauses that invited complete Cartesian coverage.
• REQ-030 divides copy/paste precedence into platform and binding permutations.

Requirements that should leave the normative product contract

• file-scoped-requirement-ids.md primarily specifies tool adoption, CI pinning, migration history, and contributor guidance.
• REQ-044 primarily prescribes pytest guards, test construction, subject-count floors, diagnostics, and an exact documentation sentence. Its security invariants belong canonically under authentication requirements; its testing details belong in verification notes.
• Parts of REQ-008, REQ-024, REQ-045, REQ-046, REQ-049, REQ-051, and REQ-054 prescribe documentation or test procedure rather than product behavior.

Implementation prescriptions

Several requirements name internal exception types, helper methods, class attributes, framework types, clock seams, parsing details, or internal command construction. They SHOULD be rewritten as outcomes observable through supported surfaces unless the mechanism itself is a compatibility contract.

Resulting test accretion

• exact prose and documentation assertions;
• digest pins for repository files, generated strings, and test scaffolding;
• approximately one fingerprint execution per packaged source file;
• Cartesian authentication and status matrices whose rows often reach identical behavior;
• tests of test-only shell/AST guards;
• duplicate bodies associated with split or renamed requirements;
• opt-in end-to-end evidence that skips in CI while its scaffolding tests always run.

Proposed work

Phase 1: establish evidence and canonical contracts

• Record collection counts, default-CI skips, wall-clock durations, and the slowest tests.
• Sample targeted mutations for the largest matrices before changing them.
• Inventory every requirement as retain, merge, downgrade, rewrite, or remove, preserving stable identifiers or explicit removal placeholders where required by the tooling.
• Identify the canonical requirement for every duplicated security or compatibility invariant.
• Record historical bugs or reviewer findings that justify otherwise surprising granularity.

Phase 2: clean up requirements

• Move tool adoption, CI commands, test construction, count floors, and review procedure into verification/maintenance notes.
• Merge lifecycle, authentication, credential, spawn, status, transcript, hint/help, and prompt-content clusters into canonical behavioral tables or contracts.
• Rewrite implementation prescriptions as observable outcomes.
• Retain agent-facing prompt requirements when the instruction materially controls behavior, but specify the obligation rather than exact paragraph layout.
• Remove only duplicated, unreachable, or negative-of-nothing requirements; preserve each security invariant exactly once.
• Avoid breaking changes to public interfaces, credential formats, routes, CLI behavior, stored data, or workflow compatibility.

Phase 3: consolidate tests against the cleaned contract

• Replace exact documentation/prose pins with tests of the actual rendered or delivered product surface where text is behavior.
• Replace frozen inventories with discovery from the composed application or real registry when appropriate.
• Collapse equivalent matrix rows to meaningful equivalence classes, boundaries, or pairwise cases after confirming mutation detection is unchanged.
• Share existing evidence across requirement IDs instead of cloning bodies.
• Remove tests of test-only machinery when the machinery disappears; do not leave compatibility or obsolete helpers behind.
• Make end-to-end evidence required in an appropriate CI job, or stop treating an always-skipped journey as complete coverage.

Acceptance criteria

• Every retained normative requirement MUST describe an observable product/delivered-text contract or a necessary security/compatibility invariant.
• Each invariant MUST have one canonical requirement; other specifications SHOULD reference it rather than restate it.
• No security scope, authentication failure shape, credential non-disclosure guarantee, supported interface, or historical regression MUST lose effective coverage.
• One test MAY cover multiple requirements; traceability MUST NOT require duplicate bodies.
• Harmless rewording, formatting, renaming, or repository reorganization MUST NOT break behavioral evidence unless the changed text is itself the supported product surface.
• Large remaining matrices MUST have evidence that their cases detect distinct plausible defects or protect a derived security inventory.
• Collection, duration, skip, and targeted-mutation comparisons MUST be reported before and after cleanup.
• Obsolete requirements, test helpers, fixtures, and compatibility scaffolding within scope MUST be removed rather than silently retained.
• Cleanup MUST be delivered through reviewed, comprehensible changes rather than one opaque bulk rewrite.

Initial audit disposition

The read-only audit classified approximately 385 requirements to retain as written, 248 to merge, 81 to downgrade, 26 to rewrite, and 21 to remove. The totals are hand-tallied and may vary by a few items; disposition judgments may move as historical evidence is inspected.

The purpose of this issue is not to ratify those numbers. It is to turn the audit into a behavior-preserving cleanup with explicit evidence for every reduction.