Skip to content

Commit 4ab6d53

Browse files
nhortonclaude
andcommitted
chore: tighten requirements_traceability instructions and add shell_code_review output format
- Condensed requirements_traceability inline instructions (~94 → ~45 lines) while preserving all review criteria - Added explicit Output Format section to shell_code_review rule Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1 parent 6678b1e commit 4ab6d53

1 file changed

Lines changed: 36 additions & 81 deletions

File tree

.deepreview

Lines changed: 36 additions & 81 deletions
Original file line numberDiff line numberDiff line change
@@ -119,96 +119,47 @@ requirements_traceability:
119119
instructions: |
120120
Review the changed files for requirements traceability.
121121

122-
This project keeps formal requirements in `doc/specs/` organized by domain.
123-
Each file follows the naming pattern `{PREFIX}-REQ-NNN-<topic>.md` where
124-
PREFIX is one of: DW-REQ, JOBS-REQ, REVIEW-REQ, LA-REQ, PLUG-REQ.
125-
Requirements are individually numbered (e.g. JOBS-REQ-004.1). Requirements
126-
must be validated by either automated tests OR DeepWork review rules.
122+
Requirements live in `doc/specs/` as `{PREFIX}-REQ-NNN-<topic>.md`
123+
(prefixes: DW-REQ, JOBS-REQ, REVIEW-REQ, LA-REQ, PLUG-REQ), with
124+
individually numbered items (e.g. JOBS-REQ-004.1). Each requirement
125+
must be validated by automated tests, DeepSchemas, or `.deepreview` rules.
127126

128127
## Choosing the right validation mechanism
129128

130-
Choosing the right mechanism is critical. The wrong choice creates
131-
false confidence (a passing test that doesn't actually verify anything)
132-
or wastes reviewer judgment on something a machine can check exactly.
133-
134-
**Use anonymous DeepSchemas** (`.deepschema.<filename>.yml`) when
135-
requirements target a specific file — whether structural or semantic:
136-
- "This config file MUST include a timeout field" — structural check
137-
for one file (use `json_schema_path` or `verification_bash_command`
138-
for exact verification)
139-
- "The learn workflow MUST accept X and Y step arguments" — the
140-
requirement governs a specific YAML file's content
141-
- "Skill MUST instruct the agent to do X" — judgment-based check
142-
of prose in one specific file
143-
- "The error message MUST include a suggestion for how to fix the
144-
problem" — governs a specific source file's behavior
145-
146-
Anonymous DeepSchemas provide both write-time validation and review-time
147-
checks, and they keep the requirement co-located with the file it governs.
148-
**Prefer them over both tests and `.deepreview` rules whenever the
149-
requirement targets a specific file** rather than a class of files.
150-
DeepSchemas can enforce structural requirements via `json_schema_path`
151-
or `verification_bash_command` just as precisely as a test, while also
152-
supporting judgment-based requirements in the same schema.
153-
154-
**Use automated tests** (`tests/`) when the requirement specifies a
155-
concrete, machine-verifiable fact that spans multiple files or is not
156-
tied to a single file's content:
157-
- File A is byte-identical to file B
158-
- A Python function returns the correct value for given inputs
159-
- A CLI command produces expected output
160-
- A data structure assembled from multiple sources has a required shape
161-
162-
Tests reference requirement IDs via docstrings and traceability comments.
163-
164-
**Use `.deepreview` rules** when evaluating the requirement requires
165-
judgment AND applies broadly across many files of a type:
166-
- "All prompts MUST use the terms X, Y, and Z" — a general standard
167-
that applies to every file matching a glob pattern
168-
- "Code MUST follow pattern Y" — does the implementation match the
169-
spirit of the pattern across multiple files?
170-
- "Documentation MUST stay in sync with code" — are the descriptions
171-
still accurate after changes?
172-
173-
Both `.deepreview` rules and DeepSchemas reference requirement IDs in
174-
their `description`, `instructions`, or `requirements` fields.
129+
Pick the mechanism that matches the requirement type. The wrong choice
130+
creates false confidence or wastes reviewer judgment.
131+
132+
**Anonymous DeepSchemas** (`.deepschema.<filename>.yml`): when the
133+
requirement targets a specific file (structural or semantic). Use
134+
`json_schema_path` / `verification_bash_command` for exact checks,
135+
or judgment-based criteria for prose. Prefer DeepSchemas over tests
136+
and `.deepreview` rules for single-file requirements.
137+
138+
**Automated tests** (`tests/`): for concrete, machine-verifiable facts
139+
spanning multiple files (function return values, CLI output, cross-file
140+
structure). Tests reference requirement IDs via docstrings/comments.
141+
142+
**`.deepreview` rules**: when evaluation requires judgment AND applies
143+
broadly across many files of a type (coding standards, documentation
144+
accuracy, prompt conventions). Rules and DeepSchemas reference
145+
requirement IDs in `description`, `instructions`, or `requirements`.
175146

176147
## Anti-patterns to flag
177148

178-
**Fragile keyword tests for judgment-based requirements.** A test that
179-
checks `"reuse" in content.lower()` to validate "MUST instruct the
180-
agent to reuse existing rules" is not deterministic verification — it
181-
is a keyword search pretending to be one. The word "reuse" could appear
182-
in an unrelated sentence, be negated ("do NOT reuse"), or be absent
183-
while the instruction clearly conveys reuse through other wording.
184-
These requirements need a review rule that can read and evaluate the
185-
instruction's meaning. Other examples of this anti-pattern:
186-
- `"parallel" in content` for "MUST launch tasks in parallel"
187-
- `"again" in content or "repeat" in content` for "MUST re-run after changes"
188-
- `"without asking" in content` for "MUST automatically apply obvious fixes"
189-
190-
**Review rules for machine-verifiable requirements.** A review rule
191-
that asks a reviewer "check whether the config file contains
192-
`--platform claude`" is wasting reviewer judgment on something
193-
`assert "--platform" in args` can verify exactly. If the requirement
194-
specifies a concrete value, path, or structure, use a test.
195-
196-
See doc/specs/validating_requirements_with_rules.md for more information.
149+
- **Fragile keyword tests for judgment requirements**: e.g.
150+
`"parallel" in content` for "MUST launch tasks in parallel" — use
151+
a review rule instead. See doc/specs/validating_requirements_with_rules.md.
152+
- **Review rules for machine-verifiable requirements**: e.g. asking a
153+
reviewer to check for a specific flag — use a test instead.
197154

198155
## Review checklist
199156

200-
1. Check that any new or changed end-user functionality has a
201-
corresponding requirement in `doc/specs/`.
202-
2. Check that every requirement touched by this change has at least
203-
one automated test OR at least one `.deepreview` rule validating
204-
it. **Verify the mechanism matches the requirement type** — flag
205-
keyword-search tests used for judgment requirements, and flag
206-
review rules used for machine-verifiable requirements.
207-
3. Flag any test modifications where the underlying requirement did
208-
not also change.
209-
4. For rule-validated requirements, verify the `.deepreview` rule's
210-
description or instructions reference the requirement ID and that
211-
the rule's scope covers the requirement's intent.
157+
1. New/changed end-user functionality has a requirement in `doc/specs/`.
158+
2. Every touched requirement has a test, DeepSchema, or `.deepreview`
159+
rule. Verify the mechanism matches the requirement type.
160+
3. Flag test modifications where the underlying requirement didn't change.
161+
4. For rule-validated requirements, verify the rule references the
162+
requirement ID and its scope covers the requirement's intent.
212163

213164
Produce a structured review with Coverage Gaps, Test Stability
214165
Violations, and a Summary with PASS/FAIL verdicts.
@@ -370,6 +321,10 @@ shell_code_review:
370321
section markers) still accurate after the changes? Flag any comments
371322
that describe behavior that no longer matches the code.
372323

324+
Output Format:
325+
- PASS: No issues found.
326+
- FAIL: List each issue with file, line, severity (high/medium/low), and a concise description.
327+
373328
agents_md_claude_md_symlink:
374329
description: "Ensure every AGENTS.md file has a sibling CLAUDE.md symlink pointing to it, because Claude Code reads CLAUDE.md but ignores AGENTS.md."
375330
match:

0 commit comments

Comments
 (0)