Skip to content

Commit 2174e03

Browse files
nhortonclaude
andcommitted
docs: add DW-REQ-012 requirement spec and test traceability
- Create DW-REQ-012-tool-requirements.md with 12 sub-requirements covering policy format, discovery, inheritance, matching, evaluation, check flow, appeal, caching, hook, sidecar, multi-instance, and startup - Add PLUG-REQ-001.15 for the PreToolUse hook registration - Add requirement ID references to all test module docstrings - Add THIS TEST VALIDATES traceability comments to critical tests Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1 parent feb32d5 commit 2174e03

9 files changed

Lines changed: 127 additions & 7 deletions

File tree

Lines changed: 104 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,104 @@
1+
# DW-REQ-012: Tool Requirements Policy Enforcement
2+
3+
## Overview
4+
5+
The tool requirements system enforces RFC 2119-style policies on AI agent tool calls. Users define rules in `.deepwork/tool_requirements/*.yml` files. A PreToolUse hook checks these rules via an HTTP sidecar server using LLM-based semantic evaluation. Failed checks can be appealed via an MCP tool. Approved calls are cached with a TTL.
6+
7+
## DW-REQ-012.1: Policy File Format
8+
9+
1. Policy files MUST be YAML files located in `.deepwork/tool_requirements/` with a `.yml` extension.
10+
2. Each policy file MUST be validated against the `tool_requirements_schema.json` JSON Schema.
11+
3. The `tools` field MUST be a non-empty array of normalized tool names (e.g., `shell`, `write_file`, `edit_file`) or MCP tool names (e.g., `mcp__server__tool`).
12+
4. The `requirements` field MUST be a mapping of requirement identifiers to objects containing a `rule` string (RFC 2119 statement) and an optional `no_exception` boolean (default: `false`).
13+
5. The `match` field MAY be a mapping of tool_input parameter names to regex patterns for parameter-level filtering.
14+
6. The `extends` field MAY be an array of policy file stems for inheritance.
15+
7. The `summary` field MAY be a human-readable description of the policy.
16+
17+
## DW-REQ-012.2: Policy Discovery
18+
19+
1. The system MUST scan `.deepwork/tool_requirements/` for `*.yml` files (single-directory, no tree walk).
20+
2. Files that fail to parse MUST be skipped with a warning logged — they MUST NOT prevent other policies from loading.
21+
3. If the `.deepwork/tool_requirements/` directory does not exist, the system MUST return an empty policy list without error.
22+
23+
## DW-REQ-012.3: Policy Inheritance
24+
25+
1. When a policy lists `extends`, the system MUST merge parent requirements into the child.
26+
2. Child requirements MUST override parent requirements with the same key.
27+
3. Unknown parent names MUST be logged as warnings and skipped — they MUST NOT cause errors.
28+
4. Circular inheritance MUST be detected and MUST NOT cause infinite loops.
29+
5. Diamond inheritance (two parents sharing a common ancestor) MUST be handled correctly — the common ancestor's requirements MUST be included once.
30+
31+
## DW-REQ-012.4: Policy Matching
32+
33+
1. A policy MUST match a tool call if the tool's normalized name is in the policy's `tools` list.
34+
2. If the policy has a `match` dict, the policy MUST match only when at least one parameter regex matches a value in `tool_input` (via `re.search`).
35+
3. If the policy has no `match` dict, it MUST match all calls to the listed tools.
36+
4. Multiple policies MAY match a single tool call; all matched requirements MUST be merged.
37+
5. If the same requirement key appears in multiple matched policies, the first occurrence MUST win.
38+
6. Invalid regex patterns in `match` MUST be skipped without error.
39+
40+
## DW-REQ-012.5: Requirement Evaluation
41+
42+
1. Requirements MUST be evaluated by an LLM evaluator (Haiku by default) that considers RFC 2119 keywords.
43+
2. `MUST`/`MUST NOT` violations MUST always result in failure.
44+
3. `SHOULD`/`SHOULD NOT` violations MUST result in failure only when the violation is clear and easily avoidable.
45+
4. `MAY` requirements MUST always pass.
46+
5. The evaluator MUST return a verdict for every requirement — requirements not evaluated MUST fail closed.
47+
6. The evaluator MUST be encapsulated behind an abstract interface to allow implementation swapping.
48+
7. Large `tool_input` values MUST be truncated to avoid exceeding LLM token limits.
49+
50+
## DW-REQ-012.6: Check Flow
51+
52+
1. When a tool call is checked, the system MUST first check the cache — if approved, it MUST allow immediately.
53+
2. If no policies match the tool call, it MUST be allowed.
54+
3. If evaluation passes all requirements, the result MUST be cached and the call MUST be allowed.
55+
4. If any requirements fail, the response MUST include ALL failures (not one at a time).
56+
5. Each failure MUST include the requirement ID and an explanation.
57+
6. `no_exception` requirements MUST be labeled as such in the failure message.
58+
7. The failure message MUST include instructions for how to appeal via the `appeal_tool_requirement` MCP tool.
59+
60+
## DW-REQ-012.7: Appeal Mechanism
61+
62+
1. The system MUST provide an `appeal_tool_requirement` MCP tool.
63+
2. The tool MUST accept `tool_name`, `tool_input`, and `policy_justification` (a dict mapping failed check IDs to justification strings).
64+
3. `no_exception` requirements MUST NOT be appealable — appeals for them MUST be rejected immediately.
65+
4. For appealable requirements, the evaluator MUST re-evaluate considering the provided justifications.
66+
5. If the appeal succeeds, the result MUST be cached so the retried tool call passes the hook.
67+
6. If the appeal fails, the response MUST list all still-failing requirements.
68+
69+
## DW-REQ-012.8: Caching
70+
71+
1. Approved tool calls MUST be cached with a 1-hour TTL.
72+
2. The cache key MUST be deterministic, derived from the tool name and tool input.
73+
3. Expired cache entries MUST be evicted on lookup.
74+
4. The cache MUST be in-memory within the sidecar server process.
75+
76+
## DW-REQ-012.9: PreToolUse Hook
77+
78+
1. The hook MUST fire on all PreToolUse events (empty matcher).
79+
2. The hook MUST skip the `appeal_tool_requirement` MCP tool to prevent infinite loops (substring match on raw tool name).
80+
3. If the sidecar is unreachable (port file missing or PID dead), the hook MUST deny with an error message instructing the user to restart the MCP server (fail-closed).
81+
4. If communication with the sidecar fails, the hook MUST deny with an error message (fail-closed).
82+
5. The hook MUST use `hookSpecificOutput.permissionDecision: "deny"` format for Claude Code PreToolUse events.
83+
6. The hook MUST use the cross-platform wrapper system (`run_hook`, `HookInput`, `HookOutput`).
84+
85+
## DW-REQ-012.10: Sidecar HTTP Server
86+
87+
1. The sidecar MUST start as a daemon thread alongside the MCP server when policy files exist.
88+
2. The sidecar MUST bind to `127.0.0.1` on a random port.
89+
3. The sidecar MUST write a port file to `.deepwork/tmp/tool_req_sidecar/<PID>.json` containing `{"pid": <PID>, "port": <PORT>}`.
90+
4. The sidecar MUST provide `POST /check` and `POST /appeal` endpoints.
91+
5. The sidecar MUST clean up its port file and any session mapping files on exit.
92+
6. Session IDs used in filenames MUST be validated against `^[a-zA-Z0-9_-]+$` to prevent path traversal.
93+
94+
## DW-REQ-012.11: Multi-Instance Support
95+
96+
1. When the first MCP tool call arrives with a `session_id`, the server MUST write a session mapping file at `.deepwork/tmp/tool_req_sidecar/session_<SESSION_ID>.json`.
97+
2. The hook MUST look for a session-specific mapping file first, then fall back to scanning PID-keyed port files for live processes.
98+
3. Stale port files (PID no longer alive) MUST be cleaned up during discovery.
99+
100+
## DW-REQ-012.12: Sidecar Startup Gating
101+
102+
1. The sidecar MUST NOT start if no `.deepwork/tool_requirements/` directory exists.
103+
2. The sidecar MUST NOT start if the directory contains no `*.yml` files.
104+
3. If sidecar startup fails, the MCP server MUST continue running — the failure MUST be logged as a warning.

doc/specs/deepwork/cli_plugins/PLUG-REQ-001-claude-code-plugin.md

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -99,6 +99,12 @@ The Claude Code plugin is the primary distribution mechanism for DeepWork on the
9999
3. The skill MUST explain how DeepSchemas automatically generate synthetic review rules.
100100
4. The skill MUST describe workflow quality gates and how `finished_step` triggers reviews on step outputs.
101101

102+
### PLUG-REQ-001.15: Tool Requirements PreToolUse Hook
103+
104+
1. The plugin MUST register a PreToolUse hook in `plugins/claude/hooks/hooks.json` with an empty matcher (matches all tool calls).
105+
2. The hook MUST delegate to `deepwork hook tool_requirements` via a shell wrapper at `plugins/claude/hooks/tool_requirements.sh`.
106+
3. The hook MUST skip the `appeal_tool_requirement` MCP tool to prevent infinite loops (see DW-REQ-012.9.2).
107+
102108
### PLUG-REQ-001.14: Default Reviewer Subagent
103109

104110
1. The plugin MUST ship a default reviewer subagent at `plugins/claude/agents/reviewer.md`.

tests/unit/test_tool_requirements_hook.py

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
"""Tests for the tool requirements PreToolUse hook."""
1+
"""Tests for the tool requirements PreToolUse hook (DW-REQ-012.9, PLUG-REQ-001.15)."""
22

33
from unittest.mock import MagicMock, patch
44

@@ -39,6 +39,8 @@ def test_skips_non_before_tool_events(self) -> None:
3939
result = tool_requirements_hook(hook_input)
4040
assert result.decision == ""
4141

42+
# THIS TEST VALIDATES A HARD REQUIREMENT (DW-REQ-012.9.2).
43+
# YOU MUST NOT MODIFY THIS TEST UNLESS THE REQUIREMENT CHANGES
4244
def test_loop_prevention_skips_all_appeal_prefixes(self) -> None:
4345
from deepwork.hooks.tool_requirements import tool_requirements_hook
4446

@@ -51,6 +53,8 @@ def test_loop_prevention_skips_all_appeal_prefixes(self) -> None:
5153
result = tool_requirements_hook(hook_input)
5254
assert result.decision == "", f"Failed for {prefix}"
5355

56+
# THIS TEST VALIDATES A HARD REQUIREMENT (DW-REQ-012.9.3).
57+
# YOU MUST NOT MODIFY THIS TEST UNLESS THE REQUIREMENT CHANGES
5458
@patch("deepwork.hooks.tool_requirements.discover_sidecar")
5559
def test_fail_closed_when_no_sidecar(self, mock_discover: MagicMock) -> None:
5660
from deepwork.hooks.tool_requirements import tool_requirements_hook

tests/unit/tool_requirements/test_cache.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
"""Tests for tool requirements TTL cache."""
1+
"""Tests for tool requirements TTL cache (DW-REQ-012.8)."""
22

33
import time
44
from unittest.mock import patch

tests/unit/tool_requirements/test_config.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
"""Tests for tool requirements config parsing."""
1+
"""Tests for tool requirements config parsing (DW-REQ-012.1)."""
22

33
from pathlib import Path
44

tests/unit/tool_requirements/test_discovery.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
"""Tests for tool requirements discovery and inheritance."""
1+
"""Tests for tool requirements discovery (DW-REQ-012.2) and inheritance (DW-REQ-012.3)."""
22

33
from pathlib import Path
44

tests/unit/tool_requirements/test_engine.py

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
"""Tests for tool requirements engine."""
1+
"""Tests for tool requirements engine (DW-REQ-012.6, DW-REQ-012.7)."""
22

33
from pathlib import Path
44
from typing import Any
@@ -61,6 +61,8 @@ async def test_no_matching_policies_allows(self, tmp_path: Path) -> None:
6161
result = await engine.check("shell", {"command": "ls"})
6262
assert result.allowed is True
6363

64+
# THIS TEST VALIDATES A HARD REQUIREMENT (DW-REQ-012.6.3, DW-REQ-012.8.1).
65+
# YOU MUST NOT MODIFY THIS TEST UNLESS THE REQUIREMENT CHANGES
6466
async def test_all_pass_allows_and_caches(self, tmp_path: Path) -> None:
6567
project = _setup_project(
6668
tmp_path,
@@ -83,6 +85,8 @@ async def test_all_pass_allows_and_caches(self, tmp_path: Path) -> None:
8385
assert result2.allowed is True
8486
assert evaluator.call_count == 1 # Not called again
8587

88+
# THIS TEST VALIDATES A HARD REQUIREMENT (DW-REQ-012.6.4).
89+
# YOU MUST NOT MODIFY THIS TEST UNLESS THE REQUIREMENT CHANGES
8690
async def test_failure_denies_with_all_errors(self, tmp_path: Path) -> None:
8791
project = _setup_project(
8892
tmp_path,
@@ -163,6 +167,8 @@ async def test_successful_appeal_caches(self, tmp_path: Path) -> None:
163167
check = await engine.check("shell", {"command": "rm file"})
164168
assert check.allowed is True
165169

170+
# THIS TEST VALIDATES A HARD REQUIREMENT (DW-REQ-012.7.3).
171+
# YOU MUST NOT MODIFY THIS TEST UNLESS THE REQUIREMENT CHANGES
166172
async def test_no_exception_blocks_appeal(self, tmp_path: Path) -> None:
167173
project = _setup_project(
168174
tmp_path,

tests/unit/tool_requirements/test_evaluator.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
"""Tests for tool requirements LLM evaluator."""
1+
"""Tests for tool requirements LLM evaluator (DW-REQ-012.5)."""
22

33
import json
44
from unittest.mock import AsyncMock, patch

tests/unit/tool_requirements/test_matcher.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
"""Tests for tool requirements policy matching."""
1+
"""Tests for tool requirements policy matching (DW-REQ-012.4)."""
22

33
from pathlib import Path
44

0 commit comments

Comments
 (0)