You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
docs: require plan-faithfulness test for DP/control-mapping fixes (#386)
PR #385 extended the shadow-price discharge gate to LOAD_SUPPORT with
only a synthetic-input unit test on _apply_period_schedule -- no
existing or new test actually exercises the branch via a real
DP-produced schedule (docs/agents/simulator.md's R == P requirement).
Wire that requirement into implement-issue's TDD step so it's the
primary RED test for DP/intent/control-mapping fixes, not an optional
supplement, with a Step 6 quality-gate backstop in case it's missed.
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
1.`./scripts/quality-check.sh` (fast suite) — if it fails, fix and re-run,
88
122
do not proceed with failures.
89
123
2.`.venv/bin/pytest -m slow` (slow suite) — same failure handling.
90
-
3. Invoke the `code-review` skill on the diff.
91
-
4. Report back: pass/fail on both suites, and any CONFIRMED code-review
124
+
3. If the diff touches the DP, intent classification, or control/rate
125
+
mapping (the Step 5 table): confirm the diff's new/changed tests include
126
+
a `run_scenario_realized` / `verify_plan_faithfulness` call, not only a
127
+
unit test on the changed function with hand-built arguments. If missing,
128
+
this is a required-before-continuing gap, not a nice-to-have — report it
129
+
as a blocking finding alongside the suite results, same severity as a
130
+
failing test.
131
+
4. Invoke the `code-review` skill on the diff.
132
+
5. Report back: pass/fail on both suites, whether check 3 passed, and any
133
+
CONFIRMED code-review
92
134
findings verbatim (everything else goes to `TODO.md`).
93
135
94
136
Do not poll — you'll be notified on completion. This is a hard session
@@ -219,6 +261,8 @@ flow above, which stops at draft-PR-open per the Step 10 constraints.
219
261
| "the user is in a hurry, just open the PR" | Time pressure from the user is not permission to skip Step 8 — it's the reason to say so explicitly and give a real ETA instead. |
220
262
| "I'll just watch the background agent run" | Defeats the point — the whole reason it's backgrounded is so the session isn't held open through the slow suite. Let the notification bring you back. |
221
263
| "the fix is small, docs don't need touching" | Small fixes are exactly what silently invalidates a one-line doc claim (a removed threshold, a renamed formula). Grep the two design docs before opening the PR, every time. |
264
+
| "a unit test on the changed function is enough" | Not for DP/intent/control-mapping changes — a synthetic-input unit test can pass while the new branch is unreachable by any real optimizer-derived scenario. `docs/agents/simulator.md` requires `R == P` for exactly this class of change. |
265
+
| "the existing suite still passes, so nothing broke" | Passing unchanged means the new code path may simply be untested, not unbroken — check whether any existing fixture actually reaches the new branch before treating a green suite as coverage. |
222
266
223
267
## Red Flags — Stop and Go Back
224
268
@@ -233,6 +277,10 @@ flow above, which stops at draft-PR-open per the Step 10 constraints.
233
277
dispatching the Step 6 background agent.
234
278
- About to open the PR without checking whether the fix invalidates a claim
235
279
in `docs/agents/bess-knowledge.md` or `docs/SOFTWARE_DESIGN.md`.
280
+
- About to write only a synthetic-input unit test for a DP/intent/control-
281
+
mapping change instead of a plan-faithfulness (`R == P`) scenario test.
282
+
- About to write a repro test from hand-built data when a user debug log/
283
+
bundle is available and `from_debug_log.py` could build it from real data.
0 commit comments