Skip to content

Commit 087b5e3

Browse files
mraadclaude
andcommitted
docs(testing): record the setup-race timeout rule
Add a second rule to Review-Proven Test Standards: a wall-clock timeout that gates test setup is a race budget, not a tuning knob. Size it against measured worst-case latency on a loaded machine rather than the idle case, record the measurement at the call site so it is not trimmed back toward the latency it exists to clear, and make a lost setup race say so rather than surfacing as an opaque parse error. Generalizes the flake fixed in a41fcb3, where the escaped-writer probe in bounded_command.rs budgeted 300ms for a descendant whose spawn-to-ready latency measured ~30ms idle but p95 233ms / max 498ms under a loaded lane. Also correct the section intro, which read "this is the test-quality rule" in the singular and attributed the section wholesale to the 25-PR review mining. This rule came from a fixed flake, not a review thread, so the intro now separates the mined rule from later additions instead of extending that provenance to them. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GecRn4WhoM35gz9fRY7Y1A Signed-off-by: mraad <mraad23@gmail.com>
1 parent a41fcb3 commit 087b5e3

1 file changed

Lines changed: 17 additions & 3 deletions

File tree

TESTING.md

Lines changed: 17 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -18,9 +18,9 @@ cargo test -p buzz-test-client -- --ignored
1818

1919
### Review-Proven Test Standards
2020

21-
Mined from the last 25 PRs' review threads (see Review-Proven Rules in
22-
[AGENTS.md](AGENTS.md)); this is the test-quality rule reviewers litigated
23-
most:
21+
The first rule below is mined from the last 25 PRs' review threads (see
22+
Review-Proven Rules in [AGENTS.md](AGENTS.md))the test-quality rule
23+
reviewers litigated most. The rest come from defects fixed since.
2424

2525
**Regression tests must bind the production seam and be falsifiable.**
2626
A guard whose removal doesn't fail any test protects nothing — mutations
@@ -31,6 +31,20 @@ combination space (PR #6807). Scope Playwright locators — unscoped
3131
`getByText` in a required smoke test is a strict-mode flake (PR #6980).
3232
(PRs #6807, #6980, #6996, #7013)
3333

34+
**A wall-clock timeout that gates test setup is a race budget, not a tuning
35+
knob.** When a fixture has to win a race against a deadline — spawning a
36+
helper, escaping a process group, writing a handshake file — that deadline
37+
bounds the *setup*, not just the behavior under test. Size it against measured
38+
worst-case latency on a loaded machine, never the idle case. The escaped-writer
39+
probe in `desktop/src-tauri/src/managed_agents/discovery/bounded_command.rs`
40+
budgeted 300ms for a descendant whose spawn-to-ready latency measured ~30ms
41+
idle but p95 233ms / max 498ms under a loaded lane, so it flaked wherever
42+
contention was CI-shaped. Record the measurement at the call site so the budget
43+
is not later trimmed back toward the latency it exists to clear. When setup does
44+
lose the race, fail with a message saying so — a lost setup race that surfaces
45+
as an opaque parse error or an assertion on missing state reads as a product
46+
bug and costs the debugging time twice.
47+
3448
---
3549

3650
## Live Local Relay

0 commit comments

Comments
 (0)