Commit 293c981
committed
feat(harness): add null baseline harness — the benchmark floor (closes #245)
A do-nothing agent that connects to nothing and takes no browser action, so the
run records 0 actions/requests and the interceptor never fires. This establishes
the benchmark FLOOR: a null agent must score ~0, proving ClawBench is not
luck-passable and that the HTTP interceptor has no false positives without
genuine agent activity.
- runtime/harnesses/null/{Dockerfile.null, setup-null.sh, run-null.sh,
usage-emitter.py} (FROM clawbench-base; no model call, so no key needed)
- registered in harnesses.yaml (name quoted to avoid the YAML null keyword) +
test_harness_registry EXPECTED_HARNESSES/AGENT_MESSAGE_SOURCES
- the interceptor false-positive rate over a null run is reported by the new
clawbench-analyze tool (#159/#255).
Verified: registry test passes (null registered); clawbench-null builds from
clawbench-base; composition check confirms the scripts are present and run-null
takes no action. Full suite green; ruff clean.1 parent ca571a0 commit 293c981
6 files changed
Lines changed: 62 additions & 0 deletions
File tree
- src/clawbench/runtime/harnesses
- null
- tests
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
112 | 112 | | |
113 | 113 | | |
114 | 114 | | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
27 | 27 | | |
28 | 28 | | |
29 | 29 | | |
| 30 | + | |
30 | 31 | | |
31 | 32 | | |
32 | 33 | | |
| |||
82 | 83 | | |
83 | 84 | | |
84 | 85 | | |
| 86 | + | |
85 | 87 | | |
86 | 88 | | |
87 | 89 | | |
| |||
0 commit comments