Gap
Three core modules have zero unit tests:
| file |
LOC |
coverage |
| `src/clawbench/run.py` |
1009 |
none |
| `src/clawbench/batch.py` |
628 |
none |
| `src/clawbench/doctor.py` |
194 |
none |
`tests/` currently ships `test_cli.py`, `test_engine.py`, `test_image.py`, `test_paths.py` — but not these three, which together are ~1.8k lines of the hottest code paths.
Concrete bugs this would catch
- run.py: `_load_runtime_env` precedence (env → ./.env → user config → wheel default). A precedence regression silently leaks or hides keys. One test per source wins.
- run.py: result-JSON serialization for the HF upload path — malformed JSON breaks `hf_upload` only at the end of a long run.
- batch.py: case-range parsing (`--case-range 1-50`), glob filtering for `--models`/`--cases`, `--all-models` + `--models` precedence. Easy off-by-one territory.
- batch.py: concurrency respecting `--stagger-delay` and `--max-concurrent` under a mocked subprocess fan-out.
- doctor.py: each `check_*` returns correct status under its known-good and known-bad fixtures. Catches the classic "doctor says all green while models.yaml is actually broken" footgun.
Scope
- 6–8 tests for run.py (focus on `_load_runtime_env`, arg parsing, result parsing)
- 5–7 tests for batch.py (case-range, glob filtering, dry-run matrix)
- 5–7 tests for doctor.py (one per check, with mocked engine + filesystem fixtures)
- No container runs in CI — everything at the subprocess/filesystem mock boundary
Priority
Medium. These modules are user-facing (`clawbench run`, `clawbench batch`, `clawbench doctor`) and the CI today would let a regression ship.
Gap
Three core modules have zero unit tests:
`tests/` currently ships `test_cli.py`, `test_engine.py`, `test_image.py`, `test_paths.py` — but not these three, which together are ~1.8k lines of the hottest code paths.
Concrete bugs this would catch
Scope
Priority
Medium. These modules are user-facing (`clawbench run`, `clawbench batch`, `clawbench doctor`) and the CI today would let a regression ship.