Skip to content

Add unit-test coverage for run.py, batch.py, and doctor.py #66

Description

@reacher-z

Gap

Three core modules have zero unit tests:

file LOC coverage
`src/clawbench/run.py` 1009 none
`src/clawbench/batch.py` 628 none
`src/clawbench/doctor.py` 194 none

`tests/` currently ships `test_cli.py`, `test_engine.py`, `test_image.py`, `test_paths.py` — but not these three, which together are ~1.8k lines of the hottest code paths.

Concrete bugs this would catch

  • run.py: `_load_runtime_env` precedence (env → ./.env → user config → wheel default). A precedence regression silently leaks or hides keys. One test per source wins.
  • run.py: result-JSON serialization for the HF upload path — malformed JSON breaks `hf_upload` only at the end of a long run.
  • batch.py: case-range parsing (`--case-range 1-50`), glob filtering for `--models`/`--cases`, `--all-models` + `--models` precedence. Easy off-by-one territory.
  • batch.py: concurrency respecting `--stagger-delay` and `--max-concurrent` under a mocked subprocess fan-out.
  • doctor.py: each `check_*` returns correct status under its known-good and known-bad fixtures. Catches the classic "doctor says all green while models.yaml is actually broken" footgun.

Scope

  • 6–8 tests for run.py (focus on `_load_runtime_env`, arg parsing, result parsing)
  • 5–7 tests for batch.py (case-range, glob filtering, dry-run matrix)
  • 5–7 tests for doctor.py (one per check, with mocked engine + filesystem fixtures)
  • No container runs in CI — everything at the subprocess/filesystem mock boundary

Priority

Medium. These modules are user-facing (`clawbench run`, `clawbench batch`, `clawbench doctor`) and the CI today would let a regression ship.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions