CI: run the suite in parallel, trace coverage on 3.12 only - #314
Conversation
lint + test spent ~600s per Python version in pytest, serially, under coverage tracing. --durations showed no handful of slow tests — the slowest is ~4s — so the time is the long tail. - all legs run pytest-xdist `-n auto`; nothing is deselected - 3.12 alone collects coverage, with COVERAGE_CORE=sysmon, and keeps --cov-fail-under=85; 3.10/3.11 run the full suite without tracing - dev.py test/cover, the pre-push hook and CONTRIBUTING match - ignore per-worker .coverage.* files left by an interrupted run Measured locally on py3.12: serial 5m27s; -n auto 1m31s; -n auto with sysmon coverage 1m33s at 88.96%. Same failure set in all three (the unrelated #293 home-pointer failures). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
🟡 Changes recommended
Add workflow contract assertions and update the stale coverage documentation.
Get a fresh assessment by requesting another Copilot review.
Pull request overview
This PR parallelizes local and CI test execution while keeping coverage enforcement on Python 3.12.
Changes:
- Adds
pytest-xdistwith-n auto. - Uses
sysmoncoverage and an 85% floor on Python 3.12. - Updates contributor and pre-commit workflows.
- Ignores per-worker coverage artifacts.
File summaries
| File | Summary |
|---|---|
dev.py |
Enables parallel test execution. |
CONTRIBUTING.md |
Documents parallel test commands. |
.pre-commit-config.yaml |
Parallelizes pre-push tests. |
.gitignore |
Ignores per-worker coverage files. |
.github/workflows/ci.yml |
Splits parallel testing and 3.12 coverage enforcement. |
Review details
Suppressed comments (1)
.github/workflows/ci.yml:77
- Adding
--cov-fail-under=85here makes the repository's stated coverage contract stale:CLAUDE.md:60-62andpyproject.toml:27-29still say that no floor exists and that it will be added later. Please update those statements to describe the new 3.12-only CLI floor (without moving it intopyproject.toml, which would change local behavior).
--cov-fail-under=85
- Files reviewed: 4/5 changed files
- Comments generated: 1
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| - name: pytest --cov | ||
| if: matrix.python-version == '3.12' | ||
| env: | ||
| COVERAGE_CORE: sysmon |
There was a problem hiding this comment.
Fixed in de753ce. Added tests/e2e/test_suite_integrity.py::test_exactly_one_leg_enforces_the_coverage_floor: exactly one lint-and-test step may carry --cov-fail-under, and its if: must name a version the matrix actually runs. Mutation-checked — it fails on a flipped condition, on a version not in the matrix, and on the floor appearing in both steps.
I deliberately didn't assert -n auto or COVERAGE_CORE=sysmon: dropping either makes CI slower, not wrong, and a red build over a speed flag is a guard nobody wants. The floor is the correctness contract, so that's what's pinned.
…oor docs Gating the floor to py3.12 makes it one `if:` edit away from gating no leg at all while every leg stays green. The new suite-integrity test fails if zero or several steps carry --cov-fail-under, or if the step's condition names a version the matrix doesn't run. Mutation-checked: it fails on a flipped condition, an absent version, and a doubled floor. CLAUDE.md and pyproject.toml still said no floor existed; they now point at the CI flag as the single source. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Re Copilot's suppressed comment on |
Closes #296.
lint + testspent ~600s per Python version in the test step, one test at a time, under coverage tracing, on all three legs.What the measurement said
pytest tests/ --durations=40, serial, no coverage, py3.12, 12-core laptop:So there was no handful of slow tests to fix — the time is the long tail, and the lever is running it in parallel.
What this changes
--cov+ floor-n auto, no coverage--cov+ floor-n auto,--cov+--cov-fail-under=85,COVERAGE_CORE=sysmon[tool.coverage] branchsetting, sosysmonruns without falling back.tests/e2e/test_suite_integrity.py(which reads this workflow) still passes.dev.py test/dev.py cover, the pre-push hook andCONTRIBUTING.mdget the same-n auto, so local runs match CI.Parallel safety
Measured locally, same machine, same commit:
-n auto-n auto+ sysmon coverage + floor (CI's 3.12 leg)With
sysmon, coverage added about 2 seconds to the parallel run. The floor is left at 85 — ratcheting it toward 89 is a separate change.The 39 failures are #293 (a
~/.config/agami/pathpointer on this machine leaking a real artifacts dir into the suite) — the same ids with and without-n auto, and CI has no such pointer. Before running it I also scanned the suite for fixed ports, shared/tmppaths, and session-scoped state that two workers could collide on, and found none that a worker process shares.The extra warnings under
-n auto(29 vs 7) are the same three deprecation warnings, reported once per worker.On CI (this PR's first run, whole job including setup)
Guarding the split
tests/e2e/test_suite_integrity.py::test_exactly_one_leg_enforces_the_coverage_floorfails if no step, or more than one, carries--cov-fail-under, or if that step's condition names a version the matrix doesn't run — so the floor can't be switched off by anif:edit while every leg stays green. Mutation-checked against a flipped condition, an absent version, and a doubled floor.CLAUDE.mdandpyproject.tomlno longer say the floor doesn't exist yet.🤖 Generated with Claude Code