Skip to content

Commit b8c1d9f

Browse files
ci: run the full make test target so the Kaggle suite executes in CI
PR #66 added the skills/72-kaggle-research runtime suite to `make test` and widened the pre-commit `files:` triggers to include that directory, but left both the CI step and the pre-commit `entry:` as a bare `unittest discover -s tests`. The 50 new tests therefore never ran in CI, and editing Kaggle code fired a hook that did not cover it. Point both at `make test` (still stdlib-only, no pip) and add assertions in tests/test_repo_tools.py so the gap cannot reopen. Verified: make check-fast green (268 + 50 tests); the Kaggle suite also passes standalone under CPython 3.9.23, matching the CI matrix floor. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
1 parent 2f457dd commit b8c1d9f

3 files changed

Lines changed: 17 additions & 2 deletions

File tree

.github/workflows/quality-evals.yml

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -42,7 +42,10 @@ jobs:
4242
--expect-categories causal-identification,reproducibility,citation-hygiene,runtime-safety,research-integrity,writing-compliance,writing-style
4343
4444
- name: Unit tests (stdlib unittest)
45-
run: python3 -m unittest discover -s tests -p "test_*.py"
45+
# `make test` rather than a bare `unittest discover -s tests`: the
46+
# Makefile target also runs the skills/72-kaggle-research runtime
47+
# suite, which would otherwise never execute in CI. Still stdlib-only.
48+
run: make test
4649

4750
- name: Benchmark (reference pipeline + strict grading)
4851
run: |

.pre-commit-config.yaml

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,11 @@ repos:
1515
files: ^eval-harness/
1616
- id: aers-unit-tests
1717
name: AERS unit tests
18-
entry: python3 -m unittest discover -s tests -p "test_*.py"
18+
# `make test` (not a bare `unittest discover -s tests`) so the hook
19+
# covers every suite the Makefile knows about — including the
20+
# skills/72-kaggle-research runtime tests that this hook's `files:`
21+
# pattern already triggers on.
22+
entry: make test
1923
language: system
2024
pass_filenames: false
2125
files: ^(tests/|eval-harness/|benchmark/|scripts/|skills/72-kaggle-research/)

tests/test_repo_tools.py

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -309,6 +309,10 @@ def test_quality_workflow_uses_non_writing_gates(self):
309309
self.assertIn("python3 benchmark/reference_pipeline.py --check", text)
310310
self.assertNotIn("python3 benchmark/reference_pipeline.py\n", text)
311311
self.assertIn("--no-write", text)
312+
# CI must invoke the Makefile target, not a bare `discover -s tests`:
313+
# the latter silently skips every suite outside tests/.
314+
self.assertIn("run: make test", text)
315+
self.assertNotIn("run: python3 -m unittest discover -s tests", text)
312316

313317
def test_pre_commit_uses_non_writing_benchmark_gate(self):
314318
text = (ROOT / ".pre-commit-config.yaml").read_text(encoding="utf-8")
@@ -320,6 +324,10 @@ def test_pre_commit_uses_non_writing_benchmark_gate(self):
320324
self.assertIn("aers-tracked-file-hygiene", text)
321325
self.assertIn("python3 scripts/check-repo-hygiene.py", text)
322326
self.assertIn("skills/72-kaggle-research/", text)
327+
# Same reason as the CI workflow: the unit-test hook runs `make test`
328+
# so it covers every suite its `files:` pattern can trigger on.
329+
self.assertIn("entry: make test", text)
330+
self.assertNotIn("entry: python3 -m unittest discover -s tests", text)
323331

324332
def test_make_check_includes_python_compatibility_compile(self):
325333
text = (ROOT / "Makefile").read_text(encoding="utf-8")

0 commit comments

Comments
 (0)