Skip to content

Commit cdd085a

Browse files
authored
ci: run the unit tests, and align the ruff pin with CI (#84)
## Unit tests were never executed (#73) `lint.yml` delegates to the skill-repo-skill reusable, whose steps are skill validation, manifest sync, markdownlint, yamllint, actionlint, shellcheck, ruff and checkpoint schemas. Ruff parses the test files. Nothing runs them. Both suites — `_lib/test_formatting.py` and `scripts/test_matrix_doctor.py`, 54 tests between them — have been passing locally and proving nothing about any pull request. The job runs each test file directly instead of using discovery, and that is deliberate: ``` $ python3 -m unittest discover -s skills -p 'test_*.py' Ran 0 tests in 0.000s NO TESTS RAN $ echo $? 0 ``` The skill directories are not importable packages (hyphens in the names), so discovery rooted above `scripts/` collects nothing and exits 0 while doing it. A discovery-based job would have gone green without running a test — the same failure mode as the bug it is meant to catch. The job therefore also fails a file that reports `Ran 0 tests`. Verified locally against both suites: ``` --- skills/matrix-communication/scripts Ran 34 tests OK --- skills/matrix-communication/scripts/_lib Ran 20 tests OK ``` Stdlib only, so the job needs no dependency install. ## ruff pin drift (#75) `.pre-commit-config.yaml` pinned `ruff-pre-commit` at v0.15.14 while the reusable CI workflow runs `uvx --no-build ruff@0.16.0`. `ISC004` does not exist in the older one, so a commit passed the hook and failed CI — that happened on #74 in this repo. Bumped to v0.16.0, with a comment naming the coupling. This aligns the two today; it does not enforce the pairing, because the CI version lives in `netresearch/skill-repo-skill` behind its own `# renovate:` comment and Renovate bumps each side independently. #75 keeps the structural question open. ## Not addressed The repo header states a CI/Hook Parity Principle — "every hook below ALSO runs in CI". The reverse now holds too, in the sense that this job has no hook counterpart: tests do not run on commit. That is deliberate (a pre-commit hook running the suites would slow every commit for a repo whose tests are this cheap to run in CI), but it is worth naming rather than leaving as an inconsistency someone finds later.
2 parents 037e4c6 + d1d582f commit cdd085a

2 files changed

Lines changed: 74 additions & 1 deletion

File tree

.github/workflows/tests.yml

Lines changed: 69 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,69 @@
1+
name: Unit Tests
2+
3+
# Nothing else in CI executes the test files. The lint workflow parses them with
4+
# ruff, which proves they are syntactically valid Python and nothing more - a
5+
# change that breaks the code under test ships green without this job.
6+
#
7+
# The suites are stdlib `unittest` and are run by path, not by discovery:
8+
# `unittest discover` rooted above the scripts directory finds nothing (the skill
9+
# directories are not importable packages - hyphens in the names) and still exits
10+
# 0, so a discovery-based job would report success while running no tests. Each
11+
# file is executed directly, the way its own docstring documents, and the run
12+
# fails if a file reports zero tests.
13+
14+
on:
15+
push:
16+
branches: [main]
17+
pull_request:
18+
19+
permissions: {}
20+
21+
jobs:
22+
unittest:
23+
name: Unit Tests
24+
runs-on: ubuntu-latest
25+
permissions:
26+
contents: read
27+
steps:
28+
- name: Harden Runner
29+
uses: step-security/harden-runner@b09bb98e06d4d774595224525879c09bc6e98c40 # v2.20.1
30+
with:
31+
egress-policy: audit
32+
33+
- name: Checkout repository
34+
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
35+
with:
36+
# The job only reads the tree. Leaving the token in .git/config would
37+
# hand it to anything the tests execute.
38+
persist-credentials: false
39+
40+
- name: Run test suites
41+
run: |
42+
set -uo pipefail
43+
mapfile -t files < <(find skills -type f -name 'test_*.py' | sort)
44+
45+
if [[ ${#files[@]} -eq 0 ]]; then
46+
echo "::error::No test files found under skills/"
47+
exit 1
48+
fi
49+
50+
echo "Found ${#files[@]} test file(s)"
51+
status=0
52+
53+
for f in "${files[@]}"; do
54+
echo "::group::$f"
55+
output=$(python3 "$f" -v 2>&1)
56+
rc=$?
57+
echo "$output"
58+
echo "::endgroup::"
59+
60+
if [[ $rc -ne 0 ]]; then
61+
echo "::error file=$f::test suite failed"
62+
status=1
63+
elif grep -q "^Ran 0 tests" <<<"$output"; then
64+
echo "::error file=$f::collected no tests"
65+
status=1
66+
fi
67+
done
68+
69+
exit $status

.pre-commit-config.yaml

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -49,8 +49,12 @@ repos:
4949
hooks:
5050
- id: actionlint
5151

52+
# Keep this rev on the same ruff version the reusable validate.yml runs
53+
# (`uvx --no-build ruff@X.Y.Z`). The two pins are in different repositories and
54+
# nothing enforces the pairing, so a hook older than CI passes locally and
55+
# fails in CI on rules the older ruff does not have yet. See issue #75.
5256
- repo: https://github.com/astral-sh/ruff-pre-commit
53-
rev: v0.15.14
57+
rev: v0.16.0
5458
hooks:
5559
- id: ruff
5660
- id: ruff-format

0 commit comments

Comments
 (0)