docs(changelog): record the upstream-attribution work #202
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| name: Quality (evals + benchmark + tests) | |
| # Complements validate-catalog.yml (which checks catalog freshness). This job | |
| # enforces the eval/benchmark/test layer. All steps are stdlib-only; no pip. | |
| on: | |
| push: | |
| branches: [main] | |
| pull_request: | |
| workflow_dispatch: | |
| permissions: | |
| contents: read | |
| jobs: | |
| quality: | |
| runs-on: ubuntu-latest | |
| timeout-minutes: 10 | |
| strategy: | |
| fail-fast: false | |
| matrix: | |
| python-version: ["3.9", "3.12"] | |
| steps: | |
| - uses: actions/checkout@v7 | |
| with: | |
| persist-credentials: false | |
| submodules: recursive | |
| - uses: actions/setup-python@v7 | |
| with: | |
| python-version: ${{ matrix.python-version }} | |
| - name: Catalog enrichment freshness | |
| run: python3 scripts/build-catalog-enrich.py --check | |
| - name: Python tooling compatibility | |
| run: make python-compat | |
| - name: Lint eval-harness scenarios | |
| run: >- | |
| python3 eval-harness/run_evals.py | |
| --min-scenarios 24 | |
| --min-auto-checks 116 | |
| --expect-categories causal-identification,reproducibility,citation-hygiene,runtime-safety,research-integrity,writing-compliance,writing-style | |
| - name: Unit tests (stdlib unittest) | |
| # `make test` rather than a bare `unittest discover -s tests`: the | |
| # Makefile target also runs the skills/72-kaggle-research runtime | |
| # suite, which would otherwise never execute in CI. Still stdlib-only. | |
| run: make test | |
| - name: Benchmark (reference pipeline + strict grading) | |
| run: | | |
| python3 benchmark/check_benchmark.py --lint | |
| python3 benchmark/reference_pipeline.py --check | |
| python3 benchmark/check_benchmark.py --strict --fail-on-partial --fail-on-orphan-results | |
| - name: Grade example candidates (smoke) | |
| run: >- | |
| python3 eval-harness/run_evals.py | |
| --grade eval-harness/candidates/_example | |
| --expect-graded 9 | |
| --expect-fail-required statspai-weak-iv | |
| --expect-graded-categories causal-identification,reproducibility,citation-hygiene,runtime-safety,research-integrity | |
| --fail-on-orphans | |
| --fail-on-partial | |
| --no-write |