docs: quote the gate figures from the merged tree, not the branch alone #284
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| name: CI | |
| on: | |
| push: | |
| branches: [main] | |
| pull_request: | |
| concurrency: | |
| group: ci-${{ github.ref }} | |
| cancel-in-progress: true | |
| jobs: | |
| test: | |
| runs-on: ubuntu-latest | |
| services: | |
| postgres: | |
| image: postgres:16 | |
| env: | |
| POSTGRES_USER: sync | |
| POSTGRES_PASSWORD: sync | |
| POSTGRES_DB: sync | |
| ports: | |
| - 5432:5432 | |
| options: >- | |
| --health-cmd "pg_isready -U sync" | |
| --health-interval 2s | |
| --health-timeout 3s | |
| --health-retries 15 | |
| env: | |
| # docker-compose.yml maps Postgres to 5433 to stay clear of a local server. | |
| # A service container has the port to itself, so the default DSN is overridden | |
| # rather than the mapping reproduced. | |
| SYNC_DSN: postgresql://sync:sync@localhost:5432/sync | |
| steps: | |
| - uses: actions/checkout@v4 | |
| - name: Install uv | |
| uses: astral-sh/setup-uv@v5 | |
| with: | |
| enable-cache: true | |
| # `.oasdiff-version` is the pin, and `scripts/bootstrap_tools.sh` reads that same file, so | |
| # a job here and a developer's checkout cannot drift apart. The last three lines assert | |
| # rather than print: a version that is only echoed is a version nobody checks, which is | |
| # how a pinned job and seven working copies came to run two different builds. | |
| - name: Install oasdiff | |
| run: | | |
| version="$(grep -m1 -Ev '^[[:space:]]*(#|$)' .oasdiff-version)" | |
| version="${version//[[:space:]]/}" | |
| mkdir -p tools | |
| curl -fsSL -o /tmp/oasdiff.tar.gz \ | |
| "https://github.com/oasdiff/oasdiff/releases/download/v${version}/oasdiff_${version}_linux_amd64.tar.gz" | |
| tar -xzf /tmp/oasdiff.tar.gz -C tools oasdiff | |
| installed="$(tools/oasdiff --version)" | |
| if [ "$installed" != "oasdiff version ${version}" ]; then | |
| echo "pinned v${version} in .oasdiff-version, installed '${installed}'" >&2 | |
| exit 1 | |
| fi | |
| echo "$installed" | |
| - name: Sync dependencies | |
| run: uv sync --all-extras --dev | |
| # Runs before the suite: a failure here is a fact about the source, and needs no | |
| # database, no binary, and no test run to establish. | |
| - name: Encoding lint | |
| run: uv run python scripts/lint_encoding.py src scripts tests | |
| - name: Import boundary | |
| run: uv run lint-imports | |
| # Being tested and being reachable are different properties, and until this step | |
| # existed only the first was checked -- four finished components shipped with no | |
| # caller anywhere in src/. The baseline lists the ones already accepted and only | |
| # shrinks: an entry that no longer violates fails here until it is deleted. | |
| - name: Dead links | |
| run: uv run python scripts/lint_dead_links.py src --baseline scripts/dead_links_baseline.txt | |
| - name: Tests | |
| run: uv run pytest | |
| # Recorded, never gated. `2026-07-27-sync-benchmark-gates.md` forbids inventing a | |
| # threshold -- a gate at an invented number either fires constantly and gets disabled or | |
| # never fires and provides false assurance -- and a percentage that fails a build is a | |
| # percentage people write tests to satisfy rather than to test something. | |
| # | |
| # `|| true` is the whole of the not-gating, and it is deliberate rather than defensive: | |
| # a coverage run that errors must not turn a green suite red either. The baseline this | |
| # is measured against is `docs/superpowers/specs/2026-07-29-sync-coverage-baseline.md`, | |
| # which also records what this number cannot see -- seven components have shipped here | |
| # fully covered and reachable from nothing, and every one of them looked healthy to a | |
| # line count. | |
| - name: Coverage (recorded, not gated) | |
| run: uv run pytest -q --cov=sync --cov-report=term-missing:skip-covered || true | |
| # The first gate on a quality number rather than on correctness, and the only one | |
| # `2026-07-27-sync-benchmark-gates.md` allows today: a directional floor on a | |
| # deterministic axis. The floors are not here. They are in `scripts/gate_corpus.py`, | |
| # beside the figure each one guards and the argument for guarding it, because a floor in | |
| # a workflow file is a string a reader meets without the number it came from and lowering | |
| # one is a config tweak rather than a reviewable act. | |
| # | |
| # No `|| true`. The coverage step above is recorded and this is gated, and the difference | |
| # between them is the whole of what this step adds. | |
| - name: Fetch the frozen corpus | |
| run: uv run python scripts/fetch_corpus_repositories.py | |
| # A database of its own. `score_corpus.py` truncates the graph once per pair, so pointing | |
| # it at the one the suite used would delete what a later step might read. | |
| - name: Score the frozen corpus | |
| run: | | |
| uv run python -c " | |
| import psycopg | |
| from psycopg import sql | |
| with psycopg.connect('postgresql://sync:sync@localhost:5432/postgres', autocommit=True) as c: | |
| c.execute(sql.SQL('DROP DATABASE IF EXISTS {} WITH (FORCE)').format(sql.Identifier('sync_benchmark'))) | |
| c.execute(sql.SQL('CREATE DATABASE {}').format(sql.Identifier('sync_benchmark'))) | |
| " | |
| uv run python scripts/score_corpus.py \ | |
| --score-dsn postgresql://sync:sync@localhost:5432/sync_benchmark \ | |
| --json corpus-score.json | |
| - name: Binding floors over the frozen corpus (gated) | |
| run: uv run python scripts/gate_corpus.py --score corpus-score.json |