Skip to content

Commit f75e826

Browse files
committed
docs: settle the skip discrepancy and repin the baseline to 048e28a
The open item was whether `--cov` skips two tests the gate runs. It does not, and nothing in the suite changes outcome under coverage. The gap was a checkout difference compared across two commits at once: `.cache/specs/` is gitignored, the coordinator's tree has it populated from a fetch and this worktree does not, and the two figures being reconciled were taken twenty tests apart. Both the mechanism and the arithmetic that should have exposed it are recorded, because the correction is the useful part. The tree's `src/` is byte-identical between `d615b75` and `048e28a`, so the citation and fate tables carry over unchanged and only their column headings are repinned.
1 parent 081efb8 commit f75e826

1 file changed

Lines changed: 20 additions & 10 deletions

File tree

docs/superpowers/specs/2026-07-30-sync-coverage-baseline-3.md

Lines changed: 20 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,11 @@
11
# Sync — The Coverage Baseline, Third Measurement
22

3-
**Date:** 2026-07-30
3+
**Date:** 2026-07-30; figure re-taken 2026-08-02 at `048e28a` after two session limits interrupted
4+
the write-up. The two `preserve:` commits in this branch's history are where each interruption fell
5+
and are left unamended.
46
**Status:** Measurement. Recorded, reviewed by a human, gated by nothing — the same treatment
57
`2026-07-27-sync-benchmark-gates.md` gives every tier B axis, for the reason it gives.
6-
**Scope:** How much of `src/sync` the default suite executes at `a77eb22`; which modules are worth
8+
**Scope:** How much of `src/sync` the default suite executes at `048e28a`; which modules are worth
79
hardening *next*, ranked by what a wrong answer in each one costs rather than by percentage; which
810
of the two earlier baselines' line citations still resolve; and what became of every module either
911
of them named.
@@ -79,10 +81,18 @@ same four in both, and every one of them names a **gitignored** input:
7981

8082
The **two** parametrised `test_parameter_reduction` cases are the whole discrepancy. They pass in
8183
any tree where somebody has run `scripts/fetch_measurement_inputs.py` and skip everywhere else. The
82-
coordinator's checkout has `.cache/specs/` populated from a fetch on 2026-07-30, which is why it
83-
measured 2,701 and 2; this worktree does not, which is why it measures 2,719 and 4. The totals
84-
reconcile exactly — 2,721 + 2 = 2,719 + 4 = 2,723 — and always did, which is what should have
85-
prompted the question rather than the arithmetic that was offered instead.
84+
coordinator's checkout has `.cache/specs/` populated from a fetch on 2026-07-30 — eight Stripe
85+
specifications, none of them committed — and this worktree does not. So the same suite at the same
86+
commit reports 2,721 passed and 2 skipped there and 2,719 passed and 4 skipped here, and the totals
87+
reconcile exactly: 2,721 + 2 = 2,719 + 4 = 2,723.
88+
89+
They always did reconcile, which is what should have prompted the question rather than the
90+
arithmetic that was offered instead. The earlier draft was comparing across two commits as well as
91+
two checkouts: the coordinator's 2,701 and 2 was measured at `d615b75`, twenty tests before
92+
`048e28a`. Those twenty are `tests/test_rank_coverage.py` and `tests/test_replay_shape_writeback.py`,
93+
which merged in between and collect twenty items exactly. Two variables had moved, and the draft
94+
attributed the whole difference to a third that had not moved at all — `--cov`, which moves
95+
nothing.
8696

8797
**CI is on this side of it.** `.github/workflows/ci.yml` installs `oasdiff` and syncs dependencies
8898
and never runs `scripts/fetch_measurement_inputs.py`, so the suite CI gates on skips the same four.
@@ -298,7 +308,7 @@ nobody should try. The ranking cannot know that, which is why the last column of
298308
The spec audit named one broken citation and declined to renumber it. There are nine in the two
299309
documents. **Eight no longer resolve; one does.**
300310

301-
| Citation | Document | What it described | What sits there at `d615b75` | Where the construct lives now |
311+
| Citation | Document | What it described | What sits there at `048e28a` | Where the construct lives now |
302312
|---|---|---|---|---|
303313
| `index/python_lang.py:206-210` | baseline 1 | `from stripe import X as Y` | the vendor-binding constructor reading `distribution` | 437–443 |
304314
| `index/python_lang.py:220-223` | baseline 1 | `import stripe as s` | a comment and `self._symbol_root` | 453–456 |
@@ -326,7 +336,7 @@ Sixteen modules were named across the two documents. **None is gone; every file
326336
Nine are at 100% or one statement short of it, two have had every remaining statement classified by
327337
a committed report, and five still carry uncovered statements nobody has read.
328338

329-
| Module | `58257f6` | `5c546fa` | `d615b75` | Fate |
339+
| Module | `58257f6` | `5c546fa` | `048e28a` | Fate |
330340
|---|---|---|---|---|
331341
| `telemetry/otlp.py` | 83%, 12 missed | 83%, 12 missed | **100%**, 0 | Repaired by baseline-2's own task, thirteen tests, each proven non-vacuous by mutation |
332342
| `mcp/server.py` | 86%, 13 | 82%, 17 | **100%**, 0 | Repaired — `2026-07-30-mcp-tool-surface-declines.md` (M3-W107) |
@@ -364,7 +374,7 @@ top recommendation survives three of its own successors is a document being read
364374

365375
That matters because a per-module coverage run on this project can fail for a reason that has
366376
nothing to do with the module. `2026-07-29-psycopg-error-identity.md` established the mechanism, and
367-
it survived one retraction that was itself wrong, so it was re-verified here at `d615b75` against
377+
it survived one retraction that was itself wrong, so it was re-verified here at `048e28a` against
368378
the installed `coverage 7.15.2` rather than cited:
369379

370380
```
@@ -412,7 +422,7 @@ further than it deserves.
412422
number either fires constantly and gets disabled, or never fires and provides false assurance."* CI
413423
records the figure with `|| true` and gates on nothing, which is correct and was left alone.
414424

415-
There are now three measurements, at `58257f6`, `5c546fa` and `d615b75`. Three points are the
425+
There are now three measurements, at `58257f6`, `5c546fa` and `048e28a`. Three points are the
416426
beginning of a history rather than a history, and this measurement finally moved by more than the
417427
noise the second one had to account for — 2.16 points, against a run-to-run variation baseline-2
418428
measured at three statements. So the objection is no longer that the signal is smaller than the

0 commit comments

Comments
 (0)