|
1 | 1 | # Sync — The Coverage Baseline, Third Measurement |
2 | 2 |
|
3 | | -**Date:** 2026-07-30 |
| 3 | +**Date:** 2026-07-30; figure re-taken 2026-08-02 at `048e28a` after two session limits interrupted |
| 4 | +the write-up. The two `preserve:` commits in this branch's history are where each interruption fell |
| 5 | +and are left unamended. |
4 | 6 | **Status:** Measurement. Recorded, reviewed by a human, gated by nothing — the same treatment |
5 | 7 | `2026-07-27-sync-benchmark-gates.md` gives every tier B axis, for the reason it gives. |
6 | | -**Scope:** How much of `src/sync` the default suite executes at `a77eb22`; which modules are worth |
| 8 | +**Scope:** How much of `src/sync` the default suite executes at `048e28a`; which modules are worth |
7 | 9 | hardening *next*, ranked by what a wrong answer in each one costs rather than by percentage; which |
8 | 10 | of the two earlier baselines' line citations still resolve; and what became of every module either |
9 | 11 | of them named. |
@@ -79,10 +81,18 @@ same four in both, and every one of them names a **gitignored** input: |
79 | 81 |
|
80 | 82 | The **two** parametrised `test_parameter_reduction` cases are the whole discrepancy. They pass in |
81 | 83 | any tree where somebody has run `scripts/fetch_measurement_inputs.py` and skip everywhere else. The |
82 | | -coordinator's checkout has `.cache/specs/` populated from a fetch on 2026-07-30, which is why it |
83 | | -measured 2,701 and 2; this worktree does not, which is why it measures 2,719 and 4. The totals |
84 | | -reconcile exactly — 2,721 + 2 = 2,719 + 4 = 2,723 — and always did, which is what should have |
85 | | -prompted the question rather than the arithmetic that was offered instead. |
| 84 | +coordinator's checkout has `.cache/specs/` populated from a fetch on 2026-07-30 — eight Stripe |
| 85 | +specifications, none of them committed — and this worktree does not. So the same suite at the same |
| 86 | +commit reports 2,721 passed and 2 skipped there and 2,719 passed and 4 skipped here, and the totals |
| 87 | +reconcile exactly: 2,721 + 2 = 2,719 + 4 = 2,723. |
| 88 | + |
| 89 | +They always did reconcile, which is what should have prompted the question rather than the |
| 90 | +arithmetic that was offered instead. The earlier draft was comparing across two commits as well as |
| 91 | +two checkouts: the coordinator's 2,701 and 2 was measured at `d615b75`, twenty tests before |
| 92 | +`048e28a`. Those twenty are `tests/test_rank_coverage.py` and `tests/test_replay_shape_writeback.py`, |
| 93 | +which merged in between and collect twenty items exactly. Two variables had moved, and the draft |
| 94 | +attributed the whole difference to a third that had not moved at all — `--cov`, which moves |
| 95 | +nothing. |
86 | 96 |
|
87 | 97 | **CI is on this side of it.** `.github/workflows/ci.yml` installs `oasdiff` and syncs dependencies |
88 | 98 | and never runs `scripts/fetch_measurement_inputs.py`, so the suite CI gates on skips the same four. |
@@ -298,7 +308,7 @@ nobody should try. The ranking cannot know that, which is why the last column of |
298 | 308 | The spec audit named one broken citation and declined to renumber it. There are nine in the two |
299 | 309 | documents. **Eight no longer resolve; one does.** |
300 | 310 |
|
301 | | -| Citation | Document | What it described | What sits there at `d615b75` | Where the construct lives now | |
| 311 | +| Citation | Document | What it described | What sits there at `048e28a` | Where the construct lives now | |
302 | 312 | |---|---|---|---|---| |
303 | 313 | | `index/python_lang.py:206-210` | baseline 1 | `from stripe import X as Y` | the vendor-binding constructor reading `distribution` | 437–443 | |
304 | 314 | | `index/python_lang.py:220-223` | baseline 1 | `import stripe as s` | a comment and `self._symbol_root` | 453–456 | |
@@ -326,7 +336,7 @@ Sixteen modules were named across the two documents. **None is gone; every file |
326 | 336 | Nine are at 100% or one statement short of it, two have had every remaining statement classified by |
327 | 337 | a committed report, and five still carry uncovered statements nobody has read. |
328 | 338 |
|
329 | | -| Module | `58257f6` | `5c546fa` | `d615b75` | Fate | |
| 339 | +| Module | `58257f6` | `5c546fa` | `048e28a` | Fate | |
330 | 340 | |---|---|---|---|---| |
331 | 341 | | `telemetry/otlp.py` | 83%, 12 missed | 83%, 12 missed | **100%**, 0 | Repaired by baseline-2's own task, thirteen tests, each proven non-vacuous by mutation | |
332 | 342 | | `mcp/server.py` | 86%, 13 | 82%, 17 | **100%**, 0 | Repaired — `2026-07-30-mcp-tool-surface-declines.md` (M3-W107) | |
@@ -364,7 +374,7 @@ top recommendation survives three of its own successors is a document being read |
364 | 374 |
|
365 | 375 | That matters because a per-module coverage run on this project can fail for a reason that has |
366 | 376 | nothing to do with the module. `2026-07-29-psycopg-error-identity.md` established the mechanism, and |
367 | | -it survived one retraction that was itself wrong, so it was re-verified here at `d615b75` against |
| 377 | +it survived one retraction that was itself wrong, so it was re-verified here at `048e28a` against |
368 | 378 | the installed `coverage 7.15.2` rather than cited: |
369 | 379 |
|
370 | 380 | ``` |
@@ -412,7 +422,7 @@ further than it deserves. |
412 | 422 | number either fires constantly and gets disabled, or never fires and provides false assurance."* CI |
413 | 423 | records the figure with `|| true` and gates on nothing, which is correct and was left alone. |
414 | 424 |
|
415 | | -There are now three measurements, at `58257f6`, `5c546fa` and `d615b75`. Three points are the |
| 425 | +There are now three measurements, at `58257f6`, `5c546fa` and `048e28a`. Three points are the |
416 | 426 | beginning of a history rather than a history, and this measurement finally moved by more than the |
417 | 427 | noise the second one had to account for — 2.16 points, against a run-to-run variation baseline-2 |
418 | 428 | measured at three statements. So the objection is no longer that the signal is smaller than the |
|
0 commit comments