Skip to content

Commit bc307f5

Browse files
committed
docs(benchmarks): parity-row note matches the committed snapshot
The committed data predates the per-linter parity split, so it carries a single `mdsmith-parity` row — but the generated results note described per-linter `mdsmith-<linter>-parity` rows that aren't in the table, which read as a contradiction. Make gen_fragments.py emit the note that matches the rows the data actually holds: the single pre-split column now, and the per-peer wording automatically once a refresh adds the four columns. Align the "Like-for-like" parenthetical to the same framing (single column reads closest to mado-parity; the next refresh replaces it with one column per peer). Regenerated; results table byte-identical; `mdsmith check .` passes. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PThu9DcNDDSAKaw1upanY4
1 parent 983bedf commit bc307f5

3 files changed

Lines changed: 29 additions & 16 deletions

File tree

docs/research/benchmarks/README.md

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -109,10 +109,10 @@ file: results.fragment.md
109109
docs/research/benchmarks/data/*.json — do not edit by hand. Re-run
110110
the harness (run.sh) and `mdsmith fix` to refresh. -->
111111

112-
`mdsmith` is the default rule set. Each `mdsmith-<linter>-parity`
113-
row runs the rule set that peer enables by default, for a
114-
like-for-like comparison against that peer (the
115-
`bench-<linter>-parity.mdsmith.yml` profiles).
112+
`mdsmith` is the default rule set. The `mdsmith-parity`
113+
row is the single pre-split parity column; the per-linter
114+
`mdsmith-<linter>-parity` columns, one per peer, land when
115+
the benchmark is next re-run.
116116

117117
**Repo corpus — 766 Markdown files** (median wall time, lower is
118118
better; `vs mado` is the ratio to mado's median):
@@ -185,9 +185,9 @@ The `mdsmith` → `mdsmith-parity` delta is the measured cost of
185185
the cross-file and generated-content layer, work users opt into.
186186
The residual gap to mado on long prose is engine headroom: the
187187
number to drive down, and the profiler loop below is how. (These
188-
figures predate the per-linter split; the `mdsmith-parity`
189-
column now selects `mado-parity`, whose 27-rule set is close to
190-
the rule class measured here.)
188+
figures predate the per-linter split: the single `mdsmith-parity`
189+
column shown here reads closest to the `mado-parity` 27-rule set,
190+
and the next refresh replaces it with one column per peer.)
191191

192192
### Why parity trails gomarklint
193193

docs/research/benchmarks/gen_fragments.py

Lines changed: 18 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -80,11 +80,24 @@ def main() -> None:
8080
repo_tbl, repo = rows(json_dir, "corpus_repo")
8181
neut_tbl, _ = rows(json_dir, "corpus_neutral")
8282

83-
note = ("`mdsmith` is the default rule set. Each "
84-
"`mdsmith-<linter>-parity`\nrow runs the rule set that "
85-
"peer enables by default, for a\nlike-for-like comparison "
86-
"against that peer (the\n`bench-<linter>-parity.mdsmith.yml` "
87-
"profiles).\n\n")
83+
# The parity columns differ across the per-linter split: the pre-split
84+
# snapshot carries a single `mdsmith-parity` row, while the current
85+
# harness emits one `mdsmith-<peer>-parity` column per peer. Describe
86+
# whichever the data actually holds so the note never contradicts the
87+
# table below it.
88+
per_linter_parity = [c for c in repo if c.startswith("mdsmith-")
89+
and c.endswith("-parity") and c != "mdsmith-parity"]
90+
if per_linter_parity:
91+
note = ("`mdsmith` is the default rule set. Each "
92+
"`mdsmith-<linter>-parity`\nrow runs the rule set that "
93+
"peer enables by default, for a\nlike-for-like comparison "
94+
"against that peer (the\n`bench-<linter>-parity.mdsmith.yml` "
95+
"profiles).\n\n")
96+
else:
97+
note = ("`mdsmith` is the default rule set. The `mdsmith-parity`\n"
98+
"row is the single pre-split parity column; the per-linter\n"
99+
"`mdsmith-<linter>-parity` columns, one per peer, land when\n"
100+
"the benchmark is next re-run.\n\n")
88101
results = (GEN + "\n" + note +
89102
f"**Repo corpus — {repo_n} Markdown files** (median "
90103
"wall time, lower is\nbetter; `vs mado` is the ratio "

docs/research/benchmarks/results.fragment.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -2,10 +2,10 @@
22
docs/research/benchmarks/data/*.json — do not edit by hand. Re-run
33
the harness (run.sh) and `mdsmith fix` to refresh. -->
44

5-
`mdsmith` is the default rule set. Each `mdsmith-<linter>-parity`
6-
row runs the rule set that peer enables by default, for a
7-
like-for-like comparison against that peer (the
8-
`bench-<linter>-parity.mdsmith.yml` profiles).
5+
`mdsmith` is the default rule set. The `mdsmith-parity`
6+
row is the single pre-split parity column; the per-linter
7+
`mdsmith-<linter>-parity` columns, one per peer, land when
8+
the benchmark is next re-run.
99

1010
**Repo corpus — 766 Markdown files** (median wall time, lower is
1111
better; `vs mado` is the ratio to mado's median):

0 commit comments

Comments
 (0)