Skip to content

Commit f80ffba

Browse files
committed
docs(bench): add "Measuring the impact of an engine change" signpost
Orient engine contributors (layout / pagination / render ordering / measurement / font changes) to the right tool for seeing perf impact: the run-benchmarks verdict gate vs the committed baseline for a quick regress check, and the ab-bench A/B scripts for an attributable per-scenario before/after vs develop. Ties together the existing verdict-gate, A/B, and baseline-refresh sections.
1 parent 85a663e commit f80ffba

1 file changed

Lines changed: 30 additions & 0 deletions

File tree

docs/operations/benchmarks.md

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -176,6 +176,36 @@ powershell -ExecutionPolicy Bypass -File .\scripts\run-benchmarks.ps1 -SkipDiff
176176
powershell -ExecutionPolicy Bypass -File .\scripts\run-benchmarks.ps1 -OpenResults
177177
```
178178

179+
## Measuring the impact of an engine change
180+
181+
Changing the engine (layout, pagination, render ordering, PDF session, text
182+
measurement, fonts) and want to see how it moves performance? Pick the view that
183+
fits, cheapest first:
184+
185+
- **"Did I regress?" — gate against the committed baseline.** Run a median and
186+
let the `11-verdict-current-speed` step score each scenario IMPROVED /
187+
NEUTRAL / REGRESSED against `baselines/current-speed-full.json` (hard gate:
188+
average latency ±10%, non-zero exit on a regression):
189+
190+
```powershell
191+
powershell -ExecutionPolicy Bypass -File .\scripts\run-benchmarks.ps1 -CurrentSpeedProfile full -Repeat 5
192+
```
193+
194+
- **"What exactly moved?" — A/B your branch against its base (any OS).** Commit
195+
your change, then compare it to `develop` with the A/B scripts (see
196+
[A/B comparison between two branches](#ab-comparison-between-two-branches)).
197+
Both sides are rebuilt and benchmarked, with a per-scenario delta:
198+
199+
```bash
200+
./scripts/ab-bench.sh -a develop -b my/engine-change -r 5
201+
```
202+
203+
If a change is *meant* to improve performance and the gate confirms it, refresh
204+
the baseline so the gate ratchets down — see
205+
[Refreshing the committed baseline](#refreshing-the-committed-baseline-perf-gate).
206+
Treat sub-~5-10% laptop deltas as inconclusive, and re-run on the final checkout
207+
before citing a number.
208+
179209
## A/B comparison between two branches
180210

181211
The wrappers above benchmark whatever is currently checked out. To answer "is

0 commit comments

Comments
 (0)