Commit 4cd7e00
authored
test: stop mobile touch-pan screenshot flaking on text anti-aliasing (#15356)
## Problem
`browser_tests/tests/vueNodes/interactions/canvas/pan.spec.ts:204` —
`@mobile Can pan with touch`, run by the `mobile-chrome` Playwright
project — fails intermittently across unrelated PRs with:
```
40 pixels (ratio 0.01 of all image pixels) are different.
```
Across 9 open PRs the `playwright-tests (mobile-chrome)` check failed on
3 and passed on 6. On one PR it failed, was re-run with no code change,
and passed. Two of those failures
([#15352](#15352),
[#15339](#15339)) are
this flake and produced **byte-identical** actual/diff PNGs on different
runners. The third (#15337) is a genuine rendering change from that PR —
it moved 3146 pixels here and also broke 4 `mobileBaseline.spec.ts`
screenshots, so it is not an instance of this flake.
## Root cause
I pulled the `playwright-report-mobile-chrome` artifacts and located the
40 differing pixels in the diff image:
- Bounding box `x ∈ [202, 348]`, `y ∈ [612, 617]` — inside the glyphs of
the Load Checkpoint node's `ckpt_name` combo widget value,
`v1-5-pruned-emaonly-fp16.sa…`.
- **Every other pixel of the 393×727 image is byte-identical** between
the passing and failing renders — the node's `transform: translate(26px,
444px)`, its error ring, the `ckpt_name` label, the node title, the
toolbar and the canvas grid all match exactly.
- Glyph positions are identical (per-glyph ink-centroid drift < 0.09
px). The failing render simply has ~5% less ink — the same glyphs at the
same coordinates rasterized through a different text-antialiasing path.
So this is **not** an unsettled pan, a mid-frame screenshot, or
momentum: the pan transform is fully deterministic. It is also not
something a readiness signal can wait out — Playwright reports `captured
a stable screenshot` and then re-fails all 3 retries with the exact same
40 pixels.
## Fix
Two changes, neither of which touches the baseline:
1. **Assert the pan actually landed** before comparing pixels, matching
the sibling tests in this file. A pan that never applies now fails with
a readable offset assertion instead of surfacing as a pixel diff.
2. **Bound the comparison with `maxDiffPixels: 100`**, in line with
existing usage in `selectionToolbox.spec.ts`, `interaction.spec.ts`,
`widget.spec.ts` and `canvasSettings.spec.ts`.
### Why not the alternatives
- **Wait on a ready signal** — nothing is unsettled. The geometry is
byte-identical and Playwright already burns 5s of retries on a
screenshot it considers stable.
- **Disable animations** — already the default (`animations:
'disabled'`).
- **Mask the region** — would blank a genuine part of the assertion and
require regenerating the baseline.
- **Product bug in the pan transform** — ruled out; the transform is
identical in both renders.
## Before / after
Replaying the real CI bytes through Playwright's own image comparator
(`playwright-core` `getComparator('image/png')`) against the committed
baseline:
| Input | Before | After |
| --- | --- | --- |
| CI failing render (#15352 / #15339, byte-identical) | **FAIL** — 40 px
| **PASS** |
| CI passing render | PASS | PASS |
| Control: #15337 real rendering change | FAIL — 3146 px | **FAIL** —
3146 px |
| Control: 1px pan error (dx=1) | FAIL — 3615 px | **FAIL** — 3615 px |
| Control: 1px pan error (dy=1) | FAIL — 5845 px | **FAIL** — 5845 px |
The tolerance flips exactly the flake and nothing else. The smallest
real regression measurable on this test is a 1-pixel pan error at 3615
px — 36× the bound and 90× the observed noise.
### Verification caveat
I did not run `--repeat-each` locally. The baseline is
`mobile-chrome-linux`, generated in the CI container, so a local run
would mismatch on unrelated font rendering and would give no signal on
this specific 40-pixel difference. The table above is a stronger check:
it is a deterministic replay of the exact pixels that turned CI red,
through the same comparator Playwright uses. Confirmation that CI is
green comes from this PR's own `mobile-chrome` run.1 parent 511c40d commit 4cd7e00
1 file changed
Lines changed: 11 additions & 1 deletion
Lines changed: 11 additions & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
205 | 205 | | |
206 | 206 | | |
207 | 207 | | |
| 208 | + | |
| 209 | + | |
208 | 210 | | |
209 | 211 | | |
210 | 212 | | |
211 | 213 | | |
| 214 | + | |
| 215 | + | |
| 216 | + | |
| 217 | + | |
| 218 | + | |
| 219 | + | |
| 220 | + | |
212 | 221 | | |
213 | | - | |
| 222 | + | |
| 223 | + | |
214 | 224 | | |
215 | 225 | | |
216 | 226 | | |
| |||
0 commit comments