Skip to content

Commit 4cd7e00

Browse files
test: stop mobile touch-pan screenshot flaking on text anti-aliasing (#15356)
## Problem `browser_tests/tests/vueNodes/interactions/canvas/pan.spec.ts:204` — `@mobile Can pan with touch`, run by the `mobile-chrome` Playwright project — fails intermittently across unrelated PRs with: ``` 40 pixels (ratio 0.01 of all image pixels) are different. ``` Across 9 open PRs the `playwright-tests (mobile-chrome)` check failed on 3 and passed on 6. On one PR it failed, was re-run with no code change, and passed. Two of those failures ([#15352](#15352), [#15339](#15339)) are this flake and produced **byte-identical** actual/diff PNGs on different runners. The third (#15337) is a genuine rendering change from that PR — it moved 3146 pixels here and also broke 4 `mobileBaseline.spec.ts` screenshots, so it is not an instance of this flake. ## Root cause I pulled the `playwright-report-mobile-chrome` artifacts and located the 40 differing pixels in the diff image: - Bounding box `x ∈ [202, 348]`, `y ∈ [612, 617]` — inside the glyphs of the Load Checkpoint node's `ckpt_name` combo widget value, `v1-5-pruned-emaonly-fp16.sa…`. - **Every other pixel of the 393×727 image is byte-identical** between the passing and failing renders — the node's `transform: translate(26px, 444px)`, its error ring, the `ckpt_name` label, the node title, the toolbar and the canvas grid all match exactly. - Glyph positions are identical (per-glyph ink-centroid drift < 0.09 px). The failing render simply has ~5% less ink — the same glyphs at the same coordinates rasterized through a different text-antialiasing path. So this is **not** an unsettled pan, a mid-frame screenshot, or momentum: the pan transform is fully deterministic. It is also not something a readiness signal can wait out — Playwright reports `captured a stable screenshot` and then re-fails all 3 retries with the exact same 40 pixels. ## Fix Two changes, neither of which touches the baseline: 1. **Assert the pan actually landed** before comparing pixels, matching the sibling tests in this file. A pan that never applies now fails with a readable offset assertion instead of surfacing as a pixel diff. 2. **Bound the comparison with `maxDiffPixels: 100`**, in line with existing usage in `selectionToolbox.spec.ts`, `interaction.spec.ts`, `widget.spec.ts` and `canvasSettings.spec.ts`. ### Why not the alternatives - **Wait on a ready signal** — nothing is unsettled. The geometry is byte-identical and Playwright already burns 5s of retries on a screenshot it considers stable. - **Disable animations** — already the default (`animations: 'disabled'`). - **Mask the region** — would blank a genuine part of the assertion and require regenerating the baseline. - **Product bug in the pan transform** — ruled out; the transform is identical in both renders. ## Before / after Replaying the real CI bytes through Playwright's own image comparator (`playwright-core` `getComparator('image/png')`) against the committed baseline: | Input | Before | After | | --- | --- | --- | | CI failing render (#15352 / #15339, byte-identical) | **FAIL** — 40 px | **PASS** | | CI passing render | PASS | PASS | | Control: #15337 real rendering change | FAIL — 3146 px | **FAIL** — 3146 px | | Control: 1px pan error (dx=1) | FAIL — 3615 px | **FAIL** — 3615 px | | Control: 1px pan error (dy=1) | FAIL — 5845 px | **FAIL** — 5845 px | The tolerance flips exactly the flake and nothing else. The smallest real regression measurable on this test is a 1-pixel pan error at 3615 px — 36× the bound and 90× the observed noise. ### Verification caveat I did not run `--repeat-each` locally. The baseline is `mobile-chrome-linux`, generated in the CI container, so a local run would mismatch on unrelated font rendering and would give no signal on this specific 40-pixel difference. The table above is a stronger check: it is a deterministic replay of the exact pixels that turned CI red, through the same comparator Playwright uses. Confirmation that CI is green comes from this PR's own `mobile-chrome` run.
1 parent 511c40d commit 4cd7e00

1 file changed

Lines changed: 11 additions & 1 deletion

File tree

  • browser_tests/tests/vueNodes/interactions/canvas

browser_tests/tests/vueNodes/interactions/canvas/pan.spec.ts

Lines changed: 11 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -205,12 +205,22 @@ test.describe('Vue Nodes Canvas Pan', { tag: '@vue-nodes' }, () => {
205205
'@mobile Can pan with touch',
206206
{ tag: '@screenshot' },
207207
async ({ comfyPage }) => {
208+
const offsetBefore = await comfyPage.canvasOps.getOffset()
209+
208210
await comfyPage.canvasOps.panWithTouch(
209211
{ x: 64, y: 64 },
210212
{ x: 256, y: 256 }
211213
)
214+
215+
// Fail on a pan that never landed here, not as a screenshot diff.
216+
await expect
217+
.poll(() => comfyPage.canvasOps.getOffset())
218+
.not.toEqual(offsetBefore)
219+
220+
// Tolerates text anti-aliasing noise in the widget value text.
212221
await expect(comfyPage.canvas).toHaveScreenshot(
213-
'vue-nodes-paned-with-touch.png'
222+
'vue-nodes-paned-with-touch.png',
223+
{ maxDiffPixels: 100 }
214224
)
215225
}
216226
)

0 commit comments

Comments
 (0)