chore(e2e): refresh screenshot baselines and drop unread snapshot files - #3005
Merged
Conversation
Several baselines had drifted far from what the extension renders. The sign-transaction one, for example, still showed docs.freighter.app and an expanded transaction details panel, while the current screen shows play.freighter.app, a Network row, a Memo row, and details collapsed behind a link. The drift went unnoticed because expectPageToHaveScreenshot compares with maxDiffPixelRatio: 0.02 — about 18k pixels of a 1280x720 shot — and Playwright's per-pixel threshold defaults to 0.2, so only text glyphs register as different. Flat panel backgrounds shifting by a few RGB steps do not count, which leaves enough headroom for a whole redesigned section to slip through unflagged. Regenerated with --update-snapshots=all so the baselines describe the screens as they are today, rather than sitting just inside the tolerance. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Playwright names screenshots {arg}{-projectName}{-snapshotSuffix}, and the only project is named chromium, so the 30 files ending in plain -darwin.png were never read by any comparison. They are leftovers from the older toMatchSnapshot naming, which omitted the project name.
Ten more had the right name but no reference: manage-assets-page and manage-assets-unverified-token under addAsset (addAssetIntegration references manage-assets-page from its own snapshot directory), domain-not-allowed and domain-not-allowed-sign-message, and all six under sendPayment.
The sendPayment ones belonged to helpers/sendPayment.ts, which no test file imports. The test it once served was rewritten and now covers the flow with element assertions instead, so the helper and its snapshot directory go too.
Every snapshot directory now matches its test file's references exactly, with nothing missing and nothing unread.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Refreshes active E2E screenshot baselines and removes unused snapshot infrastructure.
Changes:
- Refreshes 28 active Playwright baselines.
- Removes unused snapshots and the unreferenced payment helper.
- Leaves stale README references to the deleted snapshot directory.
Suppressed comments (1)
extension/e2e-tests/helpers/sendPayment.ts:1
- Deleting the send-payment snapshot set leaves the E2E guide with stale references:
extension/e2e-tests/README.md:238still presentssendPayment.test.ts-snapshots/as an active mapping, and line 330 links to that now-missing directory. Please update those references as part of this cleanup so the documented snapshot workflow has no broken link.
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Contributor
|
PR Preview build is ready: https://github.com/stellar/freighter/releases/tag/untagged-f94a75a300332e8264a6 |
The guide used sendPayment.test.ts as its example of a test-to-snapshot-directory mapping, as the target of the per-test --update-snapshots command, and as the Additional Resources link to the baselines. That directory is gone, so the link was broken and the command would have updated nothing. All three now point at onboarding.test.ts, which carries twelve baselines and is the largest set in the suite. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
JiahuiWho
approved these changes
Sep 10, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
TL;DR
The Playwright screenshot baselines had quietly drifted out of date, and a pile of
snapshot files in the repo were no longer read by any test. This refreshes the 28
baselines the suite actually compares against and deletes 40 dead snapshot files
plus one test helper that nothing imports.
The reason the drift went unnoticed is worth knowing: the shared screenshot helper
allows a 2% differing-pixel ratio, and Playwright's per-pixel colour threshold
means only text glyphs count as "different" — flat panel backgrounds shifting by a
few RGB steps do not. That is enough headroom for an entire redesigned section to
pass comparison. One baseline still showed the old
docs.freighter.appdomain andan expanded transaction details panel, while the screen today shows
play.freighter.app, a Network row, a Memo row, and details collapsed behind alink — and it passed. Tightening that tolerance is deliberately not part of this
PR; see the follow-ups below.
No test logic changed, and no product code changed.
Implementation details (for agents)
What changed
Two commits, kept separate so the binary refresh and the deletions can be reviewed
independently.
test(e2e): refresh screenshot baselines that no longer match the UI— 28 PNGsregenerated with
yarn test:e2e … --update-snapshots=all. These are everybaseline the suite currently compares against, except one (see below).
test(e2e): delete snapshot files and a helper that nothing reads— 41 deletions.Why the 40 snapshot files were dead
Playwright resolves screenshot paths as
{arg}{-projectName}{-snapshotSuffix}{ext}, and the only configured project isnamed
chromium.So the effective filename is always
<name>-chromium-darwin.png, and the 30 filesending in a plain
-darwin.pngwere never read by any comparison — leftovers fromthe older
toMatchSnapshotnaming, which omitted the project name.Ten more were correctly named but referenced by nothing:
addAsset.test.ts-snapshots/manage-assets-page— onlyintegration-tests/addAssetIntegration.test.tsreferencesmanage-assets-page.png,and it resolves to its own snapshot directory.
addAsset.test.ts-snapshots/manage-assets-unverified-tokenfreighterApiIntegration.test.ts-snapshots/domain-not-allowedanddomain-not-allowed-sign-messagesendPayment.test.ts-snapshotsThe dead helper
extension/e2e-tests/helpers/sendPayment.tsexportedsendXlmPayment, which tookthe
send-payment-to/-amount/-sentscreenshots. No test file imports it —sendPayment.test.tswas rewritten to cover the flow with element assertionsinstead. The helper, and the whole
sendPayment.test.ts-snapshotsdirectory(including three baselines whose names the helper never even referenced), are
removed together.
Verification
yarn test:e2e— full suite, repo config (retries: 5,maxFailures: 1):233 passed, 10 skipped, 0 failed, 0 flaky, exit 0.
yarn test:ci— 1774 passed, 51 skipped, 0 failed across 220 suites.yarn build:extension— exit 0.**/e2e-tests/**/*.missing, nothing unread — checked by enumerating
screenshot:references per testfile and diffing against directory contents.
sign-transaction,account-history,network-form-page,recovery-modal) to confirm they capturefully-rendered screens rather than loading states.
unrelated image of the same dimensions for one baseline, which failed as expected.
Follow-ups / out of scope
expectPageToHaveScreenshotpassesmaxDiffPixelRatio: 0.02— roughly 18k pixels of a 1280×720 shot — and Playwright's per-pixel
thresholddefaults to 0.2. Left alone on purpose: baselines are generated on developer
machines and compared on CI's macOS runner, so some of that headroom is probably
absorbing font-rendering differences, and tightening it is a team call with real
flake risk.
addAsset.test.ts-snapshots/manage-assets-verified-token-chromium-darwin.pngisreferenced only by a
test.fixmetest (navigation after "Manage Assets" does notcomplete reliably), so it never executes. It is kept rather than deleted, since
the reference still exists, but it remains stale and will need regenerating
whenever that test is fixed.
🤖 Generated with Claude Code