Skip to content

[Chore] VRT fixes#9721

Open
weronikaolejniczak wants to merge 14 commits into
elastic:mainfrom
weronikaolejniczak:fix/vrt-mobile
Open

[Chore] VRT fixes#9721
weronikaolejniczak wants to merge 14 commits into
elastic:mainfrom
weronikaolejniczak:fix/vrt-mobile

Conversation

@weronikaolejniczak

@weronikaolejniczak weronikaolejniczak commented Jun 5, 2026

Copy link
Copy Markdown
Contributor

Summary

Several issues after having merged #9594.

Still a couple of flakes

I think some of the flakes like EuiPopover Playground might be caused by font subpixel positioning and there's a Chromium flag to disable it: --disable-font-subpixel-positioning.

For now, I want to skip these flakes so that they are not blocking PRs:

  • EuiPopover > Playground
  • EuiDataGrid > Column selector (similar as ☝🏻)
  • EuiDataGrid > Compact
  • EuiCodeBlock > Virtualized code block scrolling

Lack of mobile snapshots

Turns out playwright.config.ts isn't picked up by the test-runner and is dead code but what's more curious is there's conditional logic for the viewport in the test runner. Not sure what I was thinking at the time.

Added an iteration through "variants". Called them generically because although it's only viewport for now, we may extend it (dark mode, high contrast mode etc.).

No add commit on failed VRT step

If the VRT step fails, we do not add chore(eui): add VRT baseline screenshots, we fold the new snapshots under chore(eui): update VRT baseline screenshots.

So PRs that mixed new stories with diffs on existing ones OR encountered flakes silently lost the new baselines (e.g. PaginationExample on #9726). Now we commit new baselines regardless of VRT outcome.

Cancel intermediate builds

You can read more about it here. The idea is to cancel the original build which keeps running in parallel and will e.g. comment stale visual diff.

QA instructions for reviewer

  • VRT is ran both for desktop and mobile
  • Wiki is clear and mentions: how it works in CI, running locally, skipping stories, adding variants
  • VRT baselines for mobile are added
  • Baselines are added regardless of success state
  • Intermediate builds are cancelled
  • After re-running the CI at least 5 times there are no flakes
  • VRT can be ran locally and works as expected (e.g. there are no flakes)
  • Skipping the whole story works: skip: true
  • Skipping a variant works: skip: ['mobile']
  • Scan all the snapshots and highlight anything that doesn't look correct, I'll fix it in a follow-up with the flakes

@weronikaolejniczak weronikaolejniczak self-assigned this Jun 5, 2026
Copilot AI review requested due to automatic review settings June 5, 2026 15:23
@weronikaolejniczak weronikaolejniczak requested a review from a team as a code owner June 5, 2026 15:23
@weronikaolejniczak weronikaolejniczak added the skip-changelog Use on PRs to skip changelog requirement (Don't delete - used for automation) label Jun 5, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates EUI’s Storybook visual regression test runner to snapshot each story across multiple “variants” (currently desktop + mobile viewports), and documents how variant baselines are named and how to opt out per-story.

Changes:

  • Run VRT screenshots in a loop over defined variants (desktop/mobile) and suffix baseline names with the variant (e.g. …-desktop.png, …-mobile.png).
  • Remove packages/eui/playwright.config.ts, which is not used by the current test-storybook-based VRT runner.
  • Expand the VRT wiki docs with variant behavior and parameters.vrt.skipVariants.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

File Description
wiki/contributing-to-eui/testing/visual-regression-testing.md Documents variant baselines and how to skip variants per story
packages/eui/playwright.config.ts Removes unused Playwright config file
packages/eui/.storybook/test-runner.ts Implements per-variant viewport iteration and per-variant snapshot IDs

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread wiki/contributing-to-eui/testing/visual-regression-testing.md Outdated
@weronikaolejniczak weronikaolejniczak marked this pull request as draft June 5, 2026 16:19
@weronikaolejniczak weronikaolejniczak force-pushed the fix/vrt-mobile branch 5 times, most recently from e26d9ec to 237fa42 Compare June 18, 2026 14:23
@weronikaolejniczak weronikaolejniczak marked this pull request as ready for review June 18, 2026 14:32
@weronikaolejniczak weronikaolejniczak marked this pull request as draft June 18, 2026 14:49
@weronikaolejniczak weronikaolejniczak force-pushed the fix/vrt-mobile branch 4 times, most recently from 63173bf to 99f4f96 Compare June 19, 2026 13:32
@weronikaolejniczak weronikaolejniczak changed the title [Chore] Run VRT per variant [Chore] VRT fixes Jun 25, 2026
@elastic elastic deleted a comment from infra-vault-gh-plugin-prod Bot Jun 25, 2026
@elastic-vault-github-plugin-prod

Copy link
Copy Markdown
2 visual difference(s) found - expand to review, then click Approve visual changes to update baselines

euicomment (1 difference)

StoryBeforeAfterDiff
high contrast desktop

euidatagrid (1 difference)

StoryBeforeAfterDiff
additional controls options desktop

@weronikaolejniczak weronikaolejniczak marked this pull request as ready for review June 29, 2026 14:51
@elastic elastic deleted a comment from infra-vault-gh-plugin-prod Bot Jun 29, 2026
@weronikaolejniczak

weronikaolejniczak commented Jun 29, 2026

Copy link
Copy Markdown
Contributor Author

buildkite test this 1 (after green)

@weronikaolejniczak

weronikaolejniczak commented Jun 30, 2026

Copy link
Copy Markdown
Contributor Author

buildkite test this 2 (after green)

@weronikaolejniczak

weronikaolejniczak commented Jun 30, 2026

Copy link
Copy Markdown
Contributor Author

buildkite test this 3 (after green)

@elastic elastic deleted a comment from infra-vault-gh-plugin-prod Bot Jun 30, 2026
@weronikaolejniczak

weronikaolejniczak commented Jun 30, 2026

Copy link
Copy Markdown
Contributor Author

buildkite test this 4 (after green)

Comment thread wiki/contributing-to-eui/testing/visual-regression-testing.md Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot wasn't able to review this pull request because it exceeds the maximum number of files (300). Try reducing the number of changed files and requesting a review from Copilot again.

@mgadewoll mgadewoll self-requested a review June 30, 2026 13:52
# encounters a story without one. Commit those net-new baselines now,
# *regardless* of whether VRT overall passed or failed.
new_files="$(git ls-files --others --exclude-standard -- "${REF_DIR}")"
if [[ -n "${new_files}" ]]; then

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good thinking, this is much nicer 🎉

@@ -159,7 +159,7 @@ export const VirtualizedCodeBlockScrolling: Story = {
tags: ['vrt-only'],
parameters: {
vrt: {
selector: VRT_SELECTORS.portal,
skip: true,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❓ I understand that the purpose is to skip these stories for now and get back to them.
Do you think it's valid to add a comment stating this anyway according to our convention, or will this be done in the very near future and not meaningful anyway?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think it's necessary because my plan is to immediately try to stabilize these after this PR is merged. So skips are temporary.

@elastic-vault-github-plugin-prod

Copy link
Copy Markdown
6 visual difference(s) found - expand to review, then click Approve visual changes to update baselines

euisuperdatepicker (6 differences)

StoryBeforeAfterDiff
custom time zone display desktop
custom time zone display mobile
overflowing children desktop
overflowing children mobile
quick select only desktop
quick select only mobile

@elastic-vault-github-plugin-prod

Copy link
Copy Markdown
6 visual difference(s) found - expand to review, then click Approve visual changes to update baselines

euisuperdatepicker (6 differences)

StoryBeforeAfterDiff
custom time zone display desktop
custom time zone display mobile
overflowing children desktop
overflowing children mobile
quick select only desktop
quick select only mobile

@infra-vault-gh-plugin-prod

Copy link
Copy Markdown

💚 Build Succeeded

History

cc @weronikaolejniczak

@infra-vault-gh-plugin-prod

Copy link
Copy Markdown

💚 Build Succeeded

History

cc @weronikaolejniczak

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

infrastructure skip-changelog Use on PRs to skip changelog requirement (Don't delete - used for automation)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants