Add e2e playwright tests by bramkragten · Pull Request #51929 · home-assistant/frontend

bramkragten · 2026-05-08T13:34:49Z

Proposed change

Adds playwright tests covering demo/gallery and new test app with multiple scenarios.

Screenshots

Type of change

Dependency upgrade
Bugfix (non-breaking change which fixes an issue)
New feature (thank you!)
Breaking change (fix/feature causing existing functionality to break)
Code quality improvements to existing code or addition of tests

Additional information

This PR fixes or closes issue: fixes #
This PR is related to issue or discussion:
Link to documentation pull request:
Link to developer documentation pull request:
Link to backend pull request:

Checklist

I understand the code I am submitting and can explain how it works.
The code change is tested and works locally.
There is no commented out code in this PR.
I have followed the perfect PR recommendations
Any generated code has been carefully reviewed for correctness and compliance with project standards.

If user exposed functionality or configuration variables are added/changed:

Documentation added/updated for www.home-assistant.io

To help with the load of incoming pull requests:

I have reviewed two other open pull requests in this repository.

Copilot

Pull request overview

This PR introduces Playwright-based end-to-end (e2e) testing to the Home Assistant frontend, adding coverage for the demo and gallery builds plus a dedicated “e2e test app” with controllable scenarios, and wiring it into CI.

Changes:

Add Playwright test suites + configs for demo, gallery, and a new e2e test app (with local + BrowserStack execution modes).
Add build pipeline support for the e2e test app (rspack + gulp tasks, static gathering, HTML entry generation).
Add a GitHub Actions workflow to build artifacts once and run e2e tests locally (Chromium) and on BrowserStack, then merge reports.

Reviewed changes

Copilot reviewed 32 out of 35 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
yarn.lock	Adds Playwright dependencies to the lockfile.
tsconfig.json	Excludes `test/e2e` from the main TypeScript project.
test/e2e/tsconfig.json	Adds a Node-typed TS config for e2e tests.
test/e2e/playwright.merge.config.ts	Defines merged-report reporters (HTML/JSON).
test/e2e/playwright.gallery.config.ts	Playwright config for gallery tests (local/BrowserStack + webServer).
test/e2e/playwright.demo.config.ts	Playwright config for demo tests (local/BrowserStack + webServer).
test/e2e/playwright.app.config.ts	Playwright config for the dedicated e2e test app.
test/e2e/gallery.spec.ts	Adds gallery page-smoke and a few interaction tests.
test/e2e/demo.spec.ts	Adds demo app smoke/navigation/more-info tests.
test/e2e/browserstack.capabilities.ts	Adds BrowserStack capability matrix used by configs.
test/e2e/app/src/scenarios/index.ts	Adds scenario hooks to mutate the mock hass for different test cases.
test/e2e/app/src/html/index.html.template	Adds the e2e test app HTML template.
test/e2e/app/src/ha-test.ts	Implements the `ha-test` app element backed by `provideHass` + stubs.
test/e2e/app/src/ha-test-panels.ts	Defines the panel map used by the e2e test app.
test/e2e/app/src/entrypoint.ts	Entry point importing `ha-test`.
test/e2e/app.spec.ts	Adds tests targeting the e2e test app scenarios/panels.
src/fake_data/provide_hass.ts	Adds a `formatEntityName` implementation to the mock hass object.
package.json	Adds Playwright and yarn scripts for running/merging e2e suites.
demo/src/stubs/update.ts	Adds WS stubs for update panel calls.
demo/src/stubs/cloud.ts	Adds REST stub for cloud status to keep config UI from erroring.
demo/src/stubs/assist.ts	Adds WS stubs for Assist pipeline list/run for developer tools.
build-scripts/rspack.cjs	Exposes a rspack config factory for the e2e test app.
build-scripts/paths.cjs	Adds e2e test app build/output paths.
build-scripts/gulp/rspack.js	Adds rspack dev server + prod build tasks for e2e test app.
build-scripts/gulp/index.mjs	Registers the e2e test app gulp tasks.
build-scripts/gulp/gather-static.js	Adds static asset gathering for e2e test app output.
build-scripts/gulp/entry-html.js	Adds HTML generation tasks for the e2e test app.
build-scripts/gulp/e2e-test-app.js	New gulp orchestration tasks for develop/build of the e2e test app.
build-scripts/gulp/clean.js	Adds a clean task for the e2e test app outputs.
build-scripts/get-built-in-node-module-shim.cjs	Removes a global comment header (no functional change).
build-scripts/eslint.config.mjs	Configures Node globals for build-scripts linting.
build-scripts/bundle.cjs	Adds bundle config for the e2e test app (entry/defines/output).
.serena/project.yml	Adds Serena project configuration.
.serena/.gitignore	Ignores Serena cache/local config files.
.gitignore	Ignores Playwright reports/test-results and e2e app build output.
.github/workflows/e2e.yaml	Adds CI workflow to build artifacts, run e2e, and merge reports.

silamon · 2026-05-08T17:51:59Z

Nice work! 👍

…ontext limit

…-import errors

…For on mobile

…lectors for iPhone

…nnel

…elector on iOS

…r can't pierce shadow DOM

…est for iOS

…rsal

…-and-skip

…ternal error

… calls

MindFreeze · 2026-05-20T10:38:30Z

+  # ── Run Playwright tests on BrowserStack (real devices + browsers) ─────────
+  # The BrowserStack SDK manages the Local tunnel and uploads results to the
+  # BrowserStack Automate dashboard automatically — no tunnel action needed.
+  e2e-browserstack:


Weren't we going to run this only on demand?

MindFreeze · 2026-05-20T10:41:12Z

+ * E2E tests for the HA test app (port 8095).
+ *
+ * Run with:
+ *   yarn test:e2e:app:local


Suggested change

* yarn test:e2e:app:local

* yarn test:e2e:app

MindFreeze · 2026-05-20T10:41:24Z

+ * the demo content renders without JS errors and the page element is visible.
+ *
+ * Run with:
+ *   yarn test:e2e:gallery:local


Suggested change

* yarn test:e2e:gallery:local

* yarn test:e2e:gallery

home-assistant Bot added the cla-signed label May 8, 2026

github-actions Bot added GitHub Actions Pull requests that update GitHub Actions code Build Related to building the code Demo Related to frontend demo content Design Related to Home Assistant design gallery labels May 8, 2026

github-advanced-security AI found potential problems May 8, 2026

View reviewed changes

Comment thread test/e2e/app/src/ha-test.ts Dismissed

bramkragten requested a review from Copilot May 8, 2026 13:38

Copilot started reviewing on behalf of bramkragten May 8, 2026 13:39 View session

bramkragten had a problem deploying to browserstack May 8, 2026 13:39 — with GitHub Actions Failure

Copilot AI reviewed May 8, 2026

View reviewed changes

Comment thread package.json Outdated

Comment thread package.json Outdated

Comment thread test/e2e/gallery.spec.ts

Comment thread test/e2e/app.spec.ts Outdated

Comment thread test/e2e/app/src/ha-test.ts

Comment thread test/e2e/app/src/ha-test.ts

bramkragten had a problem deploying to browserstack May 8, 2026 13:49 — with GitHub Actions Failure

Add e2e playwright tests

908a518

bramkragten force-pushed the e2e-playwright-tests branch from 7b1f0b0 to 908a518 Compare May 8, 2026 13:57

bramkragten had a problem deploying to browserstack May 8, 2026 14:03 — with GitHub Actions Failure

bramkragten added 2 commits May 8, 2026 16:05

Update e2e.yaml

2024ce0

fixes

9aa6cd4

bramkragten had a problem deploying to browserstack May 8, 2026 14:17 — with GitHub Actions Failure

fix browserstack tests

a21cf5d

bramkragten had a problem deploying to browserstack May 8, 2026 14:27 — with GitHub Actions Error

fix

d32f5b6

bramkragten had a problem deploying to browserstack May 8, 2026 14:56 — with GitHub Actions Failure

bramkragten added 2 commits May 10, 2026 23:03

fix gallery tests

6eadf2f

fix browserstack ios tests

897a339

bramkragten had a problem deploying to browserstack May 10, 2026 21:16 — with GitHub Actions Error

Update browserstack.capabilities.ts

764961b

bramkragten had a problem deploying to browserstack May 10, 2026 21:23 — with GitHub Actions Error

use bs-local.com for browserstack

c3358e0

bramkragten had a problem deploying to browserstack May 10, 2026 21:46 — with GitHub Actions Failure

Use browserstack node SDK instead

e32dd78

bramkragten closed this May 10, 2026

bramkragten reopened this May 10, 2026

bramkragten added 13 commits May 13, 2026 10:11

test(e2e): use serial mode + shared page to fix BrowserStack iPhone c…

0ccc582

…ontext limit

test(e2e): reuse pre-existing BrowserStack page; filter infra dynamic…

99c7a42

…-import errors

test(e2e): fix WebKit dynamic-import filter and avoid concurrent wait…

d16aa48

…For on mobile

test(e2e): replace class-wildcard selector with explicit hui-* tag se…

efe41b1

…lectors for iPhone

test(e2e): skip cards/dialog checks when dynamic imports fail over tu…

5ca28bd

…nnel

test(e2e): use recursive shadow DOM evaluate poll instead of waitForS…

8b41964

…elector on iOS

test(e2e): fix evaluate argument serialization on BrowserStack iOS

3fbc114

test(e2e): use evaluate for card visibility check on iOS where locato…

92e6f69

…r can't pierce shadow DOM

test(e2e): replace waitForSelector with evaluate polling in sidebar t…

8905123

…est for iOS

test(e2e): fix waitForShadow helper to use recursive shadow DOM trave…

ce5a135

…rsal

test(e2e): revert to locator.waitFor with BS iOS Internal error catch…

d355d9e

…-and-skip

test(e2e): skip on both dynamic-import errors and BrowserStack iOS In…

141dce9

…ternal error

test(e2e): handle skip return from waitForLocator in all sidebar wait…

d3c6897

… calls

MindFreeze reviewed May 20, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add e2e playwright tests#51929

Add e2e playwright tests#51929
bramkragten wants to merge 23 commits into
devfrom
e2e-playwright-tests

bramkragten commented May 8, 2026

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

silamon commented May 8, 2026

Uh oh!

MindFreeze May 20, 2026

Uh oh!

MindFreeze May 20, 2026

Uh oh!

MindFreeze May 20, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Conversation

bramkragten commented May 8, 2026

Proposed change

Screenshots

Type of change

Additional information

Checklist

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

silamon commented May 8, 2026

Uh oh!

MindFreeze May 20, 2026

Choose a reason for hiding this comment

Uh oh!

MindFreeze May 20, 2026

Choose a reason for hiding this comment

Uh oh!

MindFreeze May 20, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants