Skip to content

test: fix electron bots#41170

Merged
dgozman merged 2 commits into
microsoft:mainfrom
dgozman:test-fix-electron-bots
Jun 8, 2026
Merged

test: fix electron bots#41170
dgozman merged 2 commits into
microsoft:mainfrom
dgozman:test-fix-electron-bots

Conversation

@dgozman

@dgozman dgozman commented Jun 5, 2026

Copy link
Copy Markdown
Collaborator

@dgozman
test: fix electron bots
c2b7678 
- Update to electron 42.3.3.
- New electron does not install by default, so call `npx install-electron --no`
  and remove `ELECTRON_SKIP_BINARY_DOWNLOAD` env.
  See https://www.electronjs.org/docs/latest/breaking-changes#behavior-changed-electron-no-longer-downloads-itself-via-postinstall-script.
- Skip some tests that behave differently on Electron.
- Move tests that call `context.route()` out of page tests.
@dgozman dgozman added the CQ1 label Jun 5, 2026
@github-actions

This comment has been minimized.

Sign in to view
@github-actions

This comment has been minimized.

Sign in to view
@github-actions

github-actions Bot commented Jun 5, 2026

Copy link
Copy Markdown
Contributor

Test results for "tests others"

1 flaky ⚠️ [chromium-library] › library/video.spec.ts:275 › screencast › should capture navigation `@frozen-time-library-chromium-linux`

19962 passed, 670 skipped

Merge workflow run.

@github-actions

github-actions Bot commented Jun 5, 2026

Copy link
Copy Markdown
Contributor

Test results for "tests 2"

29 failed
❌ [installation tests] › playwright-electron-should-work.spec.ts:21 › electron should work @package-installations-ubuntu-latest-node26
❌ [installation tests] › playwright-electron-should-work.spec.ts:32 › electron should work with special characters in path @package-installations-ubuntu-latest-node26
❌ [installation tests] › playwright-electron-should-work.spec.ts:46 › should work when wrapped inside @playwright/test and trace is enabled @package-installations-ubuntu-latest-node26
❌ [chromium-library] › library/heap.spec.ts:169 › should collect frames @tracing-chromium
❌ [chromium-page] › page/workers.spec.ts:190 › should attribute network activity for worker inside iframe to the iframe @tracing-chromium
❌ [installation tests] › playwright-electron-should-work.spec.ts:21 › electron should work @package-installations-ubuntu-latest-node24
❌ [installation tests] › playwright-electron-should-work.spec.ts:32 › electron should work with special characters in path @package-installations-ubuntu-latest-node24
❌ [installation tests] › playwright-electron-should-work.spec.ts:46 › should work when wrapped inside @playwright/test and trace is enabled @package-installations-ubuntu-latest-node24
❌ [firefox-library] › library/heap.spec.ts:203 › cycle handles @tracing-firefox
❌ [firefox-library] › library/browsercontext-base-url.spec.ts:37 › should construct a new URL when a baseURL in browserType.launchPersistentContext is passed to page.goto @firefox-macos-15-large
❌ [firefox-library] › library/defaultbrowsercontext-2.spec.ts:113 › should restore state from userDataDir @firefox-macos-15-large
❌ [firefox-library] › library/defaultbrowsercontext-2.spec.ts:140 › should create userDataDir if it does not exist @firefox-macos-15-large
❌ [firefox-library] › library/proxy.spec.ts:237 › should exclude patterns @firefox-macos-15-large
❌ [webkit-page] › page/page-leaks.spec.ts:136 › expect should not leak @tracing-webkit
❌ [webkit-library] › library/har-websocket.spec.ts:80 › should include websocket handshake headers and status @webkit-windows-latest
❌ [webkit-library] › library/har-websocket.spec.ts:118 › should include websocket messages @webkit-windows-latest
❌ [webkit-library] › library/har-websocket.spec.ts:157 › should include larger websocket messages @webkit-windows-latest
❌ [webkit-library] › library/har-websocket.spec.ts:196 › should include gigantic websocket messages @webkit-windows-latest
❌ [webkit-library] › library/har-websocket.spec.ts:235 › should include binary websocket messages @webkit-windows-latest
❌ [webkit-library] › library/har-websocket.spec.ts:338 › should record websocket handshake failure @webkit-windows-latest
❌ [android-native] › android/device.spec.ts:56 › androidDevice.fill
❌ [android-native] › android/webview.spec.ts:44 › should navigate page externally
❌ [android-page] › page/page-autowaiting-basic.spec.ts:94 › should work with noWaitAfter: true
❌ [android-page] › page/page-autowaiting-basic.spec.ts:100 › should work with dblclick without noWaitAfter when navigation is stalled
❌ [android-page] › page/page-dialog.spec.ts:67 › should be able to close context with open alert
❌ [android-page] › page/page-goto.spec.ts:373 › should fail when exceeding maximum navigation timeout
❌ [android-page] › page/page-goto.spec.ts:441 › should prioritize default navigation timeout over default timeout
❌ [android-page] › page/page-request-continue.spec.ts:125 › should not allow changing protocol when overriding url
❌ [android-page] › page/page-set-content.spec.ts:153 › should handle timeout properly 2

35 flaky ⚠️ [chromium-page] › page/page-route.spec.ts:290 › should be abortable `@msedge-dev-windows-latest`
⚠️ [chromium-library] › library/chromium/chromium.spec.ts:371 › should produce network events, routing, and annotations for Service Worker `@chrome-ubuntu-22.04`
⚠️ [chromium-library] › library/inspector/cli-codegen-javascript.spec.ts:36 › should print the correct context options for custom settings `@chrome-ubuntu-22.04`
⚠️ [chromium-library] › library/defaultbrowsercontext-1.spec.ts:28 › context.cookies() should work @smoke `@msedge-windows-latest`
⚠️ [chromium-library] › library/inspector/cli-codegen-3.spec.ts:897 › cli codegen › should keep dialog open when clicking inside `@msedge-windows-latest`
⚠️ [chromium-library] › library/trace-viewer.spec.ts:682 › should capture attribute mutations inside a popup window `@chromium-windows-latest`
⚠️ [chromium-library] › library/browsercontext-fetch-algorithms.spec.ts:53 › algorithms › br decompression › should not fail if response content-length header is missing (br) `@chromium-macos-15-large`
⚠️ [chromium-library] › library/browsertype-connect.spec.ts:189 › launchServer › should ignore page.pause when headed `@chromium-macos-15-large`
⚠️ [chromium-library] › library/chromium/chromium.spec.ts:177 › serviceWorker(), and fromServiceWorker() work `@chromium-macos-15-large`
⚠️ [chromium-library] › library/chromium/chromium.spec.ts:211 › should intercept service worker requests (main and within) `@chromium-macos-15-large`
⚠️ [chromium-library] › library/heap.spec.ts:203 › cycle handles `@chromium-macos-15-large`
⚠️ [chromium-library] › library/browsertype-connect.spec.ts:189 › launchServer › should ignore page.pause when headed `@chromium-macos-15-xlarge`
⚠️ [chromium-library] › library/screenshot.spec.ts:215 › page screenshot › should not hang when event loop is blocked `@chromium-macos-15-xlarge`
⚠️ [chromium-library] › library/video.spec.ts:647 › screencast › should capture full viewport `@chromium-macos-15-xlarge`
⚠️ [chromium-library] › library/video.spec.ts:275 › screencast › should capture navigation `@chrome-beta-ubuntu-22.04`
⚠️ [firefox-library] › library/beforeunload.spec.ts:20 › should close browser with beforeunload page `@firefox-macos-15-large`
⚠️ [firefox-library] › library/browser.spec.ts:54 › should dispatch page.on(close) upon browser.close and reject evaluate `@firefox-macos-15-large`
⚠️ [firefox-library] › library/browsercontext-basic.spec.ts:36 › should be able to click across browser contexts `@firefox-macos-15-large`
⚠️ [firefox-library] › library/defaultbrowsercontext-2.spec.ts:147 › should have default URL when launching browser `@firefox-macos-15-large`
⚠️ [firefox-library] › library/defaultbrowsercontext-2.spec.ts:161 › should have passed URL when launching with ignoreDefaultArgs: true `@firefox-macos-15-large`
⚠️ [firefox-library] › library/defaultbrowsercontext-2.spec.ts:183 › should handle exception `@firefox-macos-15-large`
⚠️ [firefox-library] › library/defaultbrowsercontext-2.spec.ts:190 › should fire close event for a persistent context `@firefox-macos-15-large`
⚠️ [firefox-library] › library/defaultbrowsercontext-2.spec.ts:234 › should support har option `@firefox-macos-15-large`
⚠️ [firefox-library] › library/har.spec.ts:109 › should populate entry startedDateTime from the browser `@firefox-macos-15-large`
⚠️ [firefox-library] › library/inspector/cli-codegen-csharp.spec.ts:202 › should not print context options method override in mstest if no options were passed `@firefox-macos-15-large`
⚠️ [firefox-library] › library/inspector/cli-codegen-java.spec.ts:125 › should print a valid basic program in junit `@firefox-macos-15-large`
⚠️ [firefox-library] › library/inspector/cli-codegen-javascript.spec.ts:84 › should save the codegen output to a file if specified `@firefox-macos-15-large`
⚠️ [firefox-library] › library/inspector/cli-codegen-python-async.spec.ts:80 › should save the codegen output to a file if specified `@firefox-macos-15-large`
⚠️ [firefox-library] › library/inspector/cli-codegen-python.spec.ts:76 › should save the codegen output to a file if specified `@firefox-macos-15-large`
⚠️ [firefox-library] › library/browsercontext-reuse.spec.ts:200 › reuse connect › should not cache resources `@firefox-windows-latest`
⚠️ [firefox-library] › library/browsercontext-basic.spec.ts:163 › should propagate default viewport to the page `@firefox-macos-15-xlarge`
⚠️ [webkit-page] › page/page-click.spec.ts:1203 › should fire contextmenu event on right click in correct order `@tracing-webkit`
⚠️ [webkit-page] › page/page-set-content.spec.ts:153 › should handle timeout properly 2 `@webkit-windows-latest`
⚠️ [webkit-library] › library/video.spec.ts:371 › screencast › should scale frames down to the requested size `@webkit-macos-15-large`
⚠️ [webkit-page] › page/page-request-continue.spec.ts:194 › post data › should compute content-length from post data `@webkit-macos-15-large`

88736 passed, 4113 skipped, 1 did not run

Merge workflow run.

@dgozman
test: pin yauzl to fix electron install on bots
2b2e0d2 
Electron 42's extract-zip pulls in an incompatible yauzl on newer
Node.js, breaking install. Pin yauzl to ^3.3.2 via package.json overrides.

electron/electron#51619
@github-actions

github-actions Bot commented Jun 8, 2026

Copy link
Copy Markdown
Contributor

Test results for "MCP"

7266 passed, 1119 skipped

Merge workflow run.

@github-actions

github-actions Bot commented Jun 8, 2026

Copy link
Copy Markdown
Contributor

Test results for "tests 1"

6 flaky ⚠️ [chromium-library] › library/video.spec.ts:719 › screencast › should work with video+trace `@chromium-ubuntu-22.04-arm-node20`
⚠️ [chromium-library] › library/beforeunload.spec.ts:130 › should support dismissing the dialog multiple times `@chromium-ubuntu-22.04-node24`
⚠️ [chromium-library] › library/video.spec.ts:682 › screencast › should capture full viewport on hidpi `@chromium-ubuntu-22.04-node20`
⚠️ [chromium-library] › library/video.spec.ts:719 › screencast › should work with video+trace `@chromium-ubuntu-22.04-node20`
⚠️ [chromium-library] › library/video.spec.ts:682 › screencast › should capture full viewport on hidpi `@chromium-ubuntu-22.04-node22`
⚠️ [playwright-test] › ui-mode-trace.spec.ts:189 › should show snapshots for steps `@windows-latest-node22`

39504 passed, 774 skipped

Merge workflow run.

yury-s
yury-s approved these changes Jun 8, 2026
View reviewed changes
Comment thread tests/installation/playwright-electron-should-work.spec.ts
}

test('electron should work', async ({ exec, tsc, writeFiles, tmpWorkspace }) => {
await pinYauzl(tmpWorkspace);

@yury-s yury-s Jun 8, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How is this supposed to work for our users, are they gonna need the same?

@dgozman dgozman merged commit a545f4d into microsoft:main Jun 8, 2026
49 of 51 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@yury-s yury-s yury-s approved these changes

Assignees

No one assigned

Labels

CQ1

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@dgozman @yury-s