[Bug]: button click timeout (headless WebKit on Ubuntu 24.04.1 LTS)

[johnnygerard]

Version

1.48.0

Steps to reproduce

1. Launch a fresh instance of Ubuntu 24.04.1 LTS (see Additional context for details)
2. Run bug-playwright-button-click/init.sh (uses nodesource instructions for Node.js 20 installation)

Expected behavior

All tests should pass.

Actual behavior

Some tests fail:

/usr/bin/npm run e2e

> bug-playwright-button-click@1.0.0 e2e
> playwright test


Running 9 tests using 4 workers
  1) [webkit] › button-user-interactions.spec.ts:12:7 › Button user interactions › Click ───────────

    Test timeout of 5000ms exceeded.

    Error: locator.click: Test timeout of 5000ms exceeded.
    Call log:
      - waiting for getByLabel(/light mode/i)
      -   locator resolved to <button disabled type="button" aria-live="polite" class="clean-btn toggleButton_gllP toggleButtonDisabled_aARS" title="Switch between dark and light mode (currently dark mode)" aria-label="Switch between dark and light mode (currently dark mode)">…</button>
      - attempting click action
      -   waiting for element to be visible, enabled and stable


      11 |
      12 |   test("Click", async ({ page }) => {
    > 13 |     await button.click();
         |                  ^
      14 |   });
      15 |
      16 |   test("Press enter", async ({ page }) => {

        at ~/bug-playwright-button-click/tests/button-user-interactions.spec.ts:13:18

  2) [webkit] › button-user-interactions.spec.ts:20:7 › Button user interactions › Tap ─────────────

    Test timeout of 5000ms exceeded.

    Error: locator.tap: Test timeout of 5000ms exceeded.
    Call log:
      - waiting for getByLabel(/light mode/i)
      -   locator resolved to <button disabled type="button" aria-live="polite" class="clean-btn toggleButton_gllP toggleButtonDisabled_aARS" title="Switch between dark and light mode (currently dark mode)" aria-label="Switch between dark and light mode (currently dark mode)">…</button>
      - attempting tap action
      -   waiting for element to be visible, enabled and stable


      19 |
      20 |   test("Tap", async ({ page }) => {
    > 21 |     await button.tap();
         |                  ^
      22 |   });
      23 | });
      24 |

        at ~/bug-playwright-button-click/tests/button-user-interactions.spec.ts:21:18

  2 failed
    [webkit] › button-user-interactions.spec.ts:12:7 › Button user interactions › Click ────────────
    [webkit] › button-user-interactions.spec.ts:20:7 › Button user interactions › Tap ──────────────
  7 passed (9.9s)

  Serving HTML report at http://localhost:9323. Press Ctrl+C to quit.

Additional context

The tests only fail in specific conditions: only WebKit in --headless or --ui mode in Ubuntu 24.04.1 LTS (on my current machine and also on a fresh install).

I tried various combinations:

• Tests pass on Ubuntu 22 LTS but fail on Ubuntu 24 LTS.
• CI tests pass with GitHub Actions (ubuntu-latest still uses Ubuntu 22 LTS).
• Tests always pass when run in --headed mode regardless of Ubuntu version.
• Tests pass when running on WSL and multipass with Ubuntu 24.04.1 LTS (maybe because of the lack of a desktop environment or differences in the installed packages).

A simple workaround is to use button.press("Enter") instead of button.click() and button.tap().

Environment

System:
OS: Linux 6.8 Ubuntu 24.04.1 LTS 24.04.1 LTS (Noble Numbat)
CPU: (8) x64 Intel(R) Core(TM) i3-10100 CPU @ 3.60GHz
Memory: 10.36 GB / 15.24 GB
Container: Yes
Binaries:
Node: 20.18.0 - /usr/bin/node
npm: 10.9.0 - /usr/bin/npm
pnpm: 9.12.1 - /usr/bin/pnpm
Languages:
Bash: 5.2.21 - /usr/bin/bash
npmPackages:
@playwright/test: ^1.48.0 => 1.48.0

[jaugustin]

We have similar issue on our tests suite, most tests fail with ubuntu 24.04 when it works on 22.04 or mac.

[dgozman]

@johnnygerard Thank you for the detailed repro! I tried to reproduce this on Ubuntu 24.04 LTS (I don't have 24.04.1 readily available),@playwright/test@1.48.0 and node v20.15.0. Unfortunately, all tests pass for me in all browsers, with and without --headed or --ui, running multiple times. Any ideas on what could be different? Anything specific in your setup?

[johnnygerard]

@dgozman With the ubuntu-24.04.1-desktop-amd64.iso image installed on a USB stick, I used the "Try Ubuntu" feature before running the test script.

If you use the same approach only the hardware could make a difference.
I couldn't find your Ubuntu version.

[johnnygerard]

I got the same test results on ubuntu-24.04-desktop-amd64.iso (retrieved from Old Ubuntu Releases).
@dgozman Did you use a different Ubuntu image?

[dgozman]

@johnnygerard Have you tried debugging the failure, recording a trace? Since I am not able to reproduce, any information from your side would be helpful. Perhaps you've got some kind of proxy? Maybe the computer+OS combination is so slow that it does not load in 5000ms?

[johnnygerard]

I've updated bug-playwright-button-click GitHub Actions workflow to run with ubuntu-24.04 and the tests pass (see results).

Here is the report of the last failed test on my machine using a timeout of 30s: playwright-report.zip.
Run npx playwright show-report where the extracted folder is to view the report.

In debug mode (npx playwright test --debug), all tests pass because it runs in headed mode.
I don't use a proxy of any kind.

[dgozman]

@johnnygerard One idea I have is this could be related to prefers-color-scheme support. Trace shows that "light/dark mode switch" is disabled, perhaps because docusaurus failed to detect default color scheme? Does this webkit problem reproduce with other tests, not on playwright.dev?

[johnnygerard]

@dgozman I updated bug-playwright-button-click to test a noop button from GitHub Pages. I get the same results.

[dgozman]

@johnnygerard Could you also attach the trace/report? Since I cannot reproduce, I solely rely on your artifacts.

[johnnygerard]

@dgozman playwright-report.zip

[dgozman]

@johnnygerard Thank you! Could you also run with DEBUG=pw:protocol and share the report please? Something is definitely stuck there, perhaps rafs are not firing? I hope that protocol log would help.

[johnnygerard]

@dgozman playwright-report.zip with DEBUG=pw:protocol

[mxschmitt]

I tried to reproduce on amd and arm64 Ubuntu 24.04.1

docker run -it mcr.microsoft.com/playwright:v1.48.1-noble /bin/bash

but was not able to.

[dgozman]

After looking at the logs, it seems like the issue is with rafs not firing, specific to WPE webkit headless backend. There's been similar issues with WPE, for example this one https://bugs.webkit.org/show_bug.cgi?id=255682. Let's see what can be done upstream.