Description
Context:
- GOOD Playwright Version: 1.37.1
- BAD Playwright Version: 1.39.0
- Operating System: Linux (Mint 20.3 locally, Ubuntu on GH Actions)
- Extra: Node v18.15.0
Code Snippet
This is my config file:
import { defineConfig, devices } from "@playwright/test";
/**
* Read environment variables from file.
* https://github.com/motdotla/dotenv
*/
// require('dotenv').config();
/**
* See https://playwright.dev/docs/test-configuration.
*/
export default defineConfig({
timeout: 7500,
testDir: "./src/test/e2e",
/* Run tests in files in parallel */
fullyParallel: true,
/* Fail the build on CI if you accidentally left test.only in the source code. */
forbidOnly: !!process.env.CI,
/* Retry on CI only */
retries: process.env.CI ? 10 : 0,
/* Opt out of parallel tests on CI. */
workers: process.env.CI ? 1 : undefined,
/* Reporter to use. See https://playwright.dev/docs/test-reporters */
reporter: "html",
/* Shared settings for all the projects below. See https://playwright.dev/docs/api/class-testoptions. */
use: {
/* Base URL to use in actions like `await page.goto('/')`. */
baseURL: "http://localhost:8888",
/* Collect trace when retrying the failed test. See https://playwright.dev/docs/trace-viewer */
trace: "on-first-retry",
},
/* Configure projects for major browsers */
projects: [
{
name: "chromium",
testIgnore: /.*\.mobile.spec.ts/,
use: { ...devices["Desktop Chrome"] },
},
{
name: "firefox",
testIgnore: /.*\.mobile.spec.ts/,
use: { ...devices["Desktop Firefox"] },
},
{
name: "webkit",
testIgnore: /.*\.mobile.spec.ts/,
use: { ...devices["Desktop Safari"] },
},
/* Test against mobile viewports. */
{
name: "Mobile Chrome",
testMatch: /.*\.mobile.spec.ts/,
use: { ...devices["Pixel 5"] },
},
{
name: "Mobile Safari",
testMatch: /.*\.mobile.spec.ts/,
use: { ...devices["iPhone 12"] },
},
/* Test against branded browsers. */
// {
// name: 'Microsoft Edge',
// use: { ...devices['Desktop Edge'], channel: 'msedge' },
// },
// {
// name: 'Google Chrome',
// use: { ...devices['Desktop Chrome'], channel: 'chrome' },
// },
],
/* Run your local dev server before starting the tests */
webServer: {
command: "npm run start",
url: "http://localhost:8888",
reuseExistingServer: !process.env.CI,
},
});
This is the now failing, stand alone test:
import { test, expect } from "@playwright/test";
test.describe("Intro", () => {
test.beforeEach(async ({ page }) => {
await page.goto("/"); // <--- errors here
});
test("is not visible", async ({ page }) => {
await expect(page.getByTestId("deskIntro")).toBeHidden();
});
});
Describe the bug
The test above used to run and pass fine on 1.37.1. Now with 1.39.0, it fails:
page.goto: Page closed
=========================== logs ===========================
navigating to "http://localhost:8888/", waiting until "load"
============================================================
This happens both locally and on GH Actions.
Locally, if I curl or load localhost:8888 I'm getting a response and the website without any issues. It's only Playwright that seemingly can't get to it.
I've also tried to remove the beforeEach and put the goto directly in the test case, with the same result.
If I look in the "Network" tab of the Playwright UI, I can see the requests and HTTP 200 responses but the screenshots are white and empty. If I try to goto http://google.com, I can see the page loading.
If I use mobile Chrome, the same test passes without issues. Only mobile Webkit fails.
If I rerun the test enough times, it will pass. It roughly passes once every 6--10 runs using npx playwright test --project 'Mobile Safari' --reporter dot