test: Playwright e2e suite (first-run smoke + env-backed welcome) (#976)

* test: add Playwright e2e suite and wire it into CI

Run the existing Playwright `_electron` suite in a new e2e.yml (Node 24 — the
version Electron bundles — under xvfb, with the Electron binary cached and
materialized for the cached-binary case). Add a launch/cleanup harness:
`launchApp` isolates each test with Electron's native `--user-data-dir` (the app
honors it via app.getPath('userData')) and can seed `app-data.json`; `pageByTitle`
selects a window by title since the app opens several views, most title-less, so
`firstWindow()` is ambiguous. Smoke tests: launch, the Welcome view renders a
Welcome-specific element, the multi-view composition is present (guards the
Phase 3 BrowserView->WebContentsView migration), and the app shuts down without
hanging (close in finally; job timeout bounds it).

Note: AI-assisted (Claude Code). Manually verified: 4 smoke tests pass locally
(yarn build && yarn test:e2e); tsc, eslint, prettier clean.

* test: cover the env-backed welcome actions with a seeded Python env

When a Python env is configured the welcome local-server actions become
enabled; pin that. The test seeds `app-data.json` with a `pythonPath` (the real
config seam the app reads) and asserts the "New notebook" action loses its
`disabled` class. Gated on JLAB_TEST_PYTHON_PATH (CI provisions a venv with
jupyterlab and sets it; skipped locally when absent) so the suite stays green
everywhere. Driving the JupyterLab content itself is left to a separate
plain-Playwright suite against the server URL (Phase 3).

Note: AI-assisted (Claude Code). Manually verified: passes locally with
JLAB_TEST_PYTHON_PATH set to a jupyterlab env; skips cleanly when unset.

* ci: pin jupyterlab and scope e2e concurrency per PR

Address review: pin the CI venv to jupyterlab==4.5.7 (the app's target, matching
env_installer/jlab_server.yaml) so env-backed tests don't drift with upstream
releases; key the concurrency group on the PR number instead of head_ref so
same-named branches across forks don't cancel each other's runs.

Note: AI-assisted (Claude Code). Manually verified: prettier clean.

* test: e2e the labview booting a real Jupyter server

With a Python env seeded, clicking "New notebook" boots a real Jupyter server
and the labview WebContentsView navigates to it; assert it reached the local
server URL via waitForWindowByUrl (no driving the JupyterLab DOM through
Electron). The session reads its env from two seams, so the launch helper now
seeds both: app-data.json (pythonPath enables the welcome + userSetPythonEnvs
registers the env so the registry resolves it) and settings.json (pythonPath
the new session uses to boot). Gated on JLAB_TEST_PYTHON_PATH like the other
env-backed test.

Note: AI-assisted (Claude Code). Manually verified: both env-backed tests pass
locally with JLAB_TEST_PYTHON_PATH set (labview reaches http://localhost:PORT/lab
in ~6s); smoke tests still skip them when unset; tsc/eslint/prettier clean.

* ci: drop the electron cache and install.js, just yarn install

The ~/.cache/electron cache made launch nondeterministic: on a cache hit the
postinstall skips extraction (empty dist -> 'Electron failed to install
correctly'), and an explicit install.js step that fixes the hit case corrupts a
fresh install on a miss. Removing both and letting yarn install's postinstall
download + extract fresh each run is deterministic (~10s) and is the config that
passed before the cache was added.

Note: AI-assisted (Claude Code). Manually verified: matches the previously
passing setup; the cache/install.js permutations that failed are removed.

* ci: temp diagnostic for electron install state [skip-review]

Note: AI-assisted (Claude Code). Temporary, reverted once the cause is known.

* ci: diag electron resolution [temp]

Note: AI-assisted (Claude Code). Temporary diagnostic.

* ci: force a clean Electron binary install before e2e

The diagnostic showed node_modules/electron/dist had the resource files but not
the electron binary itself (dist/electron missing), so require('electron')
threw 'Electron failed to install correctly'. yarn install left dist
half-extracted and the postinstall skipped re-extraction because path.txt
existed. Remove dist/path.txt/the download cache and re-run install.js to
download + extract fresh, asserting the binary is present and executable.

Note: AI-assisted (Claude Code). Manually verified: diagnosed from CI logs
(dist/electron 'No such file or directory'); the test -x guard fails loud if the
binary is still missing.

* ci: debug electron install.js [temp]

Note: AI-assisted (Claude Code). Temporary @electron/get debug.

* ci: check electron zip integrity [temp]

Note: AI-assisted (Claude Code). Temporary.

* ci: extract Electron with system unzip (extract-zip drops the binary)

The downloaded electron zip is complete and valid (96MB, integrity OK), but
electron's postinstall via extract-zip leaves node_modules/electron/dist without
the electron binary on this runner, so launch fails. Let install.js fetch the
zip into the cache, then extract it with the system unzip and write path.txt.

Note: AI-assisted (Claude Code). Diagnosed from CI: zip ZIP_OK + 74 files but
dist/electron 'No such file'; extract-zip is the failing step, system unzip is
the workaround.

* ci: tidy the Electron install step comment

Note: AI-assisted (Claude Code). Comment-only cleanup of the merged iteration notes.

* test: simplify the env-backed e2e

Merge the two env-backed cases into one flow (single app launch): with a seeded
Python env, assert the welcome action is enabled and that clicking New notebook
boots a server the labview reaches. Drop the redundant pip upgrade in CI.

Note: AI-assisted (Claude Code). Manually verified: the merged env test passes
with JLAB_TEST_PYTHON_PATH set (~12s); the 4 smoke tests still pass and the env
test skips when unset; tsc/eslint/prettier clean.

* test: drop the hardcoded jupyterlab version from the e2e seed

The app re-detects the env's jupyterlab version on launch, so the seeded
userSetPythonEnvs entry does not need a versions field; pinning it to a literal
duplicated the version with the pip install in the workflow and would go stale on
a jupyterlab bump. The labview test still boots a real server without it.

Note: AI-assisted (Claude Code). Manually verified: the env-backed labview test
passes with the versions field removed (~10s); the 5-test suite stays green;
tsc/eslint/prettier clean.

* test(e2e): track the bundled jupyterlab version automatically

Read the pin from env_installer/jlab_server.yaml so a bump there propagates to
the e2e env without a second edit, with a guard that fails the job if the format
ever changes (so we notice instead of silently installing latest).

Note: AI-assisted (Claude Code). Manually verified: awk extracts '4.5.7' from
the current jlab_server.yaml; prettier clean on the workflow.

* test(e2e): tighten the jupyterlab-pin parse to a semver match

The non-empty check accepted operator-prefixed pins (`>=4.5.7`) that would
make pip fail with a confusing message; require an exact x.y.z so the job fails
with a clear hint pointing back at the env spec.

Note: AI-assisted (Claude Code). Manually verified: the regex accepts 4.5.7 and
rejects empty / operator-prefixed / multi-line inputs.

---------

Co-authored-by: Michał Krassowski <5832902+krassowski@users.noreply.github.com>

Author: notluquis

.github/workflows/e2e.yml

@@ -0,0 +1,78 @@
+name: E2E
+
+on:
+  push:
+    branches: [master]
+  pull_request:
+    branches: [master]
+
+permissions:
+  contents: read
+
+concurrency:
+  group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
+  cancel-in-progress: true
+
+jobs:
+  e2e:
+    name: 'Playwright (Linux)'
+    runs-on: ubuntu-latest
+    # Bound the job so a stuck Electron launch/close fails fast instead of
+    # tying up a runner; this suite specifically guards shutdown hangs.
+    timeout-minutes: 15
+    env:
+      # Only Electron is launched here, never Playwright's bundled browsers, so
+      # skip their download during yarn install (faster, fewer network flakes).
+      PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD: '1'
+    steps:
+      - uses: actions/checkout@93cb6efe18208431cddfb8368fd83d5badbf9bfd # v5
+      - uses: actions/setup-node@a0853c24544627f65ddf259abe73b1d18a591444 # v5
+        with:
+          # Match the Node that the target Electron bundles (Electron 42 -> Node
+          # 24); do not run ahead of it. Node 26 broke Electron's install tooling.
+          node-version: '24.x'
+          cache: 'yarn'
+      - run: yarn install --frozen-lockfile
+      # electron's postinstall (extract-zip) leaves dist without the `electron`
+      # binary on this runner even though the downloaded zip is complete and
+      # valid, so launch fails with "Electron failed to install correctly". Let
+      # install.js fetch the zip into the cache, then extract it with the system
+      # `unzip` and point path.txt at the binary.
+      - name: Install the Electron binary
+        run: |
+          set -euo pipefail
+          node node_modules/electron/install.js || true
+          zip=$(ls ~/.cache/electron/*/electron-*-linux-x64.zip | head -1)
+          rm -rf node_modules/electron/dist
+          mkdir -p node_modules/electron/dist
+          unzip -q -o "$zip" -d node_modules/electron/dist
+          printf 'electron' > node_modules/electron/path.txt
+          test -x node_modules/electron/dist/electron
+      - run: yarn build
+      # Electron needs system libraries; xvfb provides a display on the headless
+      # runner. The app launches up to four windows, so a real display beats
+      # --headless here.
+      - run: npx playwright install-deps
+      # A minimal Python env with jupyterlab so the env-backed tests run (a venv
+      # + pip is lighter than a full conda env for a smoke). The seeded
+      # app-data.json points the app at this interpreter.
+      - name: Provision a Python env for the env-backed e2e
+        run: |
+          set -euo pipefail
+          # Install the same jupyterlab the app bundles, read from the env spec so
+          # this tracks a version bump instead of drifting. Fail loudly if the
+          # pin format changes rather than silently installing latest.
+          ver=$(awk '$1=="-" && $2=="jupyterlab" {print $3}' env_installer/jlab_server.yaml)
+          [[ "$ver" =~ ^[0-9]+\.[0-9]+\.[0-9]+$ ]] || { echo "unexpected jupyterlab pin '$ver' in jlab_server.yaml (want exact x.y.z)"; exit 1; }
+          python3 -m venv "$RUNNER_TEMP/jlab-venv"
+          "$RUNNER_TEMP/jlab-venv/bin/pip" install --quiet "jupyterlab==$ver"
+          echo "JLAB_TEST_PYTHON_PATH=$RUNNER_TEMP/jlab-venv/bin/python" >> "$GITHUB_ENV"
+      - run: xvfb-run -a yarn test:e2e
+      - uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
+        if: ${{ !cancelled() }}
+        with:
+          name: playwright-report
+          path: |
+            playwright-report/
+            test-results/
+          retention-days: 14

.gitignore

@@ -8,6 +8,8 @@ node_modules
 .cache
 .eslintcache
 coverage
+test-results
+playwright-report
 .vscode/*
 !.vscode/launch.json
 *.py[co]

playwright.config.ts

@@ -4,7 +4,8 @@ export default defineConfig({
   testDir: './test/e2e',
   timeout: 60000,
   retries: process.env.CI ? 2 : 0,
-  reporter: process.env.CI ? 'github' : 'list',
+  workers: process.env.CI ? 1 : undefined,
+  reporter: process.env.CI ? [['github'], ['html', { open: 'never' }]] : 'list',
   use: {
     trace: 'on-first-retry',
     video: 'on-first-retry'

test/e2e/helpers.ts

@@ -0,0 +1,83 @@
+import { _electron as electron, ElectronApplication } from '@playwright/test';
+import { stubAllDialogs } from 'electron-playwright-helpers';
+import { mkdtempSync, rmSync, writeFileSync } from 'fs';
+import { join } from 'path';
+import { tmpdir } from 'os';
+
+// A fresh userData dir per launch keeps tests independent. Electron's native
+// --user-data-dir flag is honored because the app reads app.getPath('userData')
+// (see getUserDataDir in utils) and only overrides it on Snap. Returns the app
+// plus the temp userData dir so the caller can clean it up in a finally block.
+// If launch fails, the temp dir is removed here so repeated runs don't
+// accumulate dirs.
+//
+// Pass `pythonPath` to seed the app's config before launch so it treats that
+// interpreter as a configured environment. Two files are written, matching the
+// two seams the app reads: `app-data.json` (`pythonPath` enables the welcome
+// actions; `userSetPythonEnvs` registers the env so the session registry
+// resolves it) and `settings.json` (`pythonPath` is what a new session reads
+// via workspace settings to boot its server).
+export async function launchApp(opts?: {
+  pythonPath?: string;
+}): Promise<{ app: ElectronApplication; userDataDir: string }> {
+  const userDataDir = mkdtempSync(join(tmpdir(), 'jlab-e2e-'));
+  if (opts?.pythonPath) {
+    writeFileSync(
+      join(userDataDir, 'app-data.json'),
+      JSON.stringify({
+        pythonPath: opts.pythonPath,
+        userSetPythonEnvs: [
+          {
+            path: opts.pythonPath,
+            name: 'e2e-env',
+            type: 'path',
+            defaultKernel: 'python3'
+          }
+        ]
+      })
+    );
+    writeFileSync(
+      join(userDataDir, 'settings.json'),
+      JSON.stringify({ pythonPath: opts.pythonPath })
+    );
+  }
+  try {
+    const app = await electron.launch({
+      args: ['.', `--user-data-dir=${userDataDir}`]
+    });
+    await stubAllDialogs(app);
+    return { app, userDataDir };
+  } catch (error) {
+    cleanup(userDataDir);
+    throw error;
+  }
+}
+
+export function cleanup(userDataDir: string): void {
+  rmSync(userDataDir, { recursive: true, force: true });
+}
+
+// The app opens several windows (titlebar, welcome, session, manager) and most
+// have an empty title, so firstWindow() is ambiguous. Poll windows() and match
+// on the page title instead. Post Electron 30 a WebContentsView also surfaces
+// as its own page here, so title matching stays the reliable selector.
+export async function pageByTitle(
+  app: ElectronApplication,
+  pattern: RegExp,
+  timeout = 15000
+) {
+  const deadline = Date.now() + timeout;
+  while (Date.now() < deadline) {
+    for (const page of app.windows()) {
+      try {
+        if (pattern.test(await page.title())) {
+          return page;
+        }
+      } catch {
+        // page may be mid-navigation; retry on the next pass
+      }
+    }
+    await new Promise(resolve => setTimeout(resolve, 100));
+  }
+  throw new Error(`no window title matched ${pattern}`);
+}

test/e2e/python-env.test.ts

@@ -0,0 +1,37 @@
+import { expect, test } from '@playwright/test';
+import { waitForWindowByUrl } from 'electron-playwright-helpers';
+import { cleanup, launchApp, pageByTitle } from './helpers';
+
+// Needs a real Python env with jupyterlab. CI provisions one and points
+// JLAB_TEST_PYTHON_PATH at it; locally, set it to any python that has jupyterlab
+// installed. Skipped when absent so the suite stays green everywhere.
+const pythonPath = process.env.JLAB_TEST_PYTHON_PATH;
+
+test('with a seeded Python env, New notebook opens the labview', async () => {
+  test.skip(
+    !pythonPath,
+    'set JLAB_TEST_PYTHON_PATH to a python with jupyterlab'
+  );
+  test.setTimeout(120000);
+  const { app, userDataDir } = await launchApp({ pythonPath });
+  try {
+    const welcome = await pageByTitle(app, /welcome/i);
+    // The app validated the seeded env and emitted EnableLocalServerActions,
+    // so the local-server action loses its `disabled` class and is clickable.
+    const newNotebook = welcome.locator('#new-notebook-link');
+    await expect(newNotebook).not.toHaveClass(/disabled/, { timeout: 20000 });
+    await newNotebook.click();
+    // The session boots a real Jupyter server (via the seeded env) and the
+    // labview WebContentsView navigates to it. Assert it reached the local
+    // server URL rather than driving the JupyterLab DOM through Electron.
+    const lab = await waitForWindowByUrl(
+      app,
+      /https?:\/\/(127\.0\.0\.1|localhost):\d+/,
+      { timeout: 90000 }
+    );
+    expect(lab.url()).toMatch(/:\d+/);
+  } finally {
+    await app.close();
+    cleanup(userDataDir);
+  }
+});

test/e2e/smoke.test.ts

@@ -1,75 +1,60 @@
-import { _electron as electron, expect, test } from '@playwright/test';
-import { stubAllDialogs } from 'electron-playwright-helpers';
-import { mkdtempSync, rmSync } from 'fs';
-import { join } from 'path';
-import { tmpdir } from 'os';
-
-let tempUserData: string;
-
-test.beforeEach(() => {
-  tempUserData = mkdtempSync(join(tmpdir(), 'jlab-e2e-'));
+import { expect, test } from '@playwright/test';
+import { cleanup, launchApp, pageByTitle } from './helpers';
+
+test('app launches and opens at least one window', async () => {
+  const { app, userDataDir } = await launchApp();
+  try {
+    await app.firstWindow();
+    expect(app.windows().length).toBeGreaterThan(0);
+  } finally {
+    await app.close();
+    cleanup(userDataDir);
+  }
 });
 
-test.afterEach(async () => {
-  rmSync(tempUserData, { recursive: true, force: true });
+test('on first run the Welcome window renders', async () => {
+  const { app, userDataDir } = await launchApp();
+  try {
+    // Selecting by title rather than firstWindow(): the app opens several
+    // windows and only this one is the welcome view.
+    const welcome = await pageByTitle(app, /welcome/i);
+    await welcome.waitForLoadState('domcontentloaded');
+    // Assert a Welcome-specific element, not just body, so a blank or error
+    // page would fail rather than pass.
+    await expect(welcome.locator('#new-notebook-link')).toBeVisible();
+  } finally {
+    await app.close();
+    cleanup(userDataDir);
+  }
 });
 
-test('app launches and shows a window', async () => {
-  const app = await electron.launch({
-    args: ['.'],
-    env: {
-      ...process.env,
-      JLAB_DESKTOP_HOME: tempUserData,
-      ELECTRON_IS_TEST: '1'
-    }
-  });
-
-  await stubAllDialogs(app);
-  const window = await app.firstWindow();
-  await window.waitForLoadState('domcontentloaded');
-
-  expect(app.windows().length).toBeGreaterThan(0);
-  await app.close();
+test('the session window composes multiple views (titlebar + welcome + content)', async () => {
+  const { app, userDataDir } = await launchApp();
+  try {
+    // Once the welcome view has settled, the BrowserView/WebContentsView
+    // composition should expose several views (titlebar, welcome, content area)
+    // as separate pages. A dropped view after the Phase 3 WebContentsView
+    // migration would lower this count.
+    await pageByTitle(app, /welcome/i);
+    await expect
+      .poll(() => app.windows().length, { timeout: 10000 })
+      .toBeGreaterThanOrEqual(3);
+  } finally {
+    await app.close();
+    cleanup(userDataDir);
+  }
 });
 
-test('app exits with code 0', async () => {
-  const app = await electron.launch({
-    args: ['.'],
-    env: {
-      ...process.env,
-      JLAB_DESKTOP_HOME: tempUserData,
-      ELECTRON_IS_TEST: '1'
-    }
-  });
-
-  await stubAllDialogs(app);
-  await app.firstWindow();
-
-  const exitCode = await app.close();
-  expect(exitCode).toBe(0);
-});
-
-test('first-run shows env selection when no env configured', async () => {
-  const app = await electron.launch({
-    args: ['.'],
-    env: {
-      ...process.env,
-      JLAB_DESKTOP_HOME: tempUserData,
-      ELECTRON_IS_TEST: '1',
-      // empty userData = no prior config = first run
-      APPDATA: tempUserData
-    }
-  });
-
-  await stubAllDialogs(app);
-  const window = await app.firstWindow();
-
-  // welcome view or env select should appear within 15s
-  await expect(
-    window.locator(
-      '#welcome-view, #env-select-dialog, [data-testid="env-select"]'
-    )
-  ).toBeVisible({ timeout: 15000 });
-
-  await app.close();
+test('app shuts down cleanly without hanging', async () => {
+  const { app, userDataDir } = await launchApp();
+  try {
+    await app.firstWindow();
+  } finally {
+    // Close in finally so the Electron process is always terminated, even if
+    // an earlier step throws. A hang here surfaces via the job timeout.
+    await app.close();
+    cleanup(userDataDir);
+  }
+  // close() has resolved, so the process exited and the windows are gone.
+  expect(app.windows().length).toBe(0);
 });