Skip to content

chore: replace Karma with Playwright for browser tests#6031

Open
mostafaNazari702 wants to merge 6 commits into
mochajs:mainfrom
mostafaNazari702:migrate-karma-to-playwright
Open

chore: replace Karma with Playwright for browser tests#6031
mostafaNazari702 wants to merge 6 commits into
mochajs:mainfrom
mostafaNazari702:migrate-karma-to-playwright

Conversation

@mostafaNazari702

Copy link
Copy Markdown
Contributor

PR Checklist

Overview

This is from a very old issue but i replaced the deprecated Karma setup with Playwright which ships its own bundled Chromium (no system Chrome required). The four browser suites (bdd/tdd/qunit/esm) now run in a real Chromium page via a small static server that serves the built mocha.js and a generated per-suite runner with results asserted via @playwright/test. npm run test-browser keeps the same clean=>build=>browser=>webpack-compat flow and CI installs the browser with npx playwright install --with-deps chromium.

@mostafaNazari702

Copy link
Copy Markdown
Contributor Author

@mark-wiemer You did show some passion in opening a PR for that issue but i had the same passion. Sorry :)

@codecov

codecov Bot commented Jun 2, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 0% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 78.88%. Comparing base (a8ffaa9) to head (ceb86af).
⚠️ Report is 13 commits behind head on main.

Files with missing lines Patch % Lines
playwright.config.mjs 0.00% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #6031      +/-   ##
==========================================
+ Coverage   78.08%   78.88%   +0.79%     
==========================================
  Files          63       63              
  Lines        4581     4546      -35     
  Branches      978      969       -9     
==========================================
+ Hits         3577     3586       +9     
+ Misses       1004      960      -44     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@mark-wiemer

mark-wiemer commented Jun 7, 2026

Copy link
Copy Markdown
Member

Test failures are known issues, ref #5361

This one is on the review backlog :) it's a big backlog though!

And absolutely no problem that you opened a PR for this, I'm often much happier reviewing PRs than I am trying to write code myself :)

@JoshuaKGoldberg JoshuaKGoldberg added this to the v12.0.0 milestone Jun 7, 2026

@JoshuaKGoldberg JoshuaKGoldberg left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall LGTM! This is really exciting, karma's been a pain for a while. Just requesting changes on trimming down the large number of IMO unnecessary comments. Could you please do a pass and trim them down to what you think is useful long-term?

Comment thread playwright.config.mjs Outdated
Comment thread .gitignore Outdated
Comment thread test/browser/server.mjs Outdated
Comment thread test/browser/server.mjs Outdated
Comment thread test/browser/run.spec.mjs Outdated
Comment thread test/browser/run.spec.mjs Outdated
Comment thread test/browser/global-setup.mjs Outdated
Comment thread test/browser/config.mjs
@JoshuaKGoldberg JoshuaKGoldberg added the status: waiting for author waiting on response from OP or other posters - more information needed label Jun 7, 2026
@mark-wiemer

Copy link
Copy Markdown
Member

As Mostafa is now without laptop for a while, I'll validate and make the proposed changes and send back to you for re-review :)

@mark-wiemer

mark-wiemer commented Jun 8, 2026

Copy link
Copy Markdown
Member

hm, checking lint failure, can't repro on local quite yet...

/home/runner/work/mocha/mocha/docs/_data/supporters.cjs
Error:    20:23  error  /home/runner/work/mocha/mocha/docs/node_modules/debug/package.json (directory description file): SyntaxError: /home/runner/work/mocha/mocha/docs/node_modules/debug/package.json (directory description file): SyntaxError: /home/runner/work/mocha/mocha/docs/node_modules/debug/package.json (directory description file): SyntaxError: Unexpected end of JSON input                                                                                                                n/no-missing-require
Error:   305:11  error  /home/runner/work/mocha/mocha/docs/node_modules/debug/package.json (directory description file): SyntaxError: /home/runner/work/mocha/mocha/docs/node_modules/debug/package.json (directory description file): SyntaxError: /home/runner/work/mocha/mocha/docs/node_modules/debug/package.json (directory description file): SyntaxError: /home/runner/work/mocha/mocha/docs/node_modules/debug/package.json (directory description file): SyntaxError: Unexpected end of JSON input  n/no-missing-require

I think this was a transient thing, I can't repro with the same Node version (22.22.3) on Windows :/ going to retrigger once the rest of the workflow finished

@mark-wiemer

Copy link
Copy Markdown
Member

Transient lint failure resolved, not going to track it... yet... but I'll link #5361 for funsies

@mark-wiemer mark-wiemer removed the status: waiting for author waiting on response from OP or other posters - more information needed label Jun 8, 2026
@JoshuaKGoldberg

JoshuaKGoldberg commented Jun 13, 2026

Copy link
Copy Markdown
Member

transient lint failure

Yeah, it is - it's a failure to fully download the package.json file in node_modules. That's on the GHA side.

@JoshuaKGoldberg JoshuaKGoldberg left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Ready to ship whenever you are @mark-wiemer 🥳

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

🛠️ Repo: Stop using deprecated karma library

3 participants