👷 Run BrowserStack unit tests with Vitest (webdriverio provider)

[thomas-lebeau]

Motivation

Every previous Vitest attempt stalled on BrowserStack: Vitest's default Playwright provider can't pick exact browser versions or drive real Android devices, and Vitest's client needs Chrome 87+ / Safari 15.4+, while our unit matrix is Chrome 80, Edge 80, Firefox 78, Safari 14 and Android Chrome.

This PR moves unit-bs to Vitest using the webdriverio provider, which reaches that exact matrix on real BrowserStack devices. Stacked on #5135. The Karma BrowserStack config is still in the repo (unused) so reverting is a one-line change; it is deleted in a follow-up PR.

Changes

• yarn test:unit:bs (CI unit-bs matrix) runs vitest run --config test/unit-vitest/vite.bs.config.ts.
• test/unit-vitest/vite.bs.config.ts: webdriverio provider against hub.browserstack.com:
  • W3C Selenium (seleniumVersion: 4.0.0) on desktop browsers: BrowserStack otherwise serves Chrome 80 through Selenium 2.53, which rejects the window resize Vitest sends before each file. Viewport errors are non-fatal anyway.
  • WebDriver keepalive (+ idleTimeout: 300): Vitest talks to the page over its own WebSocket, so BrowserStack otherwise kills the session as idle.
  • Downlevel to ES2020: Vitest's browser client ships ||=/??=. A plugin transpiles dependency JS and routes Vitest's prebuilt /__vitest__/ assets through the Vite pipeline (@oxc-project/runtime for injected helpers).
  • Polyfills for the orchestrator and tester pages: Object.hasOwn, Array.prototype.at, … (from 👷 Migrate unit tests to Vitest #4874), a same-origin BroadcastChannel (Safari < 15.4) and AggregateError (Firefox < 79).
  • bs-local.com for Safari only (BrowserStack rewrites localhost there); other browsers keep localhost, a secure context needed by secure cookie tests.
  • Page error reporter to the terminal (Safari on BrowserStack exposes no console logs).
  • verbose reporter (the BrowserStack wrapper kills silent runs), per-browser JUnit file, no coverage, same exclusions as karma.bs.conf.js.
• trackRuntimeError.spec: stop() resolved to window.stop(), which made Firefox 78 reload the test iframe; it now unsubscribes from the observable.
• webdriverio 9 → 10: v9's bundled undici 6 fails every request on Node 26 (UND_ERR_INVALID_ARG).

Results

All five unit-bs jobs pass the full suite. Job durations (CI Visibility, minutes, single run):

Job	Karma avg / p50 / p90 (other PRs, last 14 days)	Vitest (this PR)
unit-bs Chrome 80	3.19 / 2.98 / 4.28	5.42
unit-bs Android Chrome	3.04 / 2.81 / 3.95	6.62
unit-bs Edge 80	3.10 / 2.87 / 4.03	4.45 (on retry, see below)
unit-bs Firefox 78	3.13 / 3.04 / 3.79	3.97
unit-bs Safari 14	5.25 / 5.33 / 6.51	3.67

Open issues

• Reliability: Vitest doesn't reconnect a lost browser (Karma had browserDisconnectTolerance). We saw one BrowserStack tunnel drop and one Edge connect timeout. Planned: bs-wrapper.ts exits with a dedicated code on these infrastructure errors + GitLab retry: exit_codes on unit-bs.
• Speed: Chrome 80 and Android are 2–3.5 min slower than Karma (single runs; to be confirmed and investigated, likely modules served one by one through the tunnel).

⚠️ Written with an AI agent (Claude Code).

Test instructions

Needs BS_USERNAME / BS_ACCESS_KEY (e.g. in .env):

BS_BROWSER=safari-desktop yarn test:unit:bs

BS_BROWSER is one of edge, firefox, safari-desktop, chrome-desktop, chrome-mobile.

Checklist

• Tested locally
• Tested on staging
• Added unit tests for this change.
• Added e2e/integration tests for this change.
• Updated documentation and/or relevant AGENTS.md file

[cit-pr-commenter-54b7da]

Bundles Sizes Evolution

📦 Bundle Name	Base Size	Local Size	𝚫	𝚫%	Status
Rum Salesforce	N/A	188.06 KiB	N/A	N/A	N/A
Rum	188.04 KiB	188.04 KiB	0 B	0.00%	✅
Rum-salesforce Profiler	N/A	8.35 KiB	N/A	N/A	N/A
Rum Profiler	8.35 KiB	8.35 KiB	0 B	0.00%	✅
Rum-salesforce Recorder	N/A	32.04 KiB	N/A	N/A	N/A
Rum Recorder	32.04 KiB	32.04 KiB	0 B	0.00%	✅
Logs	59.44 KiB	59.44 KiB	0 B	0.00%	✅
Rum Slim	145.83 KiB	145.83 KiB	0 B	0.00%	✅
Worker	22.96 KiB	22.96 KiB	0 B	0.00%	✅
Rum Shopify	N/A	218.48 KiB	N/A	N/A	N/A
Rum-shopify Profiler	N/A	8.35 KiB	N/A	N/A	N/A
Rum-shopify Recorder	N/A	4.85 KiB	N/A	N/A	N/A

[datadog-prod-us1-3]

✅ All CI checks and tests passed.

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

🎯 Code Coverage (details)
• Patch Coverage: 100.00%
• Overall Coverage: 78.16% (-0.04%)

_{This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 9db56ed | Docs | View more details | Give us feedback!}