Skip to content

test(firefox): add temporary webdriver benchmark workflow#41454

Closed
davidmurdoch wants to merge 3 commits intomainfrom
codex/firefox-benchmark-harness
Closed

test(firefox): add temporary webdriver benchmark workflow#41454
davidmurdoch wants to merge 3 commits intomainfrom
codex/firefox-benchmark-harness

Conversation

@davidmurdoch
Copy link
Copy Markdown
Contributor

Description

This is a temporary draft PR to run a focused Firefox buildWebDriver benchmark in CI against both main and codex/firefox-xpi-comment-metadata-spike from the same harness.

Changelog

CHANGELOG entry: null

Related issues

Fixes:

Manual testing steps

  1. Open the Actions run for Firefox buildWebDriver benchmark (temp).
  2. Confirm both matrix targets (main and candidate) complete.
  3. Review the generated step summary and uploaded JSON artifacts.

Pre-merge author checklist

Pre-merge reviewer checklist

  • I've manually tested the PR (e.g. pull and build branch, run the app, test code being changed).
  • I confirm that this PR addresses all acceptance criteria described in the ticket it closes and includes the necessary testing evidence such as recordings and or screenshots.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 2, 2026

CLA Signature Action: All authors have signed the CLA. You may need to manually re-run the blocking PR check if it doesn't pass in a few minutes.

@metamaskbot metamaskbot added the team-extension-platform Extension Platform team label Apr 2, 2026
@github-actions github-actions bot added the size-M label Apr 2, 2026
@metamaskbotv2
Copy link
Copy Markdown
Contributor

metamaskbotv2 bot commented Apr 2, 2026

Builds ready [8aaaa73]
⚡ Performance Benchmarks (Total: 🟢 17 pass · 🟡 1 warn · 🔴 0 fail)

Baseline (latest main): 5cc482e | Date: 6/24/58222 | Pipeline: 23917109056 | Baseline logs

Interaction Benchmarks
Benchmarkchrome-browserify
loadNewAccount🟢 [Show logs]
confirmTx🟢 [Show logs]
bridgeUserActions🟢 [Show logs]

📈 Results compared to the previous 5 runs on main

  • bridgeUserActions/bridge_load_asset_picker: +41%
Startup Benchmarks
Benchmarkchrome-browserifychrome-webpackfirefox-browserifyfirefox-webpack
startupStandardHome🟢 [Show logs]🟢 [Show logs]🟢 [Show logs]🟢 [Show logs]
startupPowerUserHome
🟡 uiStartup
[Show logs]
🟢 [Show logs]🟢 [Show logs]🟢 [Show logs]

📈 Results compared to the previous 5 runs on main

  • startupStandardHome/initialActions: +11%
  • startupPowerUserHome/backgroundConnect: -21%
  • startupPowerUserHome/numNetworkReqs: +20%
  • startupStandardHome/firstPaint: -16%
  • startupPowerUserHome/uiStartup: +13%
  • startupPowerUserHome/domInteractive: -11%
  • startupPowerUserHome/numNetworkReqs: +57%
  • startupStandardHome/initialActions: -17%
  • startupPowerUserHome/domInteractive: +10%
  • startupPowerUserHome/backgroundConnect: -18%
  • startupPowerUserHome/setupStore: +24%
  • startupStandardHome/initialActions: +11%
  • startupStandardHome/setupStore: +50%
  • startupPowerUserHome/firstReactRender: +12%
  • startupPowerUserHome/setupStore: +21%
User Journey Benchmarks
Benchmarkchrome-browserify
onboardingImportWallet🟢 [Show logs]
onboardingNewWallet🟢 [Show logs]
assetDetails🟢 [Show logs]
solanaAssetDetails🟢 [Show logs]
importSrpHome🟢 [Show logs]
sendTransactions🟢 [Show logs]
swap🟢 [Show logs]

📈 Results compared to the previous 5 runs on main

  • onboardingImportWallet/metricsToWalletReadyScreen: -17%
  • onboardingImportWallet/doneButtonToHomeScreen: -75%
  • onboardingImportWallet/openAccountMenuToAccountListLoaded: +55%
  • onboardingImportWallet/total: -33%
  • onboardingNewWallet/agreeButtonToOnboardingSuccess: -14%
  • onboardingNewWallet/doneButtonToAssetList: -33%
  • onboardingNewWallet/total: -27%
  • assetDetails/assetClickToPriceChart: -69%
  • assetDetails/total: -69%
  • solanaAssetDetails/assetClickToPriceChart: -59%
  • solanaAssetDetails/total: -59%
  • importSrpHome/loginToHomeScreen: +13%
  • importSrpHome/openAccountMenuAfterLogin: -14%
  • importSrpHome/homeAfterImportWithNewWallet: -31%
  • importSrpHome/total: -25%
  • sendTransactions/openSendPageFromHome: +65%
  • sendTransactions/reviewTransactionToConfirmationPage: +13%
  • sendTransactions/total: +13%
  • swap/openSwapPageFromHome: -82%
  • swap/fetchAndDisplaySwapQuotes: +24%
  • swap/total: +10%
🌐 Dapp Page Load Benchmarks

Current Commit: 8aaaa73 | Date: 4/2/2026

📄 Localhost MetaMask Test Dapp

Samples: 100

Summary

  • pageLoadTime-> current mean value: 1.03s (±39ms) 🟡 | historical mean value: 1.02s ⬆️ (historical data)
  • domContentLoaded-> current mean value: 711ms (±37ms) 🟢 | historical mean value: 717ms ⬇️ (historical data)
  • firstContentfulPaint-> current mean value: 85ms (±11ms) 🟢 | historical mean value: 85ms ⬇️ (historical data)

📈 Detailed Results

Metric Mean Std Dev Min Max P95 P99
pageLoadTime 1.03s 39ms 1.01s 1.31s 1.09s 1.31s
domContentLoaded 711ms 37ms 690ms 969ms 758ms 969ms
firstPaint 85ms 11ms 68ms 168ms 96ms 168ms
firstContentfulPaint 85ms 11ms 68ms 168ms 96ms 168ms
largestContentfulPaint 0ms 0ms 0ms 0ms 0ms 0ms
Bundle size diffs [🚨 Warning! Bundle size has increased!]
  • background: 503 Bytes (0.01%)
  • ui: -11.54 KiB (-0.14%)
  • common: 15.67 KiB (0.12%)

@sonarqubecloud
Copy link
Copy Markdown

sonarqubecloud bot commented Apr 2, 2026

@metamaskbotv2
Copy link
Copy Markdown
Contributor

metamaskbotv2 bot commented Apr 2, 2026

Builds ready [41453f0]
⚡ Performance Benchmarks (Total: 🟢 18 pass · 🟡 0 warn · 🔴 0 fail)

Baseline (latest main): 5cc482e | Date: 6/24/58222 | Pipeline: 23918534050 | Baseline logs

Interaction Benchmarks
Benchmarkchrome-browserify
loadNewAccount🟢 [Show logs]
confirmTx🟢 [Show logs]
bridgeUserActions🟢 [Show logs]
Startup Benchmarks
Benchmarkchrome-browserifychrome-webpackfirefox-browserifyfirefox-webpack
startupStandardHome🟢 [Show logs]🟢 [Show logs]🟢 [Show logs]🟢 [Show logs]
startupPowerUserHome🟢 [Show logs]🟢 [Show logs]🟢 [Show logs]🟢 [Show logs]

📈 Results compared to the previous 5 runs on main

  • startupStandardHome/firstPaint: +12%
  • startupStandardHome/initialActions: -44%
  • startupPowerUserHome/domInteractive: +14%
  • startupPowerUserHome/firstPaint: +12%
  • startupPowerUserHome/numNetworkReqs: +58%
  • startupStandardHome/initialActions: +67%
  • startupPowerUserHome/setupStore: +39%
  • startupStandardHome/initialActions: -44%
  • startupStandardHome/setupStore: -42%
  • startupPowerUserHome/domInteractive: -19%
User Journey Benchmarks
Benchmarkchrome-browserify
onboardingImportWallet🟢 [Show logs]
onboardingNewWallet🟢 [Show logs]
assetDetails🟢 [Show logs]
solanaAssetDetails🟢 [Show logs]
importSrpHome🟢 [Show logs]
sendTransactions🟢 [Show logs]
swap🟢 [Show logs]

📈 Results compared to the previous 5 runs on main

  • onboardingImportWallet/metricsToWalletReadyScreen: -32%
  • onboardingImportWallet/doneButtonToHomeScreen: -74%
  • onboardingImportWallet/openAccountMenuToAccountListLoaded: +55%
  • onboardingImportWallet/total: -32%
  • onboardingNewWallet/doneButtonToAssetList: -30%
  • onboardingNewWallet/total: -24%
  • assetDetails/assetClickToPriceChart: -58%
  • assetDetails/total: -58%
  • solanaAssetDetails/assetClickToPriceChart: -54%
  • solanaAssetDetails/total: -54%
  • importSrpHome/homeAfterImportWithNewWallet: -34%
  • importSrpHome/total: -29%
  • sendTransactions/openSendPageFromHome: +29%
  • sendTransactions/selectTokenToSendFormLoaded: +18%
  • sendTransactions/reviewTransactionToConfirmationPage: +15%
  • sendTransactions/total: +15%
  • swap/openSwapPageFromHome: -84%
  • swap/fetchAndDisplaySwapQuotes: +24%
  • swap/total: +10%
🌐 Dapp Page Load Benchmarks

Current Commit: 41453f0 | Date: 4/2/2026

📄 Localhost MetaMask Test Dapp

Samples: 100

Summary

  • pageLoadTime-> current mean value: 1.07s (±41ms) 🟡 | historical mean value: 1.02s ⬆️ (historical data)
  • domContentLoaded-> current mean value: 738ms (±38ms) 🟢 | historical mean value: 717ms ⬆️ (historical data)
  • firstContentfulPaint-> current mean value: 88ms (±10ms) 🟢 | historical mean value: 85ms ⬆️ (historical data)

📈 Detailed Results

Metric Mean Std Dev Min Max P95 P99
pageLoadTime 1.07s 41ms 1.03s 1.34s 1.13s 1.34s
domContentLoaded 738ms 38ms 704ms 1.00s 799ms 1.00s
firstPaint 88ms 10ms 72ms 168ms 104ms 168ms
firstContentfulPaint 88ms 10ms 72ms 168ms 104ms 168ms
largestContentfulPaint 0ms 0ms 0ms 0ms 0ms 0ms
Bundle size diffs [🚨 Warning! Bundle size has increased!]
  • background: 503 Bytes (0.01%)
  • ui: -11.54 KiB (-0.14%)
  • common: 15.67 KiB (0.12%)

@github-actions github-actions bot locked and limited conversation to collaborators Apr 9, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

size-L team-extension-platform Extension Platform team

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants