Skip to content

test: Add E2E test for first install #31435

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 6 commits into
base: main
Choose a base branch
from
Draft

Conversation

Gudahtt
Copy link
Member

@Gudahtt Gudahtt commented Mar 31, 2025

Description

Add E2E test to ensure a window opens upon first install.

This test needs to use a production-like build rather than a standard E2E build, so the existing "vault decryptor" job was repurposed to be more generically for E2E tests using a production-like build.

A global function reloadExtension is added to stateHooks for use by this test. This function had to be enabled for production builds because the test uses a production build.

Open in GitHub Codespaces

Related issues

This is an E2E test for #31332

Manual testing steps

  • Create an MV3 production-like build (yarn dist)
  • Run yarn test:e2e:single ./test/e2e/first-install.spec.ts --browser chrome to run the test on Chrome
  • Create an MV2 production-like build (yarn dist:mv2)
  • Run yarn test:e2e:single ./test/e2e/first-install.spec.ts --browser firefox to run the test on Firefox

Screenshots/Recordings

N/A

Pre-merge author checklist

Pre-merge reviewer checklist

  • I've manually tested the PR (e.g. pull and build branch, run the app, test code being changed).
  • I confirm that this PR addresses all acceptance criteria described in the ticket it closes and includes the necessary testing evidence such as recordings and or screenshots.

@metamaskbot
Copy link
Collaborator

✨ Files requiring CODEOWNER review ✨

🔒 @MetaMask/extension-security-team

  • .circleci/config.yml

@Gudahtt Gudahtt force-pushed the first-time-install-e2e-test branch 2 times, most recently from 897848a to 5efe7d9 Compare March 31, 2025 14:20
@Gudahtt Gudahtt marked this pull request as ready for review March 31, 2025 14:21
@Gudahtt Gudahtt requested a review from a team as a code owner March 31, 2025 14:21
@Gudahtt Gudahtt marked this pull request as draft March 31, 2025 14:21
@metamaskbot
Copy link
Collaborator

Builds ready [5efe7d9]
UI Startup Metrics (1193 ± 56 ms)
PlatformBuildTypePageMetricMean (ms)Min (ms)Max (ms)Std Dev (ms)P 75 (ms)P 95 (ms)
ChromeBrowserifyHomeuiStartup1193108413475612311293
load1038934117854938993
domContentLoaded1032928117453941990
domInteractive17133951629
firstPaint790791188384260993
backgroundConnect96535910
firstReactRender19155251931
getState12434868
initialActions001001
loadScripts81672395551847897
setupStore7514279
WebpackHomeuiStartup968780126083964991
load828620101260853914
domContentLoaded822599100460844903
domInteractive16124471436
firstPaint568541013336838888
backgroundConnect16115081539
firstReactRender14122831426
getState7412278
initialActions001000
loadScripts82058899459844896
setupStore7514278
FirefoxBrowserifyHomeuiStartup13921203214516614151835
load12521083199915812671650
domContentLoaded12511083199915812671649
domInteractive10238294358796
firstPaintNaNNaNNaNNaNNaNNaN
backgroundConnect2416123142430
firstReactRender23194452429
getState7336478
initialActions001001
loadScripts12281061197515312481621
setupStore6436368
WebpackHomeuiStartup10398721700170931966
load9067561455150835957
domContentLoaded9067561455150834956
domInteractive118342112814188
firstPaintNaNNaNNaNNaNNaNNaN
backgroundConnect2414110152335
firstReactRender20163732025
getState104661089
initialActions001001
loadScripts8877341432146823966
setupStore9565989
Bundle size diffs
  • background: 0 Bytes (0%)
  • ui: 67 Bytes (0%)
  • common: 0 Bytes (0%)

@Gudahtt Gudahtt marked this pull request as ready for review March 31, 2025 16:37
Comment on lines +289 to +305
/**
* Reload the extension.
*
* This is used for the `first-install` E2E test, which uses a production-like build. This
* function must be present even if `process.env.IN_TEST` is false.
*/
window.stateHooks.reloadExtension = () => {
browser.runtime.reload();
};
Copy link
Contributor

@davidmurdoch davidmurdoch Mar 31, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a quick drive by comment: Since we use browser.runtime.reload() in a few places in our code, it might make sense to update those to use this new stateHooks.reloadExtension line... that way we can say we aren't shipping test functions in production 😆

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, interesting idea, but it would probably be better to minimize the reliance on functions we make accessible outside the root compartment. Just to minimize the complexity of any audits into what could go wrong if they're tampered with somehow.

davidmurdoch
davidmurdoch previously approved these changes Mar 31, 2025
@Gudahtt Gudahtt marked this pull request as draft April 14, 2025 17:18
Add E2E test to ensure a window opens upon first install.

This test needs to use a production-like build rather than a standard E2E
build, so the existing "vault decryptor" job was repurposed to be more
generically for E2E tests using a production-like build.

A global function `reloadExtension` is added to `stateHooks` for use by this
test. This function had to be enabled for production builds because the test
uses a production build.
@metamaskbot
Copy link
Collaborator

Builds ready [837e3b0]
UI Startup Metrics (1195 ± 66 ms)
PlatformBuildTypePageMetricMean (ms)Min (ms)Max (ms)Std Dev (ms)P 75 (ms)P 95 (ms)
ChromeBrowserifyHomeuiStartup1195109415056612311318
load1042949121957949988
domContentLoaded1036941121657944986
domInteractive17135961628
firstPaint7061061165412239985
backgroundConnect6318278
firstReactRender21164762132
getState14548979
initialActions001001
loadScripts80070696656826913
setupStore7522378
WebpackHomeuiStartup21281711253517022342356
load16361324203714117391842
domContentLoaded16301320202814017351833
domInteractive161088121347
firstPaint179763535123176
backgroundConnect279321323262
firstReactRender215543881216291
getState1431571679
initialActions317136
loadScripts16251319202614017331830
setupStore25729741329
FirefoxBrowserifyHomeuiStartup13591155168611114421571
load12091025155710712671419
domContentLoaded12081024155710612671418
domInteractive10539265358897
firstPaintNaNNaNNaNNaNNaNNaN
backgroundConnect2213170172336
firstReactRender22193632230
getState1041782379
initialActions001001
loadScripts11881009154110712521404
setupStore841761867
WebpackHomeuiStartup15181336191612515691819
load12981152167011013451569
domContentLoaded12971152167011013451568
domInteractive7935158237893
firstPaintNaNNaNNaNNaNNaNNaN
backgroundConnect231491112444
firstReactRender35295153744
getState10436779
initialActions101011
loadScripts12781136158110613261547
setupStore9553789

@Gudahtt
Copy link
Member Author

Gudahtt commented Apr 14, 2025

Some changes needed here due to #31363, which migrated the vault decryptor test to GHA

@Gudahtt Gudahtt force-pushed the first-time-install-e2e-test branch from 81c10ba to cbc2979 Compare April 14, 2025 19:47
@metamaskbot
Copy link
Collaborator

Builds ready [cbc2979]
UI Startup Metrics (1204 ± 63 ms)
PlatformBuildTypePageMetricMean (ms)Min (ms)Max (ms)Std Dev (ms)P 75 (ms)P 95 (ms)
ChromeBrowserifyHomeuiStartup1204111314326312501322
load104595012756010771169
domContentLoaded103894612686010711163
domInteractive17126461629
firstPaint71983128041810601143
backgroundConnect7440479
firstReactRender21165872143
getState1344181831
initialActions001000
loadScripts803708101959833924
setupStore74142812
WebpackHomeuiStartup21811749272619723122553
load16811349215516717702043
domContentLoaded16711345211316017621969
domInteractive161168111348
firstPaint1836237164220318
backgroundConnect329267423281
firstReactRender20155385119317359
getState164265301535
initialActions317136
loadScripts16631344211016017591954
setupStore247295432137
FirefoxBrowserifyHomeuiStartup13201136178412713551627
load1179992163812812141470
domContentLoaded1178992163712812131470
domInteractive1064325034116177
firstPaintNaNNaNNaNNaNNaNNaN
backgroundConnect201371102045
firstReactRender22193432229
getState74364813
initialActions001001
loadScripts1160973162212911981448
setupStore64354616
WebpackHomeuiStartup14881329181010215611688
load1274114615639213261460
domContentLoaded1274114615609213251459
domInteractive77361932284123
firstPaintNaNNaNNaNNaNNaNNaN
backgroundConnect20154962137
firstReactRender34295153646
getState83305827
initialActions002111
loadScripts1255112515429213031441
setupStore85315823

Gudahtt added 2 commits April 15, 2025 10:20
…e-test

* origin/main: (57 commits)
  fix: cp-12.17.0 Fix remove nft (#32102)
  chore: rename migration 154 to 152.1 (#32109)
  feat: bump multichain-api-client 0.2.0 (#32104)
  chore: upgrade assets controllers to v58.0.0 (#32019)
  feat: upload test runs (#32103)
  chore: improving disabled functionality to tabs component (#32081)
  ci: add a soft gate for our page load benchmarks (#31912)
  test: Added bridge user action benchmark (#31952)
  fix: chromedriver crash on Windows (#31962)
  fix: add retries to get-job-id (#32015)
  feat: Adds Remote Mode hardware wallet gating (#32012)
  chore: clean up tech debt for `MegaETH Testnet` integration (#31867)
  chore: bump `@metamask/{polling,gas-fee}-controller` to `v13`,`v23` (#32035)
  chore(snaps): Bump Snaps packages (#32042)
  feat: auto create solana account after ondboarding or srp import cp-12.17.0 (#32038)
  feat: add option to switch from standard account <-> smart account on accounts details modal (#31899)
  fix: cp-12.17.0 - Center buttons in wider popup (#31897)
  feat: add non EVM testnets (#31702)
  feat: expose Multichain API via window.postMessage for Firefox (#30142)
  feat: Add "Create Solana Account" in connection flow when Solana is requested chain by dapp (#31781)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants