Ensure test build is the same as prod build

[danfinlay]

We recently merged a PR whose tests passed, but whose prod build failed. We could avoid this with consistent builds between those environments!

[kumavis]

this requires some build sys refactor

[Gudahtt]

While we do have some problems with indeterminism in our CI tests, there don't exist any differences anymore between the tests run on prod v.s. on PRs or develop.

We have a few different builds at the moment which differ in a variety of necessary ways, but the non-dev builds are pretty close to being identical to prod at the moment. We should still document the remaining differences though, and explore whether it's possible to minimize changes even further.

[Gudahtt]

We currently have four different builds: prod, dev, test, and devTest. Primarily we use prod for production and RC testing, dev for development, and test for e2e testing. devTest is primarily used for debugging e2e test failures.

I'll summarize the main differences between dev and prod, and between test and prod. We can consider devTest to be a mash-up of dev and test.

• dev
  • Includes the chromereload script, to make development on Chrome more convenient
  • Includes SCSS sourcemaps (they are omitted in the prod build)
  • Includes the webRequestBlocking permission
  • Omits the -libs bundles, instead opting to bundle all of the JavaScript together (both in the UI and the background)
  • Includes inlined sourcemaps (they are generated as external .map files in the prod build)
  • Skips JavaScript minification
  • Includes React devtools library
  • Automatically unlocks extension if a password is configured in .metamaskrc
• test
  • Includes the webRequestBlocking permission
  • Includes host permissions for localhost (required for e2e testing)

Additionally, there are environment variables that are injected into the JavaScript bundle that differ between environments:

• METAMASK_DEBUG
  • Indicates that the extension is being run in development. This has the following effects:
    • Changes the default log level (debug in development, warn otherwise)
    • Enables setup of global helper functions like metamaskGetState
    • Suppresses the automatic opening of the extension onboarding page upon install
    • Disables suppression of contentscript log messages with a level of info
    • Changes the default network from Mainnet to Rinkeby
    • Disables metrics, instead using a mock Segment API (unless SEGMENT_HOST is also set)
    • Disables Sentry
    • Enables Redux Dev Tools (it's always included in the bundle, even in prod, but it's only enabled in development)
• IN_TEST:
  • Indicates that the extension is being run in an e2e test. This has the following effects:
    • Suppresses the automatic opening of the extension onboarding page upon install
    • Replaces 3box with a mock 3box library
    • Introduces middleware in the JSON RPC client to slow down gas estimates, to get more production-like behavior (this was introduced because ganache was too fast to reproduce a bug that depended on slower gas estimates)
    • Changes the default network from Mainnet to localhost:8545 (i.e. ganache)
    • Disables metrics, instead using a mock Segment API (unless SEGMENT_HOST is also set)
    • Changes the error handling behaviour of missing en locale messages from a warning to an exception, so that these mistakes are easier to spot.
    • Changes send flow and confirmation rendering logic to make the ganache network be treated more like Mainnet, to allow e2e testing of Mainnet-specific UI
• METAMASK_ENVIRONMENT
  • Affects how Sentry events are categorized; purely descriptive
• NODE_ENV
  • Set to development in development, prod otherwise. Has no effect in our code, but maybe affects dependencies? 🤷
• PUBNUB_SUB_KEY and PUBNUB_PUB_KEY
  • Required for mobile sync. Injected via environment variable, so they are typically present in CI builds but missing otherwise
• SENTRY_DSN
  • Required for Sentry to work. Injected via environment variable, so typically present in CI builds but missing otherwise.
• INFURA_PROJECT_ID
  • Required for Infura to work correctly, configured via environment variable or .metamaskrc
  • Sets to zeros for test builds (i.e. an invalid value), to ensure we're mocking out Infura requests correctly.
• SEGMENT_HOST
  • Required for metrics to be enabled in non-production builds, configured via environment variable or .metamaskrc
  • Typically set to localhost:9090 for testing with the development/mock-sevment.js server, or to api.segment.io to test with the real Segment
• SEGMENT_WRITE_KEY and SEGMENT_LEGACY_WRITE_KEY
  • Required for metrics to work. Injected via environment variable or .metamaskrc; typically present in CI builds but missing otherwise

Many of these differences are here for good reason, and should stay. But we could do a lot to reduce this list.