ci: pre-warm Namespace cache for unit/component-view test shards (#30098)

tommasini · web-flow · commit b30bd2af32d7 · 2026-05-15T21:38:00.000Z
## Summary Introduces a \`prepare-ci-js-deps\` job and a \`setup-ci-js-deps\` composite action to eliminate redundant \`yarn install\` + \`yarn setup:github-ci --node\` across 13 CI shards (10 unit-test + 2 component-view + 1 merge job). ### What changed **New job \`prepare-ci-js-deps\` (\`ci.yml\`)** Runs before \`unit-tests\`, \`component-view-tests\`, and \`merge-unit-and-component-view-tests\` and installs dependencies once on behalf of all consumers. - **Namespace runners**: mounts the persistent cache volume and runs install only when the volume is cold (yarn.lock changed in the pool). Consumers restore from the same volume — install is skipped entirely on a warm cache. - **Non-Namespace runners** (current default for all PRs): runs install, then packs \`node_modules\` + generated files into a gzipped tarball (\`ci-js-deps.tar.gz\`) and uploads it as a workflow-run artifact. Consumers download and extract the tarball instead of running their own install. > **TEMP**: The tarball artifact path is a temporary fallback while Namespace is on trial. Once Namespace becomes the default runner, the pack/upload steps in the producer and the download/extract steps in each consumer can be deleted. The \`setup-ci-js-deps\` composite action requires no changes. **New composite action \`.github/actions/setup-ci-js-deps\`** Shared setup sequence used by the producer and all three consumer jobs: 1. Mount Namespace cache (Namespace only) 2. \`actions/setup-node\` 3. File-based freshness check (\`node_modules\` dir + \`termsOfUseContent.ts\`) 4. \`yarn install --immutable\` + \`yarn setup:github-ci --node\` — only if step 3 finds files missing On non-Namespace consumers, step 3 finds the tarball already extracted, so step 4 is skipped. **Consumer jobs (\`unit-tests\`, \`component-view-tests\`, \`merge-unit-and-component-view-tests\`)** Each job's setup block is replaced with: - TEMP: download + extract \`ci-js-deps.tar.gz\` (non-Namespace only; tar preserves permissions so no separate restore step is needed) - Call to \`setup-ci-js-deps\` (which skips install when files are already present) **\`check-all-jobs-pass\`** \`prepare-ci-js-deps\` added to the gate. ### Not changed - \`setup-node-modules.yml\` — untouched - E2E jobs — untouched (Phase 2 scope) ## Estimated impact | Lane | Shards | Before | After (non-Namespace) | After (Namespace) | |---|---|---|---|---| | \`unit-tests\` | 10 | ~2 min install each | ~30s download+extract | ~10s cache hit | | \`component-view-tests\` | 2 | ~2 min install each | ~30s download+extract | ~10s cache hit | | \`merge-unit-and-component-view-tests\` | 1 | ~2 min install | ~30s download+extract | ~10s cache hit | | **Saved per run** | | | **~22 runner-min** | **~26 runner-min** | ## Removal guide (when Namespace passes trial) Search for \`# TEMP\` in \`ci.yml\` — there are exactly 4 clearly marked blocks: 1. \`Pack CI JS deps\` + \`Upload CI JS deps artifact\` in \`prepare-ci-js-deps\` 2. \`Download CI JS deps artifact\` + \`Extract CI JS deps\` in each of the 3 consumer jobs Delete those blocks. No changes needed to \`setup-ci-js-deps/action.yml\`. ## Manual testing steps \`\`\`gherkin Given a pull request runs CI on a non-Namespace runner (ubuntu-latest) When prepare-ci-js-deps completes Then a ci-js-deps.tar.gz artifact is uploaded And each unit-tests and component-view-tests shard downloads and extracts it And all shards skip yarn install and pass Given a pull request runs CI on a Namespace runner When prepare-ci-js-deps completes on a warm cache volume Then all consumer shards mount the cache volume and skip yarn install And all shards pass Given the Namespace cache volume is cold (yarn.lock changed) When prepare-ci-js-deps runs Then it installs dependencies and warms the volume And all consumer shards hit the warm volume and skip install \`\`\` ## References - ADR: \`docs/ci-dependency-setup-reuse-adr.md\` - Related action: \`.github/actions/restore-node-modules-permissions\`  --- > [!NOTE] > **Medium Risk** > Changes CI job dependencies and dependency-caching behavior for unit/component-view shards; misconfiguration could cause flaky installs or missing artifacts, but impact is limited to CI execution. > > **Overview** > **Reduces redundant JS dependency setup across CI test shards.** Adds a new composite action `setup-ci-js-deps` that configures Namespace cache, sets up Node, and conditionally runs `yarn install --immutable` + `yarn setup:github-ci --node` based on workspace freshness. > > Introduces a `prepare-ci-js-deps` job that runs once before `unit-tests`, `component-view-tests`, and `merge-unit-and-component-view-tests` to warm Namespace shared cache; for non-Namespace runners it packs and uploads a short-lived `ci-js-deps` artifact that consumer shards download/extract to skip their own installs. > > Updates test/merge jobs to depend on `prepare-ci-js-deps`, replaces their inline dependency setup with the new action + temporary artifact restore steps, tightens `if` conditions to require successful `get_requirements`, and adds `prepare-ci-js-deps` to the `check-all-jobs-pass` gate. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit d9e8501. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup>
diff --git a/.github/actions/setup-ci-js-deps/action.yml b/.github/actions/setup-ci-js-deps/action.yml
@@ -0,0 +1,60 @@
+name: 'Setup CI JS dependencies'
+description: >
+  Sets up node_modules and project build outputs for CI jobs.
+  On Namespace runners, mounts the shared cache volume then runs
+  yarn install --immutable to sync with the current yarn.lock.
+  On non-Namespace runners, skips install when node_modules is
+  already present from a same-run artifact; otherwise installs from scratch.
+
+inputs:
+  runner_provider:
+    description: 'Runner provider (`namespace` or any GitHub-hosted value).'
+    required: false
+    default: 'current'
+
+runs:
+  using: 'composite'
+  steps:
+    - name: Configure Namespace cache
+      if: ${{ inputs.runner_provider == 'namespace' }}
+      uses: namespacelabs/nscloud-cache-action@15799a6b54e5765f85b2aac25b3f0df43ed571c0 # v1
+      with:
+        path: |
+          ~/.cache/yarn
+          .metamask
+          node_modules
+          .yarn/cache
+
+    - uses: actions/setup-node@v6
+      with:
+        node-version-file: '.nvmrc'
+        cache: ${{ inputs.runner_provider != 'namespace' && 'yarn' || '' }}
+
+    # Namespace: always run install so the shared volume stays in sync with yarn.lock.
+    # Non-Namespace: skip install when node_modules was extracted from a same-run
+    # artifact; run install when starting from a clean workspace.
+    - name: Determine if install is needed
+      id: check-deps
+      shell: bash
+      run: |
+        if [ "${{ inputs.runner_provider }}" != "namespace" ] && \
+           [ -d node_modules ] && \
+           [ -f app/util/termsOfUse/termsOfUseContent.ts ]; then
+          echo "needs-install=false" >> "$GITHUB_OUTPUT"
+        else
+          echo "needs-install=true" >> "$GITHUB_OUTPUT"
+        fi
+
+    - name: Install Yarn dependencies with retry
+      if: ${{ steps.check-deps.outputs.needs-install == 'true' }}
+      uses: nick-fields/retry@ce71cc2ab81d554ebbe88c79ab5975992d79ba08 # v3.0.2
+      with:
+        timeout_minutes: 10
+        max_attempts: 3
+        retry_wait_seconds: 30
+        command: yarn install --immutable
+
+    - name: Run project setup
+      if: ${{ steps.check-deps.outputs.needs-install == 'true' }}
+      shell: bash
+      run: yarn setup:github-ci --node
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
@@ -600,12 +600,41 @@ jobs:
         run: ${{ steps.download-actionlint.outputs.executable }} -color -config-file .github/actionlint.yaml
         shell: bash
 
+  # Warms the Namespace cache volume before test shards run. On non-Namespace
+  # runners uploads a ci-js-deps artifact so consumers can skip their own install.
+  prepare-ci-js-deps:
+    name: Prepare CI JS dependencies
+    runs-on: ${{ inputs.runner_provider == 'namespace' && 'namespace-profile-metamask-ci-linux' || 'ubuntu-latest' }}
+    if: ${{ needs.get_requirements.outputs.skip_everything != 'true' }}
+    needs:
+      - get_requirements
+    steps:
+      - uses: actions/checkout@v6
+      - uses: ./.github/actions/setup-ci-js-deps
+        with:
+          runner_provider: ${{ inputs.runner_provider }}
+      # TEMP: artifact fallback for non-Namespace runners.
+      # Remove these two steps once Namespace passes the trial and becomes the default.
+      - name: Pack CI JS deps
+        if: ${{ inputs.runner_provider != 'namespace' }}
+        run: tar -czf ci-js-deps.tar.gz node_modules app/util/termsOfUse/termsOfUseContent.ts
+      - name: Upload CI JS deps artifact
+        if: ${{ inputs.runner_provider != 'namespace' }}
+        uses: actions/upload-artifact@v4
+        with:
+          name: ci-js-deps
+          path: ci-js-deps.tar.gz
+          retention-days: 1
+          compression-level: 0
+          if-no-files-found: error
+
   unit-tests:
     name: Unit tests (${{ matrix.shard }})
     runs-on: ${{ inputs.runner_provider == 'namespace' && 'namespace-profile-metamask-ci-linux' || 'ubuntu-latest' }}
-    if: ${{ needs.get_requirements.outputs.skip_everything != 'true' }}
+    if: ${{ !cancelled() && needs.get_requirements.result == 'success' && needs.get_requirements.outputs.skip_everything != 'true' }}
     needs:
       - get_requirements
+      - prepare-ci-js-deps
     strategy:
       matrix:
         shard: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
@@ -614,28 +643,20 @@ jobs:
         if: ${{ inputs.runner_provider == 'namespace' }}
       - uses: actions/checkout@v6
         if: ${{ inputs.runner_provider != 'namespace' }}
-      - name: Configure Namespace cache
-        if: ${{ inputs.runner_provider == 'namespace' }}
-        uses: namespacelabs/nscloud-cache-action@15799a6b54e5765f85b2aac25b3f0df43ed571c0 # v1
-        with:
-          path: |
-            ~/.cache/yarn
-            .metamask
-            node_modules
-            .yarn/cache
-      - uses: actions/setup-node@v6
+      # TEMP: artifact fallback for non-Namespace runners.
+      # Remove these two steps once Namespace passes the trial and becomes the default.
+      - name: Download CI JS deps artifact
+        if: ${{ inputs.runner_provider != 'namespace' && needs.prepare-ci-js-deps.result == 'success' }}
+        uses: actions/download-artifact@v4
         with:
-          node-version-file: '.nvmrc'
-          cache: ${{ inputs.runner_provider != 'namespace' && 'yarn' || '' }}
-      - name: Install Yarn dependencies with retry
-        uses: nick-fields/retry@ce71cc2ab81d554ebbe88c79ab5975992d79ba08 #v3.0.2
+          name: ci-js-deps
+          path: .
+      - name: Extract CI JS deps
+        if: ${{ inputs.runner_provider != 'namespace' && needs.prepare-ci-js-deps.result == 'success' }}
+        run: tar -xzf ci-js-deps.tar.gz && rm ci-js-deps.tar.gz
+      - uses: ./.github/actions/setup-ci-js-deps
         with:
-          timeout_minutes: 10
-          max_attempts: 3
-          retry_wait_seconds: 30
-          command: yarn install --immutable
-      - name: Clean state and following up dependencies installation
-        run: yarn setup:github-ci --node
+          runner_provider: ${{ inputs.runner_provider }}
       - name: Prepare results directory
         run: mkdir -p tests/results
       # The "10" in this command is the total number of shards. It must be kept
@@ -681,45 +702,27 @@ jobs:
   # threshold calculation is accurate.
   merge-unit-and-component-view-tests:
     runs-on: ${{ inputs.runner_provider == 'namespace' && 'namespace-profile-metamask-ci-linux' || 'ubuntu-latest' }}
-    needs: [unit-tests, component-view-tests]
+    needs: [prepare-ci-js-deps, unit-tests, component-view-tests]
     if: ${{ !cancelled() && github.event_name != 'merge_group' }}
     steps:
       - uses: namespacelabs/nscloud-checkout-action@938f5d2d403d6224d9a0c0dc559b1dae09c2ede4 # v8.1.1
         if: ${{ inputs.runner_provider == 'namespace' }}
       - uses: actions/checkout@v6
         if: ${{ inputs.runner_provider != 'namespace' }}
-      - name: Configure Namespace cache
-        if: ${{ inputs.runner_provider == 'namespace' }}
-        uses: namespacelabs/nscloud-cache-action@15799a6b54e5765f85b2aac25b3f0df43ed571c0 # v1
-        with:
-          path: |
-            ~/.cache/yarn
-            .metamask
-            node_modules
-            .yarn/cache
-      - name: Restore node_modules cache
-        if: ${{ inputs.runner_provider != 'namespace' }}
-        id: cache-node-modules
-        uses: actions/cache@v4
-        with:
-          path: |
-            node_modules
-            .yarn/install-state.gz
-          key: ${{ runner.os }}-node-modules-${{ hashFiles('yarn.lock') }}
-      - uses: actions/setup-node@v6
+      # TEMP: artifact fallback for non-Namespace runners.
+      # Remove these two steps once Namespace passes the trial and becomes the default.
+      - name: Download CI JS deps artifact
+        if: ${{ inputs.runner_provider != 'namespace' && needs.prepare-ci-js-deps.result == 'success' }}
+        uses: actions/download-artifact@v4
         with:
-          node-version-file: '.nvmrc'
-          cache: ${{ inputs.runner_provider != 'namespace' && 'yarn' || '' }}
-      - name: Install Yarn dependencies with retry
-        if: ${{ inputs.runner_provider == 'namespace' || steps.cache-node-modules.outputs.cache-hit != 'true' }}
-        uses: nick-fields/retry@ce71cc2ab81d554ebbe88c79ab5975992d79ba08 #v3.0.2
+          name: ci-js-deps
+          path: .
+      - name: Extract CI JS deps
+        if: ${{ inputs.runner_provider != 'namespace' && needs.prepare-ci-js-deps.result == 'success' }}
+        run: tar -xzf ci-js-deps.tar.gz && rm ci-js-deps.tar.gz
+      - uses: ./.github/actions/setup-ci-js-deps
         with:
-          timeout_minutes: 10
-          max_attempts: 3
-          retry_wait_seconds: 30
-          command: yarn install --immutable
-      - name: Clean state and following up dependencies installation
-        run: yarn setup:github-ci --node
+          runner_provider: ${{ inputs.runner_provider }}
       - name: Download coverage shards (Namespace)
         if: ${{ inputs.runner_provider == 'namespace' }}
         uses: namespace-actions/download-artifact@7cbad919e4b0e09f17e9d6311a444ff002992b5b # v2.0.1
@@ -879,9 +882,10 @@ jobs:
   component-view-tests:
     name: Component view tests
     runs-on: ${{ inputs.runner_provider == 'namespace' && 'namespace-profile-metamask-ci-linux' || 'ubuntu-latest' }}
-    if: ${{ needs.get_requirements.outputs.skip_everything != 'true' }}
+    if: ${{ !cancelled() && needs.get_requirements.result == 'success' && needs.get_requirements.outputs.skip_everything != 'true' }}
     needs:
       - get_requirements
+      - prepare-ci-js-deps
     strategy:
       matrix:
         shard: [1, 2]
@@ -890,38 +894,20 @@ jobs:
         if: ${{ inputs.runner_provider == 'namespace' }}
       - uses: actions/checkout@v6
         if: ${{ inputs.runner_provider != 'namespace' }}
-      - name: Configure Namespace cache
-        if: ${{ inputs.runner_provider == 'namespace' }}
-        uses: namespacelabs/nscloud-cache-action@15799a6b54e5765f85b2aac25b3f0df43ed571c0 # v1
-        with:
-          path: |
-            ~/.cache/yarn
-            .metamask
-            node_modules
-            .yarn/cache
-      - name: Restore node_modules cache
-        if: ${{ inputs.runner_provider != 'namespace' }}
-        id: cache-node-modules
-        uses: actions/cache@v4
-        with:
-          path: |
-            node_modules
-            .yarn/install-state.gz
-          key: ${{ runner.os }}-node-modules-${{ hashFiles('yarn.lock') }}
-      - uses: actions/setup-node@v6
+      # TEMP: artifact fallback for non-Namespace runners.
+      # Remove these two steps once Namespace passes the trial and becomes the default.
+      - name: Download CI JS deps artifact
+        if: ${{ inputs.runner_provider != 'namespace' && needs.prepare-ci-js-deps.result == 'success' }}
+        uses: actions/download-artifact@v4
         with:
-          node-version-file: '.nvmrc'
-          cache: ${{ inputs.runner_provider != 'namespace' && 'yarn' || '' }}
-      - name: Install Yarn dependencies with retry
-        if: ${{ inputs.runner_provider == 'namespace' || steps.cache-node-modules.outputs.cache-hit != 'true' }}
-        uses: nick-fields/retry@ce71cc2ab81d554ebbe88c79ab5975992d79ba08 #v3.0.2
+          name: ci-js-deps
+          path: .
+      - name: Extract CI JS deps
+        if: ${{ inputs.runner_provider != 'namespace' && needs.prepare-ci-js-deps.result == 'success' }}
+        run: tar -xzf ci-js-deps.tar.gz && rm ci-js-deps.tar.gz
+      - uses: ./.github/actions/setup-ci-js-deps
         with:
-          timeout_minutes: 10
-          max_attempts: 3
-          retry_wait_seconds: 30
-          command: yarn install --immutable
-      - name: Clean state and following up dependencies installation
-        run: yarn setup:github-ci --node
+          runner_provider: ${{ inputs.runner_provider }}
       - name: Prepare results directory
         run: mkdir -p tests/results
       - run: |
@@ -1288,6 +1274,7 @@ jobs:
       - unit-tests
       - component-view-tests
       - check-workflows
+      - prepare-ci-js-deps
       - js-bundle-size-check
       - sonar-cloud-quality-gate-status
       - build-android-apks
