Skip to content

Commit da19bad

Browse files
authored
ci: reuse native E2E builds across commits and PRs (#29247)
<!-- Please submit this PR as a draft initially. Do not mark it as "Ready for review" until this PR meets the canonical Definition of Ready For Review in `docs/readme/ready-for-review.md`. In short: the template must be materially complete (not just section titles present), all status checks must be currently passing, and the only expected follow-up commits must be reviewer-driven. --> ## **Description** Replaces the branch-keyed `cirruslabs/cache` reuse layer for native E2E builds with a content-addressable cross-run lookup, driven by a `build-source-hash` commit status that is computed once on a stable runner. ### How it works ```mermaid flowchart TD A["post-build-source-hash<br/>(ubuntu-latest, pinned to PR head SHA)<br/>fingerprint = yarn fingerprint:generate"] -->|posts build-source-hash status<br/>+ exposes as job output| B["Build iOS / Android job"] B --> C{"check-force-builds<br/>label OR [force-builds] in commit?"} C -- yes --> H["Heavy native-build path<br/>setup-e2e-env + Gradle/Xcode + compile"] C -- no --> D["find-reusable-build<br/>(GitHub Actions REST API)"] D --> D1["Tier 1: same-branch runs"] D1 -- miss --> D2["Tier 2: base-branch (main) runs"] D2 -- miss --> D3["Tier 3: cross-PR runs<br/>(event=pull_request, no branch filter)"] D1 -- hit --> E["actions/download-artifact@v4<br/>by run-id"] D2 -- hit --> E D3 -- hit --> E D3 -- miss --> H E --> F["Lean repack path<br/>Node + yarn install + repack:{ios,android}"] H --> I["Upload .app / release.apk / androidTest.apk"] F --> I I -.->|feeds future runs| D ``` The probe runs **before** any heavy setup. On a hit, iOS skips Ruby/Bundler/CocoaPods/`pod install`; Android skips `setup-e2e-env`/Gradle. Only the JS bundle is rebuilt and re-packed into the cached native shell. ### Benefits vs the previous `cirruslabs/cache` layer | | Before | After (this PR) | |---|---|---| | **Reuse keyspace** | `${ref_name}` baked into key → every PR isolated; only `main → PR` fallback crossed branches | Content-addressable by fingerprint; any PR can reuse any other PR's build | | **When the lookup runs** | *After* `setup-e2e-env` (cache key needs `node_modules` to compute fingerprint) | Before any heavy setup | | **Fingerprint stability** | Recomputed per runner → drifts across macOS / Linux build / Linux CI runners; shifts on every `main` push | Computed once on `ubuntu-latest`, pinned to `pull_request.head.sha` | | **iOS reuse-hit wall time** | ~6m setup + ~5m repack | ~30-60s setup + ~1-2m repack | | **Android reuse-hit wall time** | ~5m23s | ~1-2m warm / ~2-3m cold-cache | | **Cold build (no reuse)** | ~20-25m | unchanged | ### What changed - **`ci.yml`** — new `post-build-source-hash` job emits the canonical fingerprint and exposes it as a job output consumed by both build jobs. - **New composite actions** — `find-reusable-build` (3-tier scan), `check-force-builds` (label + `[force-builds]` commit-message escape hatch; reads commit message via REST API to survive shallow checkout), `post-build-source-hash` (factored out for reuse). - **`build-{ios,android}-e2e.yml`** — probe → gate → lean-repack vs heavy-native paths. Lean paths skip the work `yarn build:repack:*` doesn't need. - **`setup-e2e-env`** — new `install-foundry` input (default `true`, opt-out for build workflows where `yarn setup:github-ci` already runs `install:foundryup`). - **Repack throughput** — `METRO_MAX_WORKERS` bumped `2 → 6` on the lean path (no Gradle/Xcode competing for RAM). - **Removed** — branch- and main-scoped `cirruslabs/cache` for `MetaMask.app` / `release.apk` / `release-androidTest.apk`. Gradle, Xcode DerivedData, and `.metamask` caches kept (gated to the heavy path). ### Safety - Fingerprint job failure or forked PR (no `statuses: write`) → empty `source-fingerprint` → all reuse steps skipped → fresh build runs. Never "no build". - The same `build-source-hash` status gates the OTA path in `push-eas-update.yml`, so a JS-only OTA cannot ship after native code changes. - Full architecture, decision tables, and failure-mode catalog: `docs/ci-build-reuse.md` (new in this PR). ## **Changelog** CHANGELOG entry: null ## **Related issues** Refs: Prior art in MetaMask extension — MetaMask/metamask-extension#41435 No issue: CI infrastructure improvement; no Jira ticket. ## **Manual testing steps** ```gherkin Feature: Reusable native E2E builds on CI Scenario: Fresh run populates the reuse pool Given this branch has no prior completed CI run with a matching fingerprint When CI runs on the latest commit Then post-build-source-hash posts a build-source-hash commit status on the PR head SHA And Build iOS E2E Apps runs setup-e2e-env and the full Xcode native compile And Build Android E2E APKs runs setup-e2e-env and the full Gradle build And both jobs upload artifacts named ${build_type}-${env}-MetaMask.app, ${build_type}-${env}-release.apk, and ${build_type}-${env}-release-androidTest.apk Scenario: Empty commit reuses the prior native build (same-branch tier) Given the branch has a prior successful run with non-expired artifacts When an empty commit is pushed Then post-build-source-hash computes and posts the same fingerprint as before And find-reusable-build reports a hit via the same-branch tier And Build iOS E2E Apps skips Ruby/Bundler/CocoaPods/Xcode setup and only runs yarn build:repack:ios And Build Android E2E APKs skips setup-e2e-env and only runs yarn build:repack:android And E2E tests run against the repacked artifacts Scenario: Cross-PR reuse finds an unrelated PR with matching fingerprint Given PR A on branch feature-a completed a fresh native build and uploaded artifacts And PR B on branch feature-b touches only JS/tests/docs (no native-affecting files) When PR B's CI runs Then find-reusable-build misses the same-branch tier (feature-b runs) And misses the base-branch tier (no recent main run with the matching fingerprint) And matches PR A's run via the cross-PR tier (event=pull_request, no branch filter) And PR B downloads PR A's artifacts by run-id and runs yarn build:repack:* Scenario: force-builds label bypasses reuse Given a PR that would otherwise hit the reuse path When the force-builds label is added and CI re-runs Then check-force-builds reports force=true And find-reusable-build does not run And both iOS and Android run a fresh native compile And when the label is removed and CI re-runs, reuse resumes normally Scenario: [force-builds] commit tag bypasses reuse under shallow checkout Given a PR whose head commit message contains [force-builds] When CI runs Then check-force-builds reads the commit message via the GitHub REST API (not git show) And reports force=true even with actions/checkout's default fetch-depth: 1 And both iOS and Android run a fresh native compile Scenario: Fingerprint job failure degrades gracefully to fresh build Given post-build-source-hash fails (e.g. broken fingerprint.config.js, transient yarn install error, or missing statuses: write on a fork) When the build workflows run Then inputs.source-fingerprint is empty And every fingerprint-keyed step is skipped And both iOS and Android run a fresh native compile And no E2E build is blocked Scenario: Metro transform cache persists across repack runs Given a reuse hit on the repack path When yarn build:repack:{ios,android} runs Then Metro uses METRO_CACHE_DIR as its FileStore root And the persisted transform cache is honored (no --reset-cache) And the E2E build runs with updates disabled (no app.manifest generation) ``` ## **Screenshots/Recordings** ### **Before** N/A — CI-only change; no UI impact. ### **After** N/A — CI-only change; no UI impact. ## **Pre-merge author checklist** - [x] I've followed [MetaMask Contributor Docs](https://github.com/MetaMask/contributor-docs) and [MetaMask Mobile Coding Standards](https://github.com/MetaMask/metamask-mobile/blob/main/.github/guidelines/CODING_GUIDELINES.md). - [x] I've completed the PR template to the best of my ability - [x] I've included tests if applicable - [x] I've documented my code using [JSDoc](https://jsdoc.app/) format if applicable - [x] I've applied the right labels on the PR (see [labeling guidelines](https://github.com/MetaMask/metamask-mobile/blob/main/.github/guidelines/LABELING_GUIDELINES.md)). Not required for external contributors. #### Performance checks (if applicable) - [x] I've tested on Android - [x] I've tested with a power user scenario - [x] I've instrumented key operations with Sentry traces for production performance metrics For performance guidelines and tooling, see the [Performance Guide](https://consensyssoftware.atlassian.net/wiki/spaces/TL1/pages/400085549067/Performance+Guide+for+Engineers). ## **Pre-merge reviewer checklist** - [ ] I've manually tested the PR (e.g. pull and build branch, run the app, test code being changed). - [ ] I confirm that this PR addresses all acceptance criteria described in the ticket it closes and includes the necessary testing evidence such as recordings and or screenshots. <!-- Generated with the help of the pr-description AI skill --> <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Medium Risk** > Changes CI build/reuse logic and caching paths for iOS/Android E2E artifacts, which could cause unexpected stale/invalid artifacts or missed rebuilds if the fingerprint/status or artifact lookup is wrong. Scoped to CI/workflows, but failures can block or slow builds across PRs. > > **Overview** > Enables **cross-run, cross-PR reuse** of native E2E build artifacts by introducing a canonical `@expo/fingerprint` published as a `build-source-hash` commit status and using it to locate/download matching artifacts from prior workflow runs. > > Build workflows now **gate between a full native build vs a lean repack path**: they first check a `force-builds` override (PR label or `[force-builds]` commit token), then attempt to find/download reusable artifacts; on a hit they skip heavy setup (Gradle/Xcode) and only run repack + lightweight dependency setup. > > CI wiring is updated to add the `post-build-source-hash` job and pass `source-fingerprint` into iOS/Android build jobs (with added `actions/statuses/pull-requests` read permissions), `setup-e2e-env` gains an `install-foundry` toggle to avoid redundant installs, and repack throughput is tuned (e.g., `METRO_MAX_WORKERS` increased on reuse paths) with clearer guidance when repack detects a broken cached iOS app. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit 89e6bc5. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY -->
1 parent f244684 commit da19bad

8 files changed

Lines changed: 762 additions & 128 deletions

File tree

Lines changed: 81 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,81 @@
1+
name: 'Check force-builds override'
2+
description: >-
3+
Detects whether the current workflow run should bypass native build reuse
4+
(both the GHA cache and the cross-run artifact lookup) and always compile
5+
fresh. The override is honored on `pull_request` events via a `force-builds`
6+
label OR a `[force-builds]` token in the head commit message. It is
7+
intentionally ignored on `merge_group` and `push` events so the merge queue
8+
always uses hash-verified reuse.
9+
10+
inputs:
11+
github-token:
12+
description: >-
13+
GitHub token with `pull-requests: read` (for label lookup) and
14+
`contents: read` (to fetch the head commit message via the REST API).
15+
required: true
16+
label-name:
17+
description: 'PR label that, when present, forces fresh builds'
18+
required: false
19+
default: 'force-builds'
20+
commit-tag:
21+
description: 'Case-sensitive substring in the head commit message that forces fresh builds'
22+
required: false
23+
default: '[force-builds]'
24+
25+
outputs:
26+
force:
27+
description: "'true' when fresh builds should be forced, otherwise 'false'"
28+
value: ${{ steps.compute.outputs.force }}
29+
30+
runs:
31+
using: 'composite'
32+
steps:
33+
- name: Compute force-builds flag
34+
id: compute
35+
shell: bash
36+
env:
37+
GH_TOKEN: ${{ inputs.github-token }}
38+
LABEL_NAME: ${{ inputs.label-name }}
39+
COMMIT_TAG: ${{ inputs.commit-tag }}
40+
EVENT_NAME: ${{ github.event_name }}
41+
HEAD_COMMIT_HASH: ${{ github.event.pull_request.head.sha }}
42+
PR_NUMBER: ${{ github.event.pull_request.number }}
43+
REPOSITORY: ${{ github.repository }}
44+
run: |
45+
FORCE="false"
46+
47+
if [[ "$EVENT_NAME" != "pull_request" ]]; then
48+
echo "Event is $EVENT_NAME; force-builds override is ignored outside pull_request events."
49+
echo "force=$FORCE" >> "$GITHUB_OUTPUT"
50+
exit 0
51+
fi
52+
53+
# Commit-message tag.
54+
COMMIT_MESSAGE=""
55+
if COMMIT_MESSAGE=$(gh api \
56+
"repos/$REPOSITORY/commits/$HEAD_COMMIT_HASH" \
57+
--jq '.commit.message' 2>/dev/null); then
58+
if printf '%s' "$COMMIT_MESSAGE" \
59+
| grep --fixed-strings --quiet "$COMMIT_TAG"; then
60+
echo "-> force=true because '$COMMIT_TAG' was found in commit message of $HEAD_COMMIT_HASH"
61+
FORCE="true"
62+
fi
63+
else
64+
echo "::warning::Failed to fetch commit message for $HEAD_COMMIT_HASH via GitHub API; commit-tag force-builds check skipped for this run (the '$LABEL_NAME' label path still works)."
65+
fi
66+
67+
# PR label
68+
if [[ -n "$PR_NUMBER" ]]; then
69+
if gh pr view "$PR_NUMBER" --repo "$REPOSITORY" \
70+
--json labels --jq '.labels[].name' \
71+
| grep --fixed-strings --line-regexp --quiet "$LABEL_NAME"; then
72+
echo "-> force=true because '$LABEL_NAME' label is applied to PR #$PR_NUMBER"
73+
FORCE="true"
74+
fi
75+
fi
76+
77+
if [[ "$FORCE" == "false" ]]; then
78+
echo "No force-builds override active."
79+
fi
80+
81+
echo "force=$FORCE" >> "$GITHUB_OUTPUT"
Lines changed: 255 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,255 @@
1+
name: 'Find reusable build from prior run'
2+
description: >-
3+
Searches recent workflow runs across three tiers (same branch, base branch,
4+
then any open PR branch) for a run whose `build-source-hash` commit status
5+
matches the current fingerprint AND whose required build artifacts are still
6+
available. If a match is found, outputs the run id so a subsequent
7+
`actions/download-artifact` step can pull the artifacts directly instead of
8+
triggering a fresh native build.
9+
10+
The third (cross-PR) tier is required because GitHub's `listWorkflowRuns`
11+
`branch` parameter filters against `head_branch` — the PR source branch for
12+
`pull_request` events — so branch-scoped lookups can never discover other
13+
PRs' runs. The cross-PR tier drops the branch filter and instead uses
14+
`event: pull_request` to let the fingerprint itself act as the cross-PR
15+
deduplication key.
16+
17+
inputs:
18+
fingerprint:
19+
description: 'The @expo/fingerprint hash the candidate must match'
20+
required: true
21+
artifact-names:
22+
description: 'JSON array of artifact names that must all be present on the candidate run'
23+
required: true
24+
github-token:
25+
description: 'GitHub token with `actions: read` and `statuses: read` permissions'
26+
required: true
27+
workflow-file:
28+
description: 'Workflow filename whose runs will be searched'
29+
required: false
30+
default: 'ci.yml'
31+
base-branch:
32+
description: 'Fallback branch when no same-branch match is found'
33+
required: false
34+
default: 'main'
35+
status-context:
36+
description: 'Commit status context that carries the fingerprint'
37+
required: false
38+
default: 'build-source-hash'
39+
max-candidates-per-branch:
40+
description: 'How many recent runs to inspect per branch-scoped tier (same-branch, base-branch)'
41+
required: false
42+
default: '10'
43+
max-candidates-cross-pr:
44+
description: >-
45+
How many recent `pull_request`-event runs (across all branches) to inspect
46+
in the cross-PR tier. The fingerprint filter is highly discriminating, so
47+
the practical cost is one `getCombinedStatusForRef` call per candidate
48+
until a match is found.
49+
required: false
50+
default: '30'
51+
52+
outputs:
53+
found:
54+
description: "'true' when a reusable run was found"
55+
value: ${{ steps.lookup.outputs.found }}
56+
run-id:
57+
description: 'Workflow run id that produced the reusable artifacts'
58+
value: ${{ steps.lookup.outputs.run-id }}
59+
source-sha:
60+
description: 'Commit SHA of the reusable run'
61+
value: ${{ steps.lookup.outputs.source-sha }}
62+
source-branch:
63+
description: 'Branch of the reusable run (same-branch or base-branch)'
64+
value: ${{ steps.lookup.outputs.source-branch }}
65+
66+
runs:
67+
using: 'composite'
68+
steps:
69+
- name: Search prior runs for matching fingerprint
70+
id: lookup
71+
uses: actions/github-script@v7
72+
continue-on-error: true
73+
env:
74+
TARGET_FINGERPRINT: ${{ inputs.fingerprint }}
75+
ARTIFACT_NAMES_JSON: ${{ inputs.artifact-names }}
76+
WORKFLOW_FILE: ${{ inputs.workflow-file }}
77+
BASE_BRANCH: ${{ inputs.base-branch }}
78+
STATUS_CONTEXT: ${{ inputs.status-context }}
79+
MAX_CANDIDATES: ${{ inputs.max-candidates-per-branch }}
80+
MAX_CANDIDATES_CROSS_PR: ${{ inputs.max-candidates-cross-pr }}
81+
HEAD_BRANCH: ${{ github.head_ref || github.ref_name }}
82+
HEAD_SHA: ${{ github.event.pull_request.head.sha || github.sha }}
83+
CURRENT_RUN_ID: ${{ github.run_id }}
84+
with:
85+
github-token: ${{ inputs.github-token }}
86+
script: |
87+
const {
88+
TARGET_FINGERPRINT,
89+
ARTIFACT_NAMES_JSON,
90+
WORKFLOW_FILE,
91+
BASE_BRANCH,
92+
STATUS_CONTEXT,
93+
MAX_CANDIDATES,
94+
MAX_CANDIDATES_CROSS_PR,
95+
HEAD_BRANCH,
96+
HEAD_SHA,
97+
CURRENT_RUN_ID,
98+
} = process.env;
99+
100+
const setNotFound = () => {
101+
core.setOutput('found', 'false');
102+
core.setOutput('run-id', '');
103+
core.setOutput('source-sha', '');
104+
core.setOutput('source-branch', '');
105+
};
106+
107+
if (!TARGET_FINGERPRINT) {
108+
core.warning('No fingerprint provided; skipping lookup');
109+
setNotFound();
110+
return;
111+
}
112+
113+
let requiredArtifacts;
114+
try {
115+
requiredArtifacts = JSON.parse(ARTIFACT_NAMES_JSON);
116+
} catch (err) {
117+
core.warning(`Could not parse artifact-names input: ${err.message}`);
118+
setNotFound();
119+
return;
120+
}
121+
if (!Array.isArray(requiredArtifacts) || requiredArtifacts.length === 0) {
122+
core.warning('artifact-names must be a non-empty JSON array');
123+
setNotFound();
124+
return;
125+
}
126+
127+
const maxCandidates = Number(MAX_CANDIDATES) || 10;
128+
const maxCandidatesCrossPr = Number(MAX_CANDIDATES_CROSS_PR) || 30;
129+
const currentRunId = String(CURRENT_RUN_ID);
130+
131+
// Three-tier discovery:
132+
// 1. same-branch — fastest path, catches retries and new commits
133+
// on the current PR.
134+
// 2. base-branch — catches post-merge CI runs on `main`. Only
135+
// matches `push`-event runs (pull_request runs
136+
// have head_branch=<source branch>, not main).
137+
// 3. cross-pr — searches recent `pull_request` runs across
138+
// ALL source branches so two unrelated PRs with
139+
// the same fingerprint can reuse each other's
140+
// artifacts. This tier deliberately drops the
141+
// `branch` filter; without it, branch-scoped
142+
// lookups can never discover another PR's run
143+
// (GitHub filters `branch` against head_branch,
144+
// which is the PR source branch).
145+
const tiers = [
146+
{
147+
label: `same-branch (branch=${HEAD_BRANCH})`,
148+
params: { branch: HEAD_BRANCH, per_page: maxCandidates },
149+
},
150+
];
151+
if (BASE_BRANCH && BASE_BRANCH !== HEAD_BRANCH) {
152+
tiers.push({
153+
label: `base-branch (branch=${BASE_BRANCH})`,
154+
params: { branch: BASE_BRANCH, per_page: maxCandidates },
155+
});
156+
}
157+
tiers.push({
158+
label: `cross-pr (event=pull_request, any branch, last ${maxCandidatesCrossPr} runs)`,
159+
params: { event: 'pull_request', per_page: maxCandidatesCrossPr },
160+
// Skip runs already visited by the same-branch tier to avoid
161+
// wasting API calls on duplicates.
162+
skipHeadBranch: HEAD_BRANCH,
163+
});
164+
165+
async function getFingerprintForSha(sha) {
166+
try {
167+
const { data } = await github.rest.repos.getCombinedStatusForRef({
168+
owner: context.repo.owner,
169+
repo: context.repo.repo,
170+
ref: sha,
171+
per_page: 100,
172+
});
173+
const status = data.statuses.find((s) => s.context === STATUS_CONTEXT);
174+
return status ? status.description : null;
175+
} catch (err) {
176+
core.info(`getCombinedStatusForRef failed for ${sha}: ${err.message}`);
177+
return null;
178+
}
179+
}
180+
181+
async function hasAllArtifacts(runId) {
182+
try {
183+
const artifacts = await github.paginate(
184+
github.rest.actions.listWorkflowRunArtifacts,
185+
{
186+
owner: context.repo.owner,
187+
repo: context.repo.repo,
188+
run_id: runId,
189+
per_page: 100,
190+
},
191+
);
192+
const available = new Set(
193+
artifacts
194+
.filter((a) => !a.expired)
195+
.map((a) => a.name),
196+
);
197+
const missing = requiredArtifacts.filter((n) => !available.has(n));
198+
if (missing.length > 0) {
199+
core.info(`Run ${runId} missing artifacts: ${missing.join(', ')}`);
200+
return false;
201+
}
202+
return true;
203+
} catch (err) {
204+
core.info(`listWorkflowRunArtifacts failed for ${runId}: ${err.message}`);
205+
return false;
206+
}
207+
}
208+
209+
const seenRunIds = new Set();
210+
seenRunIds.add(currentRunId);
211+
212+
for (const tier of tiers) {
213+
core.info(`Searching tier: ${tier.label}`);
214+
let runs;
215+
try {
216+
const { data } = await github.rest.actions.listWorkflowRuns({
217+
owner: context.repo.owner,
218+
repo: context.repo.repo,
219+
workflow_id: WORKFLOW_FILE,
220+
...tier.params,
221+
});
222+
runs = data.workflow_runs || [];
223+
} catch (err) {
224+
core.warning(`listWorkflowRuns failed for tier "${tier.label}": ${err.message}`);
225+
continue;
226+
}
227+
228+
for (const run of runs) {
229+
const runIdStr = String(run.id);
230+
if (seenRunIds.has(runIdStr)) continue;
231+
seenRunIds.add(runIdStr);
232+
233+
if (tier.skipHeadBranch && run.head_branch === tier.skipHeadBranch) continue;
234+
235+
if (run.status !== 'completed' && run.status !== 'in_progress') continue;
236+
237+
const fingerprint = await getFingerprintForSha(run.head_sha);
238+
if (!fingerprint) continue;
239+
if (fingerprint !== TARGET_FINGERPRINT) continue;
240+
241+
if (!(await hasAllArtifacts(run.id))) continue;
242+
243+
core.info(
244+
`Match: tier="${tier.label}" run=${run.id} sha=${run.head_sha} branch=${run.head_branch} url=${run.html_url}`,
245+
);
246+
core.setOutput('found', 'true');
247+
core.setOutput('run-id', runIdStr);
248+
core.setOutput('source-sha', run.head_sha);
249+
core.setOutput('source-branch', run.head_branch || '');
250+
return;
251+
}
252+
}
253+
254+
core.info('No reusable build found across any tier');
255+
setNotFound();

0 commit comments

Comments
 (0)