-
-
Notifications
You must be signed in to change notification settings - Fork 1.6k
ci: reuse native E2E builds across commits and PRs #29247
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 20 commits
Commits
Show all changes
48 commits
Select commit
Hold shift + click to select a range
b947600
reusable build e2e and force-builds label to force if it is failing
tommasini 75a51d6
Merge branch 'main' into ci/reusable-build-e2e
tommasini 7276cfa
fix permissions
tommasini c4e511a
empty commit to test reusable e2e builds
tommasini a176548
Merge branch 'main' into ci/reusable-build-e2e
tommasini 21c39e4
minimize fingerprint changes detection; fix fingerprint missmatch for…
tommasini 1dc79bb
address cursor bug bots
tommasini cc8decd
Merge branch 'main' into ci/reusable-build-e2e
tommasini d669669
fix cursor bug bot
tommasini b0c3bb5
Merge branch 'main' into ci/reusable-build-e2e
tommasini 6dc1273
test cache
tommasini a0b8627
v2
tommasini caf870a
test cache v2
tommasini 6d4fefe
v3
tommasini d9c1383
v3.1
tommasini 0b80d53
Merge branch 'main' into ci/reusable-build-e2e-v3
tommasini 5e125bf
metro cache dep
tommasini fe59281
deduplicate
tommasini 9fe885b
Merge branch 'main' into ci/reusable-build-e2e-v3
tommasini b753174
test cache
tommasini a050212
revert fingerprint changes and keep ota fingerprint config intact
tommasini f628dbc
remove repack optimizations
tommasini 22e1eb0
revert fingerprint config changes
tommasini 6cd7c44
ci: add native build artifact reuse across PRs
tommasini 8088a26
merge main and solve conflicts
tommasini 15ea6a0
remove unnecessary foundry installation if cache is missed, since it …
tommasini 6d14511
bring back !isCancelled to build ios and build android
tommasini 65f6838
test cache
tommasini ba160a7
test cache 2 (last commit was to distant, probably cache was already …
tommasini 2d346d9
Drop the conclusion filter. Keep only the status filter (completed/in…
tommasini f3197fd
test cache
tommasini a542923
revert unnecessary changes
tommasini 3774644
merge main and solve conflicts
tommasini b0e51f0
merge main and fix conflitcs
tommasini 9ec0fef
give missing permissions to build ios and build android
tommasini 0dde74a
address to low sev cursor bug bot
tommasini 4b8f7f0
reuse setup e2e env to clean android e2e build file and add condition…
tommasini a06210c
remove unnecessary permission on build ios e2e
tommasini a1a60ec
Merge branch 'main' into ci/reusable-build-e2e-v3
tommasini 29a075e
remove unnecessary permission on build ios e2e
tommasini 2dfff54
Missing JDK / ANDROID_HOME / keystore on Namespace; Redundant cirrusl…
tommasini d7dd2da
Added steps.force-builds.outputs.force != 'true' to the if of both AP…
tommasini d2959a4
test cache
tommasini b30dc0e
add retry to build ios e2e
tommasini d7c9445
accept all current, conflicts from code that was added to rn update t…
tommasini 362701c
address cursor bug bot
tommasini f833d69
Merge branch 'main' into ci/reusable-build-e2e-v3
tommasini 89e6bc5
Merge branch 'main' into ci/reusable-build-e2e-v3
tommasini File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,81 @@ | ||
| name: 'Check force-builds override' | ||
| description: >- | ||
| Detects whether the current workflow run should bypass native build reuse | ||
| (both the GHA cache and the cross-run artifact lookup) and always compile | ||
| fresh. The override is honored on `pull_request` events via a `force-builds` | ||
| label OR a `[force-builds]` token in the head commit message. It is | ||
| intentionally ignored on `merge_group` and `push` events so the merge queue | ||
| always uses hash-verified reuse. | ||
|
|
||
| inputs: | ||
| github-token: | ||
| description: >- | ||
| GitHub token with `pull-requests: read` (for label lookup) and | ||
| `contents: read` (to fetch the head commit message via the REST API). | ||
| required: true | ||
| label-name: | ||
| description: 'PR label that, when present, forces fresh builds' | ||
| required: false | ||
| default: 'force-builds' | ||
| commit-tag: | ||
| description: 'Case-sensitive substring in the head commit message that forces fresh builds' | ||
| required: false | ||
| default: '[force-builds]' | ||
|
|
||
| outputs: | ||
| force: | ||
| description: "'true' when fresh builds should be forced, otherwise 'false'" | ||
| value: ${{ steps.compute.outputs.force }} | ||
|
|
||
| runs: | ||
| using: 'composite' | ||
| steps: | ||
| - name: Compute force-builds flag | ||
| id: compute | ||
| shell: bash | ||
| env: | ||
| GH_TOKEN: ${{ inputs.github-token }} | ||
| LABEL_NAME: ${{ inputs.label-name }} | ||
| COMMIT_TAG: ${{ inputs.commit-tag }} | ||
| EVENT_NAME: ${{ github.event_name }} | ||
| HEAD_COMMIT_HASH: ${{ github.event.pull_request.head.sha || github.sha }} | ||
| PR_NUMBER: ${{ github.event.pull_request.number }} | ||
| REPOSITORY: ${{ github.repository }} | ||
| run: | | ||
| FORCE="false" | ||
|
|
||
| if [[ "$EVENT_NAME" != "pull_request" ]]; then | ||
| echo "Event is $EVENT_NAME; force-builds override is ignored outside pull_request events." | ||
| echo "force=$FORCE" >> "$GITHUB_OUTPUT" | ||
| exit 0 | ||
| fi | ||
|
|
||
| # Commit-message tag. | ||
| COMMIT_MESSAGE="" | ||
| if COMMIT_MESSAGE=$(gh api \ | ||
| "repos/$REPOSITORY/commits/$HEAD_COMMIT_HASH" \ | ||
| --jq '.commit.message' 2>/dev/null); then | ||
| if printf '%s' "$COMMIT_MESSAGE" \ | ||
| | grep --fixed-strings --quiet "$COMMIT_TAG"; then | ||
| echo "-> force=true because '$COMMIT_TAG' was found in commit message of $HEAD_COMMIT_HASH" | ||
| FORCE="true" | ||
| fi | ||
| else | ||
| echo "::warning::Failed to fetch commit message for $HEAD_COMMIT_HASH via GitHub API; commit-tag force-builds check skipped for this run (the '$LABEL_NAME' label path still works)." | ||
| fi | ||
|
|
||
| # PR label | ||
| if [[ -n "$PR_NUMBER" ]]; then | ||
| if gh pr view "$PR_NUMBER" --repo "$REPOSITORY" \ | ||
| --json labels --jq '.labels[].name' \ | ||
| | grep -qx "$LABEL_NAME"; then | ||
|
cursor[bot] marked this conversation as resolved.
Outdated
|
||
| echo "-> force=true because '$LABEL_NAME' label is applied to PR #$PR_NUMBER" | ||
| FORCE="true" | ||
| fi | ||
| fi | ||
|
|
||
| if [[ "$FORCE" == "false" ]]; then | ||
| echo "No force-builds override active." | ||
| fi | ||
|
|
||
| echo "force=$FORCE" >> "$GITHUB_OUTPUT" | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,262 @@ | ||
| name: 'Find reusable build from prior run' | ||
| description: >- | ||
| Searches recent workflow runs across three tiers (same branch, base branch, | ||
| then any open PR branch) for a run whose `build-source-hash` commit status | ||
| matches the current fingerprint AND whose required build artifacts are still | ||
| available. If a match is found, outputs the run id so a subsequent | ||
| `actions/download-artifact` step can pull the artifacts directly instead of | ||
| triggering a fresh native build. | ||
|
|
||
| The third (cross-PR) tier is required because GitHub's `listWorkflowRuns` | ||
| `branch` parameter filters against `head_branch` — the PR source branch for | ||
| `pull_request` events — so branch-scoped lookups can never discover other | ||
| PRs' runs. The cross-PR tier drops the branch filter and instead uses | ||
| `event: pull_request` to let the fingerprint itself act as the cross-PR | ||
| deduplication key. | ||
|
|
||
| inputs: | ||
| fingerprint: | ||
| description: 'The @expo/fingerprint hash the candidate must match' | ||
| required: true | ||
| artifact-names: | ||
| description: 'JSON array of artifact names that must all be present on the candidate run' | ||
| required: true | ||
| github-token: | ||
| description: 'GitHub token with `actions: read` and `statuses: read` permissions' | ||
| required: true | ||
| workflow-file: | ||
| description: 'Workflow filename whose runs will be searched' | ||
| required: false | ||
| default: 'ci.yml' | ||
| base-branch: | ||
| description: 'Fallback branch when no same-branch match is found' | ||
| required: false | ||
| default: 'main' | ||
| status-context: | ||
| description: 'Commit status context that carries the fingerprint' | ||
| required: false | ||
| default: 'build-source-hash' | ||
| max-candidates-per-branch: | ||
| description: 'How many recent runs to inspect per branch-scoped tier (same-branch, base-branch)' | ||
| required: false | ||
| default: '10' | ||
| max-candidates-cross-pr: | ||
| description: >- | ||
| How many recent `pull_request`-event runs (across all branches) to inspect | ||
| in the cross-PR tier. The fingerprint filter is highly discriminating, so | ||
| the practical cost is one `getCombinedStatusForRef` call per candidate | ||
| until a match is found. | ||
| required: false | ||
| default: '30' | ||
|
|
||
| outputs: | ||
| found: | ||
| description: "'true' when a reusable run was found" | ||
| value: ${{ steps.lookup.outputs.found }} | ||
| run-id: | ||
| description: 'Workflow run id that produced the reusable artifacts' | ||
| value: ${{ steps.lookup.outputs.run-id }} | ||
| source-sha: | ||
| description: 'Commit SHA of the reusable run' | ||
| value: ${{ steps.lookup.outputs.source-sha }} | ||
| source-branch: | ||
| description: 'Branch of the reusable run (same-branch or base-branch)' | ||
| value: ${{ steps.lookup.outputs.source-branch }} | ||
|
|
||
| runs: | ||
| using: 'composite' | ||
| steps: | ||
| - name: Search prior runs for matching fingerprint | ||
| id: lookup | ||
| uses: actions/github-script@v7 | ||
| continue-on-error: true | ||
| env: | ||
| TARGET_FINGERPRINT: ${{ inputs.fingerprint }} | ||
| ARTIFACT_NAMES_JSON: ${{ inputs.artifact-names }} | ||
| WORKFLOW_FILE: ${{ inputs.workflow-file }} | ||
| BASE_BRANCH: ${{ inputs.base-branch }} | ||
| STATUS_CONTEXT: ${{ inputs.status-context }} | ||
| MAX_CANDIDATES: ${{ inputs.max-candidates-per-branch }} | ||
| MAX_CANDIDATES_CROSS_PR: ${{ inputs.max-candidates-cross-pr }} | ||
| HEAD_BRANCH: ${{ github.head_ref || github.ref_name }} | ||
| HEAD_SHA: ${{ github.event.pull_request.head.sha || github.sha }} | ||
| CURRENT_RUN_ID: ${{ github.run_id }} | ||
| with: | ||
| github-token: ${{ inputs.github-token }} | ||
| script: | | ||
| const { | ||
| TARGET_FINGERPRINT, | ||
| ARTIFACT_NAMES_JSON, | ||
| WORKFLOW_FILE, | ||
| BASE_BRANCH, | ||
| STATUS_CONTEXT, | ||
| MAX_CANDIDATES, | ||
| MAX_CANDIDATES_CROSS_PR, | ||
| HEAD_BRANCH, | ||
| HEAD_SHA, | ||
| CURRENT_RUN_ID, | ||
| } = process.env; | ||
|
|
||
| const setNotFound = () => { | ||
| core.setOutput('found', 'false'); | ||
| core.setOutput('run-id', ''); | ||
| core.setOutput('source-sha', ''); | ||
| core.setOutput('source-branch', ''); | ||
| }; | ||
|
|
||
| if (!TARGET_FINGERPRINT) { | ||
| core.warning('No fingerprint provided; skipping lookup'); | ||
| setNotFound(); | ||
| return; | ||
| } | ||
|
|
||
| let requiredArtifacts; | ||
| try { | ||
| requiredArtifacts = JSON.parse(ARTIFACT_NAMES_JSON); | ||
| } catch (err) { | ||
| core.warning(`Could not parse artifact-names input: ${err.message}`); | ||
| setNotFound(); | ||
| return; | ||
| } | ||
| if (!Array.isArray(requiredArtifacts) || requiredArtifacts.length === 0) { | ||
| core.warning('artifact-names must be a non-empty JSON array'); | ||
| setNotFound(); | ||
| return; | ||
| } | ||
|
|
||
| const maxCandidates = Number(MAX_CANDIDATES) || 10; | ||
| const maxCandidatesCrossPr = Number(MAX_CANDIDATES_CROSS_PR) || 30; | ||
| const currentRunId = String(CURRENT_RUN_ID); | ||
|
|
||
| // Three-tier discovery: | ||
| // 1. same-branch — fastest path, catches retries and new commits | ||
| // on the current PR. | ||
| // 2. base-branch — catches post-merge CI runs on `main`. Only | ||
| // matches `push`-event runs (pull_request runs | ||
| // have head_branch=<source branch>, not main). | ||
| // 3. cross-pr — searches recent `pull_request` runs across | ||
| // ALL source branches so two unrelated PRs with | ||
| // the same fingerprint can reuse each other's | ||
| // artifacts. This tier deliberately drops the | ||
| // `branch` filter; without it, branch-scoped | ||
| // lookups can never discover another PR's run | ||
| // (GitHub filters `branch` against head_branch, | ||
| // which is the PR source branch). | ||
| const tiers = [ | ||
| { | ||
| label: `same-branch (branch=${HEAD_BRANCH})`, | ||
| params: { branch: HEAD_BRANCH, per_page: maxCandidates }, | ||
| }, | ||
| ]; | ||
| if (BASE_BRANCH && BASE_BRANCH !== HEAD_BRANCH) { | ||
| tiers.push({ | ||
| label: `base-branch (branch=${BASE_BRANCH})`, | ||
| params: { branch: BASE_BRANCH, per_page: maxCandidates }, | ||
| }); | ||
| } | ||
| tiers.push({ | ||
| label: `cross-pr (event=pull_request, any branch, last ${maxCandidatesCrossPr} runs)`, | ||
| params: { event: 'pull_request', per_page: maxCandidatesCrossPr }, | ||
| // Skip runs already visited by the same-branch tier to avoid | ||
| // wasting API calls on duplicates. | ||
| skipHeadBranch: HEAD_BRANCH, | ||
| }); | ||
|
|
||
| async function getFingerprintForSha(sha) { | ||
| try { | ||
| const { data } = await github.rest.repos.getCombinedStatusForRef({ | ||
| owner: context.repo.owner, | ||
| repo: context.repo.repo, | ||
| ref: sha, | ||
| per_page: 100, | ||
| }); | ||
| const status = data.statuses.find((s) => s.context === STATUS_CONTEXT); | ||
| return status ? status.description : null; | ||
| } catch (err) { | ||
| core.info(`getCombinedStatusForRef failed for ${sha}: ${err.message}`); | ||
| return null; | ||
| } | ||
| } | ||
|
|
||
| async function hasAllArtifacts(runId) { | ||
| try { | ||
| const artifacts = await github.paginate( | ||
| github.rest.actions.listWorkflowRunArtifacts, | ||
| { | ||
| owner: context.repo.owner, | ||
| repo: context.repo.repo, | ||
| run_id: runId, | ||
| per_page: 100, | ||
| }, | ||
| ); | ||
| const available = new Set( | ||
| artifacts | ||
| .filter((a) => !a.expired) | ||
| .map((a) => a.name), | ||
| ); | ||
| const missing = requiredArtifacts.filter((n) => !available.has(n)); | ||
| if (missing.length > 0) { | ||
| core.info(`Run ${runId} missing artifacts: ${missing.join(', ')}`); | ||
| return false; | ||
| } | ||
| return true; | ||
| } catch (err) { | ||
| core.info(`listWorkflowRunArtifacts failed for ${runId}: ${err.message}`); | ||
| return false; | ||
| } | ||
| } | ||
|
|
||
| const seenRunIds = new Set(); | ||
| seenRunIds.add(currentRunId); | ||
|
|
||
| for (const tier of tiers) { | ||
| core.info(`Searching tier: ${tier.label}`); | ||
| let runs; | ||
| try { | ||
| const { data } = await github.rest.actions.listWorkflowRuns({ | ||
| owner: context.repo.owner, | ||
| repo: context.repo.repo, | ||
| workflow_id: WORKFLOW_FILE, | ||
| ...tier.params, | ||
| }); | ||
| runs = data.workflow_runs || []; | ||
| } catch (err) { | ||
| core.warning(`listWorkflowRuns failed for tier "${tier.label}": ${err.message}`); | ||
| continue; | ||
| } | ||
|
|
||
| for (const run of runs) { | ||
| const runIdStr = String(run.id); | ||
| if (seenRunIds.has(runIdStr)) continue; | ||
| seenRunIds.add(runIdStr); | ||
|
|
||
| if (tier.skipHeadBranch && run.head_branch === tier.skipHeadBranch) continue; | ||
| if (run.status !== 'completed' && run.status !== 'in_progress') continue; | ||
| if ( | ||
| run.status === 'completed' && | ||
| run.conclusion && | ||
| run.conclusion !== 'success' && | ||
| run.conclusion !== 'neutral' | ||
| ) { | ||
| continue; | ||
| } | ||
|
|
||
| const fingerprint = await getFingerprintForSha(run.head_sha); | ||
| if (!fingerprint) continue; | ||
| if (fingerprint !== TARGET_FINGERPRINT) continue; | ||
|
|
||
| if (!(await hasAllArtifacts(run.id))) continue; | ||
|
|
||
| core.info( | ||
| `Match: tier="${tier.label}" run=${run.id} sha=${run.head_sha} branch=${run.head_branch} url=${run.html_url}`, | ||
| ); | ||
| core.setOutput('found', 'true'); | ||
| core.setOutput('run-id', runIdStr); | ||
| core.setOutput('source-sha', run.head_sha); | ||
| core.setOutput('source-branch', run.head_branch || ''); | ||
| return; | ||
| } | ||
| } | ||
|
|
||
| core.info('No reusable build found across any tier'); | ||
| setNotFound(); |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Again these are two different commits. I think the one we want here is always
github.event.pull_request.head.sha, since we only use this value forpull_requesttriggers, and since we're looking for a specific commit message (in cases wheregithub.shadiffers, it would be the auto-generated merge commit, which is never what we want)