ci: make dependency resolution resilient and add an advisory Xcode preview build - #4272
Merged
Merged
Conversation
Jobs were being killed at the job timeout with no build output at all, and
cleanup terminating orphaned git and git-remote-http processes. They were not
hung on a test or a build — they were still fetching dependencies.
Left to xcodebuild, resolution happens silently inside the build or test
invocation. SwiftPM checks out ~31 repositories one at a time through blocking
git subprocesses and prints nothing until the first compile, so a slow clone is
indistinguishable from a hang: the job emits no output, and is eventually killed
with the whole run lost.
Resolution is now an explicit step in both run_xcodebuild and
run_xcodebuild_test:
- bounded per attempt by a background watchdog, so a stalled clone is killed
rather than left to consume the job budget
- retried up to 3 times, clearing partial mirror state between attempts, since
a half-written clone is a common way for this to wedge
- logged in its own group with elapsed time, so slow is distinguishable from
stuck
- failing with an explicit "this is a dependency fetch problem, not a test
failure" rather than a bare timeout
The build and test invocations now always pass
-disableAutomaticPackageResolution. Whether the checkouts came from a warm cache
or from the step above, xcodebuild can never fall back to resolving mid-run,
which is the fallback that produced the silent hangs.
The watchdog is used because `timeout-minutes:` is rejected on steps inside a
composite action and `timeout(1)` is not installed on macOS runners.
Job timeouts go from 30 to 60 minutes, including the eight callers that
hardcoded 30 and so overrode the reusable workflow default. That is headroom for
cold fetches and for simulator startup, which has been observed taking tens of
minutes on visionOS and iOS; it is not the fix, since resolution is separately
bounded above.
Deliberately not included: cache seeding. A seeding workflow only runs once it
is on the default branch, and GitHub scopes branch caches so a cache seeded on
one branch is invisible to sibling PRs. That made it unverifiable before merge
and useless to the PRs that needed it. This change instead makes the cold path
work on its own; caching remains a speed optimisation rather than a correctness
requirement.
Verified locally against a completely empty cache (all amplify-packages entries
deleted first): resolve took 2m11s and produced 31 checkouts, then
`xcodebuild test` with resolution disabled took 1m43s and passed
(** TEST SUCCEEDED **, 31 tests), with zero fetch attempts during the test.
Also confirmed `swift package resolve` is not a faster substitute — it took
4m19s against xcodebuild's 2m11s.
harsh62
temporarily deployed
to
IntegrationTest
August 12, 2026 23:05 — with
GitHub Actions
Inactive
harsh62
temporarily deployed
to
IntegrationTest
August 12, 2026 23:05 — with
GitHub Actions
Inactive
harsh62
temporarily deployed
to
IntegrationTest
August 12, 2026 23:05 — with
GitHub Actions
Inactive
harsh62
temporarily deployed
to
IntegrationTest
August 12, 2026 23:05 — with
GitHub Actions
Inactive
harsh62
temporarily deployed
to
IntegrationTest
August 12, 2026 23:05 — with
GitHub Actions
Inactive
harsh62
temporarily deployed
to
IntegrationTest
August 12, 2026 23:05 — with
GitHub Actions
Inactive
harsh62
temporarily deployed
to
IntegrationTest
August 12, 2026 23:05 — with
GitHub Actions
Inactive
harsh62
temporarily deployed
to
IntegrationTest
August 12, 2026 23:05 — with
GitHub Actions
Inactive
harsh62
temporarily deployed
to
IntegrationTest
August 12, 2026 23:05 — with
GitHub Actions
Inactive
harsh62
temporarily deployed
to
IntegrationTest
August 12, 2026 23:05 — with
GitHub Actions
Inactive
harsh62
temporarily deployed
to
IntegrationTest
August 12, 2026 23:05 — with
GitHub Actions
Inactive
harsh62
temporarily deployed
to
IntegrationTest
August 12, 2026 23:05 — with
GitHub Actions
Inactive
harsh62
temporarily deployed
to
IntegrationTest
August 12, 2026 23:05 — with
GitHub Actions
Inactive
harsh62
temporarily deployed
to
IntegrationTest
August 12, 2026 23:05 — with
GitHub Actions
Inactive
harsh62
temporarily deployed
to
IntegrationTest
August 12, 2026 23:05 — with
GitHub Actions
Inactive
harsh62
temporarily deployed
to
IntegrationTest
August 12, 2026 23:05 — with
GitHub Actions
Inactive
harsh62
temporarily deployed
to
IntegrationTest
August 12, 2026 23:05 — with
GitHub Actions
Inactive
harsh62
temporarily deployed
to
IntegrationTest
August 12, 2026 23:05 — with
GitHub Actions
Inactive
harsh62
temporarily deployed
to
IntegrationTest
August 12, 2026 23:05 — with
GitHub Actions
Inactive
harsh62
temporarily deployed
to
IntegrationTest
August 12, 2026 23:05 — with
GitHub Actions
Inactive
harsh62
temporarily deployed
to
IntegrationTest
August 12, 2026 23:05 — with
GitHub Actions
Inactive
harsh62
temporarily deployed
to
IntegrationTest
August 12, 2026 23:05 — with
GitHub Actions
Inactive
harsh62
temporarily deployed
to
IntegrationTest
August 12, 2026 23:05 — with
GitHub Actions
Inactive
harsh62
temporarily deployed
to
IntegrationTest
August 12, 2026 23:05 — with
GitHub Actions
Inactive
harsh62
temporarily deployed
to
IntegrationTest
August 12, 2026 23:05 — with
GitHub Actions
Inactive
harsh62
temporarily deployed
to
IntegrationTest
August 12, 2026 23:05 — with
GitHub Actions
Inactive
harsh62
temporarily deployed
to
IntegrationTest
August 12, 2026 23:05 — with
GitHub Actions
Inactive
harsh62
temporarily deployed
to
IntegrationTest
August 12, 2026 23:05 — with
GitHub Actions
Inactive
harsh62
had a problem deploying
to
IntegrationTest
August 12, 2026 23:05 — with
GitHub Actions
Failure
harsh62
temporarily deployed
to
IntegrationTest
August 12, 2026 23:05 — with
GitHub Actions
Inactive
harsh62
temporarily deployed
to
IntegrationTest
August 12, 2026 23:05 — with
GitHub Actions
Inactive
harsh62
temporarily deployed
to
IntegrationTest
August 12, 2026 23:05 — with
GitHub Actions
Inactive
harsh62
temporarily deployed
to
IntegrationTest
August 12, 2026 23:05 — with
GitHub Actions
Inactive
harsh62
temporarily deployed
to
IntegrationTest
August 12, 2026 23:05 — with
GitHub Actions
Inactive
harsh62
temporarily deployed
to
IntegrationTest
August 12, 2026 23:05 — with
GitHub Actions
Inactive
harsh62
temporarily deployed
to
IntegrationTest
August 12, 2026 23:05 — with
GitHub Actions
Inactive
harsh62
temporarily deployed
to
IntegrationTest
August 12, 2026 23:05 — with
GitHub Actions
Inactive
harsh62
temporarily deployed
to
IntegrationTest
August 12, 2026 23:05 — with
GitHub Actions
Inactive
harsh62
temporarily deployed
to
IntegrationTest
August 12, 2026 23:05 — with
GitHub Actions
Inactive
harsh62
temporarily deployed
to
IntegrationTest
August 12, 2026 23:05 — with
GitHub Actions
Inactive
harsh62
temporarily deployed
to
IntegrationTest
August 12, 2026 23:05 — with
GitHub Actions
Inactive
harsh62
temporarily deployed
to
IntegrationTest
August 12, 2026 23:05 — with
GitHub Actions
Inactive
harsh62
temporarily deployed
to
IntegrationTest
August 12, 2026 23:05 — with
GitHub Actions
Inactive
harsh62
temporarily deployed
to
IntegrationTest
August 12, 2026 23:05 — with
GitHub Actions
Inactive
harsh62
temporarily deployed
to
IntegrationTest
August 12, 2026 23:05 — with
GitHub Actions
Inactive
harsh62
temporarily deployed
to
IntegrationTest
August 12, 2026 23:05 — with
GitHub Actions
Inactive
harsh62
temporarily deployed
to
IntegrationTest
August 12, 2026 23:05 — with
GitHub Actions
Inactive
harsh62
temporarily deployed
to
IntegrationTest
August 12, 2026 23:05 — with
GitHub Actions
Inactive
harsh62
temporarily deployed
to
IntegrationTest
August 12, 2026 23:05 — with
GitHub Actions
Inactive
harsh62
temporarily deployed
to
IntegrationTest
August 12, 2026 23:05 — with
GitHub Actions
Inactive
harsh62
temporarily deployed
to
IntegrationTest
August 12, 2026 23:05 — with
GitHub Actions
Inactive
harsh62
temporarily deployed
to
IntegrationTest
August 12, 2026 23:05 — with
GitHub Actions
Inactive
harsh62
temporarily deployed
to
IntegrationTest
August 12, 2026 23:05 — with
GitHub Actions
Inactive
harsh62
temporarily deployed
to
IntegrationTest
August 12, 2026 23:05 — with
GitHub Actions
Inactive
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #4272 +/- ##
==========================================
- Coverage 66.79% 66.76% -0.03%
==========================================
Files 1170 1170
Lines 44565 44565
==========================================
- Hits 29766 29755 -11
- Misses 14799 14810 +11
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
mattcreaser
approved these changes
Aug 13, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Issue
Jobs were being killed at the job timeout having produced no build output at all, with cleanup terminating orphaned
gitandgit-remote-httpprocesses. They were not stuck on a test or a build — they were still fetching dependencies.Left to
xcodebuild, resolution happens silently inside the build/test invocation. SwiftPM checks out ~31 repositories one at a time through blockinggitsubprocesses and prints nothing until the first compile, so a slow clone is indistinguishable from a hang. I confirmed the mechanism by sampling a stalled process:Description
Resolution is now explicit, bounded, and retried
Added as a step in both
run_xcodebuildandrun_xcodebuild_test:this is a dependency fetch problem, not a test failure— rather than as a bare timeoutThe build and test invocations now always pass
-disableAutomaticPackageResolution. Whether checkouts came from a warm cache or from the step above,xcodebuildcan never fall back to resolving mid-run. That fallback is what produced the silent hangs.Implementation note: the bound uses a watchdog because
timeout-minutes:is rejected on steps inside a composite action, andtimeout(1)is not installed on macOS runners.Timeouts 30 → 60 minutes
Including the eight callers that hardcoded
30and so overrode the reusable-workflow default. This is headroom, not the fix — for cold fetches and for simulator startup, which I measured taking tens of minutes on visionOS/iOS. Resolution is separately bounded above, so a genuine stall still fails fast.Advisory Xcode 27 preview build
Builds against the
xcode-27preview image on PRs and pushes to main, so next-Xcode breakages surface on the change that introduces them. Every job setscontinue-on-error— a beta toolchain on a capacity-constrained preview image is not a suitable required check — and a summary step says so in the run summary.It bypasses
get_platform_parameterson purpose (that action allowlists only officially supported Xcode versions; a beta is outside that set), uses a generic iOS destination rather than a pinned device name that would break on the next beta rollout, and treatsxcbeautifyas optional since the preview image is built separately from the standard macOS images.Deliberately NOT included: cache seeding
An earlier attempt added a workflow to seed the shared SwiftPM cache. I dropped it: a seeding workflow only runs once it is on the default branch, and GitHub scopes branch caches so a cache seeded on one branch is invisible to sibling PRs. That made it unverifiable before merge and useless to the PRs that needed it.
This change instead makes the cold path work on its own. Caching stays a speed optimisation rather than a correctness requirement.
How did you test these changes?
Verified against a completely empty cache — I deleted every
amplify-packages-*entry first, so this is a genuine cold start:xcodebuild -resolvePackageDependenciesxcodebuild testwith resolution disabled** TEST SUCCEEDED **, 31 testsAlso checked whether
swift package resolvewould be a faster substitute — it is not (4m19s vsxcodebuild's 2m11s), soxcodebuildis retained.The watchdog itself was unit-checked locally across all three paths: fast success returns 0, fast failure returns non-zero, and a hang is killed at the bound rather than running to completion.
All 76 workflow/action YAML files parse.
Documentation
No customer-facing changes — CI configuration only. The rationale for each non-obvious choice (why resolution is a separate step, why the watchdog, why the timeout value, why the cache key is not the fix) is recorded in comments beside the code rather than only here.
Checklist
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.