Optimize CI performance with sccache and Windows dev drive#2190
Open
Optimize CI performance with sccache and Windows dev drive#2190
Conversation
Use samypr100/setup-dev-drive to create a ReFS-formatted virtual disk on Windows runners. The C: drive on GitHub-hosted Windows runners does ~4,262 IOPS while a ReFS VHDX achieves ~127,823 IOPS (~30x faster). For both the `test` and `build-and-test` jobs: - Move CARGO_HOME and RUSTUP_HOME to the dev drive via env-mapping - Create a filesystem junction for the cargo target directory so it transparently redirects to the dev drive (avoids conflicts with pixi's CARGO_TARGET_DIR activation and Swatinem/rust-cache paths) https://claude.ai/code/session_012DzrboBAeV8z9BT8EYn1Sd
- Redirect TMP and TEMP to the ReFS dev drive on Windows so pytest tmp_path and other temp file operations also benefit from faster I/O - Add mozilla-actions/sccache-action for compilation caching, only active on main branch pushes (RUSTC_WRAPPER is empty on PRs so there is zero overhead); main builds seed the sccache, PRs continue to rely on Swatinem/rust-cache https://claude.ai/code/session_012DzrboBAeV8z9BT8EYn1Sd
Fix ensure-pinned-actions CI check by replacing version tags with commit SHAs for setup-dev-drive and sccache-action. https://claude.ai/code/session_012DzrboBAeV8z9BT8EYn1Sd
PowerShell's New-Item -ItemType Junction requires the target directory to exist. Create them on the dev drive first, and pipe output to Out-Null to avoid pipeline noise under $ErrorActionPreference = 'Stop'. https://claude.ai/code/session_012DzrboBAeV8z9BT8EYn1Sd
The TMP/TEMP redirect to ReFS likely causes test failures due to filesystem behavior differences between ReFS and NTFS. Remove it — the main performance wins come from the target directory junction and CARGO_HOME/RUSTUP_HOME on the dev drive anyway. https://claude.ai/code/session_012DzrboBAeV8z9BT8EYn1Sd
10GB was too small — a debug build target dir alone can be 15-20GB, and RUSTUP_HOME + CARGO_HOME add another 2GB+. Bump to 50GB (the VHDX is dynamic so it only consumes actual disk space as needed). Also remove RUSTUP_HOME from the dev drive: the toolchain is installed once then only read from, so it doesn't benefit much from faster I/O and just wastes space on the VHDX. https://claude.ai/code/session_012DzrboBAeV8z9BT8EYn1Sd
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR improves CI performance by implementing compiler caching with sccache on main branch builds and leveraging Windows ReFS dev drives for significantly faster I/O operations during builds.
Key Changes
RUSTC_WRAPPER=sccacheenvironment variable for main branch builds only, with PR builds continuing to use the existing Swatinem/rust-cache. This seeds the sccache cache on main while avoiding cache thrashing from PR builds.samypr100/setup-dev-driveaction to create a 10GB ReFS dev drive on Windows runners, providing ~30x faster I/O compared to the standard C: drive.testjob: redirectstargetdirectoryreleasejob: redirectstarget-pixidirectory (matching pixi's CARGO_TARGET_DIR configuration)testandreleaseCI jobs for consistent performance improvements.Implementation Details
github.ref == 'refs/heads/main') to avoid unnecessary cache operations in PR buildsif: runner.os == 'Windows') to avoid errors on Linux/macOS runners-Forceflag for reliabilityhttps://claude.ai/code/session_012DzrboBAeV8z9BT8EYn1Sd