Enable ccache for CI builds with GHCR caching#3960
Open
taalexander wants to merge 36 commits intoNVIDIA:mainfrom
Open
Enable ccache for CI builds with GHCR caching#3960taalexander wants to merge 36 commits intoNVIDIA:mainfrom
taalexander wants to merge 36 commits intoNVIDIA:mainfrom
Conversation
Signed-off-by: Thomas Alexander <talexander@nvidia.com>
Signed-off-by: Thomas Alexander <talexander@nvidia.com>
Signed-off-by: Thomas Alexander <talexander@nvidia.com>
Signed-off-by: Thomas Alexander <talexander@nvidia.com>
Signed-off-by: Thomas Alexander <talexander@nvidia.com>
Signed-off-by: Thomas Alexander <talexander@nvidia.com>
Signed-off-by: Thomas Alexander <talexander@nvidia.com>
Signed-off-by: Thomas Alexander <talexander@nvidia.com>
Signed-off-by: Thomas Alexander <talexander@nvidia.com>
Signed-off-by: Thomas Alexander <talexander@nvidia.com>
Signed-off-by: Thomas Alexander <talexander@nvidia.com>
Signed-off-by: Thomas Alexander <talexander@nvidia.com>
…n every commit. Signed-off-by: Thomas Alexander <talexander@nvidia.com>
Signed-off-by: Thomas Alexander <talexander@nvidia.com>
Signed-off-by: Thomas Alexander <talexander@nvidia.com>
Signed-off-by: Thomas Alexander <talexander@nvidia.com>
…10% changed. Signed-off-by: Thomas Alexander <talexander@nvidia.com>
Signed-off-by: Thomas Alexander <talexander@nvidia.com>
Signed-off-by: Thomas Alexander <talexander@nvidia.com>
Signed-off-by: Thomas Alexander <talexander@nvidia.com>
Signed-off-by: Thomas Alexander <talexander@nvidia.com>
|
CUDA Quantum Docs Bot: A preview of the documentation can be found here. |
Signed-off-by: Thomas Alexander <talexander@nvidia.com>
|
CUDA Quantum Docs Bot: A preview of the documentation can be found here. |
Signed-off-by: Thomas Alexander <talexander@nvidia.com>
|
CUDA Quantum Docs Bot: A preview of the documentation can be found here. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
CI builds currently recompile everything from scratch on every run, even when most source files haven't changed. This PR adds ccache across all CI workflows so that unchanged translation units are served from cache, which should cut build times substantially for most development work.
Cache data is stored in GHCR using ORAS. When a build starts, it looks for a cache in priority order: first the current PR's previous cache, then the PR's target branch, then main. After building, the updated cache is pushed back if the miss rate exceeded 10%. When a PR is closed, its cache artifacts are cleaned up automatically.
The ccache logic is implemented as three reusable composite actions (ccache-setup, ccache-restore, ccache-push) shared across all workflows. For macOS jobs, ccache runs on the host. For Linux jobs, cache data is injected into Docker builds via a BuildKit build-context and extracted afterward for persistence. This should seamlessly fallback to a local cache on the local devcontainer (which conveniently should also implement ccache support where it previously wasn't being used).
ci.yml/macos_ci.yml times (need to run):
Before:
After (hot-cache):
Publishing: (Need to run)
Deployments: (Need to run)
The build cudaq/wheel stages are down to 4-5 mins instead of ~15-20 they were before. Now the big blocker is test times (I've been thinking about some inefficiencies in parallel tests).