Skip to content

CI: cache Superbuild externals build output (actions/cache) #2520

Description

@dcwhite

Tracked under #1487 ("External dependency caching"); this is the detailed plan for the caching item in #2028.

Summary

SCIRun's CI rebuilds the Superbuild externals (the heavy third-party C++ dependencies) from scratch on every run — none of the nine workflows (ccpp.yml, mac.yml, windows.yml, reusable-build.yml, etc.) use any caching. We can cut a large fraction of CI time by caching the externals build output, keyed on the files that define how those externals are built.

This mirrors how our sibling project SCIInstitute/shapeworks handles it: its biggest CI win is a GitHub-native actions/cache of the compiled dependencies (ITK, VTK, OpenVDB, …), keyed on the dependency recipe rather than the project source, so the multi-hour external build is reused across every PR and commit.

The pattern (from ShapeWorks)

Split restore / save, with the build and the save both gated on a cache miss:

- name: Restore Externals Cache
  id: cache-externals
  uses: actions/cache/restore@v4
  with:
    path: <superbuild externals install/build dir>
    key: ${{ runner.os }}-externals-${{ hashFiles(
      'Superbuild/CMakeLists.txt',
      'Superbuild/**/*.cmake') }}

- name: Build Externals
  if: steps.cache-externals.outputs.cache-hit != 'true'
  run: <superbuild configure+build>

- name: Save Externals Cache
  if: steps.cache-externals.outputs.cache-hit != 'true'
  uses: actions/cache/save@v4
  with:
    path: <same path>
    key: <same key>

Why it works:

  • Key hashes the externals recipe, not the source. Keyed on the Superbuild CMake files (+ toolchain/compiler), not on SCIRun's own .cpp/.h. The cache only invalidates when the externals definition changes, so it survives across normal PRs.
  • Split restore + save (not the combined action) so we can skip both the build and the re-save on a hit.
  • Per-variant keys to avoid collisions: include runner.os, build type (Debug/Release), and on macOS the deployment target / arch.

Scope (this issue = "Step 1" only)

  • Identify the exact Superbuild externals output path(s) to cache, per platform
  • Decide what goes into the cache key (Superbuild CMake files + toolchain + build type + arch)
  • Wire actions/cache/restore + gated build + actions/cache/save into reusable-build.yml (and any workflow that builds externals directly)
  • Verify a cold run populates the cache and a warm run restores it and skips the externals build
  • Confirm cache size stays within GitHub's per-repo limits

Out of scope (separate follow-up): adding ccache/sccache as a compiler launcher for SCIRun's own sources.

Context

Faster CI directly helps unblock the in-progress C++ modernization work, which is currently gated on new CI testing jobs.

Reference: shapeworks/.github/workflowsbuild-linux.yml, gha_deps.sh, common.sh.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions