Skip to content

Commit 06a7f9b

Browse files
mickemmickem
committed
ci: Cache Linux compiler output with ccache
Nothing was cached between runs, so every push recompiled all ~820 translation units from scratch in each of the three Linux jobs (two package builds plus the sanitizer job) - and a push typically only touches a handful of files. Wire ccache in as a CMake compiler launcher and keep its directory in the GitHub Actions cache. The launcher is only passed when ccache is actually on PATH, so a distro that stops shipping it degrades to today's behaviour instead of failing to configure; on Rocky the package comes from EPEL, which the job already enables. The cache directory sits inside the workspace, which is the one path spelled the same way by actions/cache (which resolves `path:` relative to the workspace) and by the build running inside the container, where ${{ github.workspace }} is the host's view and not the container's. It is restored after checkout, since checkout prunes untracked files. Each job writes an entry keyed by sha and restores by prefix, so a run always starts from the most recent cache for its distro and arch, and a new branch inherits main's. Capped at 500M per job to stay well inside the repository's 10G cache budget. Every job prints `ccache --show-stats` after building, so the hit rate is visible in the log rather than inferred from the timings. Assisted-by: Claude Code:claude-opus-5 Signed-off-by: Michael Medin <michael@medin.name>
1 parent 13991a7 commit 06a7f9b

3 files changed

Lines changed: 111 additions & 4 deletions

File tree

‎.github/workflows/build-debian.yml‎

Lines changed: 45 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -55,10 +55,46 @@ jobs:
5555
done
5656
}
5757
retry apt-get update
58-
retry env DEBIAN_FRONTEND=noninteractive apt-get install -y git curl unzip libboost-all-dev libgtest-dev protobuf-compiler libprotobuf-dev openssl libssl-dev libgmock-dev libcrypto++-dev ${{ matrix.lua_package }} cmake build-essential python3-pip libdbus-1-dev libzip-dev libtinyxml2-dev libmariadb-dev pkg-config
58+
# ccache: see the "Configure ccache" step below. zstd: actions/cache
59+
# compresses with it when present and falls back to a slower gzip when
60+
# it is not.
61+
retry env DEBIAN_FRONTEND=noninteractive apt-get install -y git curl unzip libboost-all-dev libgtest-dev protobuf-compiler libprotobuf-dev openssl libssl-dev libgmock-dev libcrypto++-dev ${{ matrix.lua_package }} cmake build-essential python3-pip libdbus-1-dev libzip-dev libtinyxml2-dev libmariadb-dev pkg-config ccache zstd
5962
6063
- uses: actions/checkout@v6
6164

65+
- name: Configure ccache
66+
shell: bash
67+
run: |
68+
# Every push recompiles all ~820 translation units from scratch, six
69+
# times over (four packages, the sanitizer job, three Windows builds),
70+
# so a compiler cache is the biggest lever left after -j.
71+
#
72+
# The cache lives inside the workspace on purpose: `path:` in the cache
73+
# step is resolved relative to it, and it is the one directory whose
74+
# location is spelled the same way by actions/cache and by the build
75+
# running inside this container. It has to be restored *after* checkout,
76+
# which prunes untracked files from the workspace.
77+
if command -v ccache > /dev/null; then
78+
echo "CCACHE_DIR=$GITHUB_WORKSPACE/.ccache" >> "$GITHUB_ENV"
79+
echo "CCACHE_MAXSIZE=500M" >> "$GITHUB_ENV"
80+
# Read by the CMake step; stays empty when ccache is unavailable so
81+
# the build still configures, just without a launcher.
82+
echo "CCACHE_LAUNCHER=ccache" >> "$GITHUB_ENV"
83+
else
84+
echo "::warning::ccache not available - building without a compiler cache"
85+
fi
86+
87+
- name: Restore ccache
88+
uses: actions/cache@v4
89+
with:
90+
path: .ccache
91+
# The sha in the key makes every run write a fresh entry; restore-keys
92+
# then picks the most recent one for this distro/arch. A branch also
93+
# sees main's caches, so a new branch starts warm.
94+
key: ccache-deb-${{ matrix.distro }}-${{ matrix.distro_version }}-${{ matrix.arch }}-${{ github.sha }}
95+
restore-keys: |
96+
ccache-deb-${{ matrix.distro }}-${{ matrix.distro_version }}-${{ matrix.arch }}-
97+
6298
- name: make dirs
6399
run: |
64100
mkdir -p tmp/nscp
@@ -108,6 +144,7 @@ jobs:
108144
# container; $GITHUB_WORKSPACE is set per-runtime to the
109145
# container-mounted workspace path (`/__w/nscp/nscp`).
110146
cmake ../.. \
147+
${CCACHE_LAUNCHER:+-DCMAKE_C_COMPILER_LAUNCHER=$CCACHE_LAUNCHER -DCMAKE_CXX_COMPILER_LAUNCHER=$CCACHE_LAUNCHER} \
111148
-DBUILD_VERSION=${{ inputs.version }} \
112149
-DCPACK_GENERATOR=DEB \
113150
-DCMAKE_BUILD_TYPE=Release \
@@ -126,8 +163,15 @@ jobs:
126163
# the graph is wide enough to keep every core busy (a full build
127164
# measured 1433% CPU at -j16). A bare `make` left three of the
128165
# runner's four cores idle for 40+ minutes.
166+
if [ -n "${CCACHE_LAUNCHER:-}" ]; then ccache --zero-stats; fi
129167
make -j"$(nproc)"
130168
169+
- name: ccache statistics
170+
if: always()
171+
shell: bash
172+
run: |
173+
if [ -n "${CCACHE_LAUNCHER:-}" ]; then ccache --show-stats; fi
174+
131175
- name: CPack
132176
working-directory: tmp/nscp
133177
run: |

‎.github/workflows/build-redhat.yml‎

Lines changed: 33 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -80,9 +80,33 @@ jobs:
8080
retry dnf update -y
8181
# Node.js / npm are no longer needed here: the web bundle ships
8282
# separately and is installed at runtime via `nscp web install-ui`.
83-
retry dnf install -y --allowerasing coreutils bash file findutils git boost-devel gtest-devel protobuf-compiler protobuf-devel openssl openssl-devel gmock-devel cryptopp-devel lua-devel cmake gcc-c++ python3-pip rpm-build python3-devel dbus-devel bash libzip-devel tinyxml2-devel mariadb-connector-c-devel pkgconf-pkg-config
83+
# ccache comes from EPEL, which is already enabled above. zstd lets
84+
# actions/cache compress with zstd instead of falling back to gzip.
85+
retry dnf install -y --allowerasing coreutils bash file findutils git boost-devel gtest-devel protobuf-compiler protobuf-devel openssl openssl-devel gmock-devel cryptopp-devel lua-devel cmake gcc-c++ python3-pip rpm-build python3-devel dbus-devel bash libzip-devel tinyxml2-devel mariadb-connector-c-devel pkgconf-pkg-config ccache zstd
8486
- uses: actions/checkout@v6
8587

88+
- name: Configure ccache
89+
shell: bash
90+
run: |
91+
# See build-debian.yml for the rationale and for why the cache lives
92+
# inside the workspace. ccache is an EPEL package here, so the guard
93+
# below is what keeps the build working if it ever goes missing.
94+
if command -v ccache > /dev/null; then
95+
echo "CCACHE_DIR=$GITHUB_WORKSPACE/.ccache" >> "$GITHUB_ENV"
96+
echo "CCACHE_MAXSIZE=500M" >> "$GITHUB_ENV"
97+
echo "CCACHE_LAUNCHER=ccache" >> "$GITHUB_ENV"
98+
else
99+
echo "::warning::ccache not available - building without a compiler cache"
100+
fi
101+
102+
- name: Restore ccache
103+
uses: actions/cache@v4
104+
with:
105+
path: .ccache
106+
key: ccache-rpm-${{ matrix.distro }}-${{ matrix.distro_version }}-${{ matrix.arch }}-${{ github.sha }}
107+
restore-keys: |
108+
ccache-rpm-${{ matrix.distro }}-${{ matrix.distro_version }}-${{ matrix.arch }}-
109+
86110
- name: make dirs
87111
run: |
88112
mkdir -p tmp/nscp
@@ -122,6 +146,7 @@ jobs:
122146
# resolve to the container-mounted workspace, not the runner
123147
# host's view. See build-debian.yml for the full rationale.
124148
cmake ../.. \
149+
${CCACHE_LAUNCHER:+-DCMAKE_C_COMPILER_LAUNCHER=$CCACHE_LAUNCHER -DCMAKE_CXX_COMPILER_LAUNCHER=$CCACHE_LAUNCHER} \
125150
-DBUILD_VERSION=${{ inputs.version }} \
126151
-DCPACK_GENERATOR=RPM \
127152
-DCMAKE_BUILD_TYPE=Release \
@@ -137,8 +162,15 @@ jobs:
137162
run: |
138163
# See build-debian.yml: this step is ~97% of the job and the module
139164
# graph is wide, so a serial `make` wasted three of the four cores.
165+
if [ -n "${CCACHE_LAUNCHER:-}" ]; then ccache --zero-stats; fi
140166
make -j"$(nproc)"
141167
168+
- name: ccache statistics
169+
if: always()
170+
shell: bash
171+
run: |
172+
if [ -n "${CCACHE_LAUNCHER:-}" ]; then ccache --show-stats; fi
173+
142174
- name: CPack
143175
working-directory: tmp/nscp
144176
run: |

‎.github/workflows/tests-sanitizers.yml‎

Lines changed: 33 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -44,10 +44,32 @@ jobs:
4444
protobuf-compiler libprotobuf-dev \
4545
openssl libssl-dev libcrypto++-dev \
4646
liblua5.4-dev libdbus-1-dev \
47-
libzip-dev libtinyxml2-dev libmariadb-dev
47+
libzip-dev libtinyxml2-dev libmariadb-dev \
48+
ccache zstd
4849
4950
- uses: actions/checkout@v6
5051

52+
- name: Configure ccache
53+
shell: bash
54+
run: |
55+
# See build-debian.yml. This job is the slowest of the Linux three
56+
# (sanitizer instrumentation plus -g), so it has the most to gain.
57+
if command -v ccache > /dev/null; then
58+
echo "CCACHE_DIR=$GITHUB_WORKSPACE/.ccache" >> "$GITHUB_ENV"
59+
echo "CCACHE_MAXSIZE=500M" >> "$GITHUB_ENV"
60+
echo "CCACHE_LAUNCHER=ccache" >> "$GITHUB_ENV"
61+
else
62+
echo "::warning::ccache not available - building without a compiler cache"
63+
fi
64+
65+
- name: Restore ccache
66+
uses: actions/cache@v4
67+
with:
68+
path: .ccache
69+
key: ccache-asan-ubuntu-24.04-x64-${{ github.sha }}
70+
restore-keys: |
71+
ccache-asan-ubuntu-24.04-x64-
72+
5173
- uses: actions/setup-node@v6
5274
with:
5375
node-version: 20
@@ -74,6 +96,7 @@ jobs:
7496
# binary — sanitizer tests don't need it, and building the Rust
7597
# client just for this run is wasteful.
7698
cmake ../.. \
99+
${CCACHE_LAUNCHER:+-DCMAKE_C_COMPILER_LAUNCHER=$CCACHE_LAUNCHER -DCMAKE_CXX_COMPILER_LAUNCHER=$CCACHE_LAUNCHER} \
77100
-DBUILD_VERSION=${{ inputs.version }} \
78101
-DCMAKE_BUILD_TYPE=RelWithDebInfo \
79102
-DNSCP_SANITIZE=address,undefined \
@@ -90,7 +113,15 @@ jobs:
90113
# its arena on exit. Disable leak-check during the build phase only;
91114
# we re-enable it for the test phase below.
92115
ASAN_OPTIONS: detect_leaks=0
93-
run: cmake --build . -j$(nproc)
116+
run: |
117+
if [ -n "${CCACHE_LAUNCHER:-}" ]; then ccache --zero-stats; fi
118+
cmake --build . -j$(nproc)
119+
120+
- name: ccache statistics
121+
if: always()
122+
shell: bash
123+
run: |
124+
if [ -n "${CCACHE_LAUNCHER:-}" ]; then ccache --show-stats; fi
94125
95126
- name: Run unit tests under sanitizers
96127
working-directory: tmp/nscp

0 commit comments

Comments
 (0)