Skip to content

Commit c685293

Browse files
dgarskedgarske
authored
Merge pull request #10685 from julek-wolfssl/ci-cache-offload
CI: offload ccache/apt/buildx caches off the GitHub Actions cache
2 parents 70883a4 + 634ac9b commit c685293

58 files changed

Lines changed: 757 additions & 26 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/actions/ccache-setup/action.yml‎

Lines changed: 40 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,9 @@ description: >
44
directory from a previous run, and prepend the ccache compiler-symlink
55
dir to PATH. Subsequent gcc/cc/g++/c++/clang invocations are
66
transparently intercepted by ccache, so no other workflow step needs to
7-
change.
7+
change. On scheduled (cron) runs the cache is reseeded from clean
8+
compiles (CCACHE_RECACHE) instead of only being updated incrementally,
9+
so it can't drift indefinitely.
810
911
inputs:
1012
workflow-id:
@@ -21,6 +23,15 @@ inputs:
2123
description: 'Per-job ccache max size (passed to ccache -M).'
2224
required: false
2325
default: '500M'
26+
read-only:
27+
description: >
28+
When 'true', restore the cache but do NOT save it (no post-job
29+
upload). Callers should set this to the result of the expression
30+
github.event_name == 'pull_request' so PR runs consume the shared
31+
cache read-only - no per-PR entries, no churn - while scheduled/push
32+
runs (read-only false) refresh it.
33+
required: false
34+
default: 'false'
2435

2536
runs:
2637
using: 'composite'
@@ -41,7 +52,10 @@ runs:
4152
exit 1
4253
fi
4354
55+
# read-only=false (default): restore + post-job save (the run_id in the
56+
# key never hits, so it always saves its contribution).
4457
- name: Restore + save ccache
58+
if: inputs.read-only != 'true'
4559
uses: actions/cache@v5
4660
with:
4761
path: ~/.ccache
@@ -52,6 +66,19 @@ runs:
5266
restore-keys: |
5367
ccache-${{ inputs.workflow-id }}-${{ runner.os }}-${{ runner.arch }}-${{ inputs.config-hash }}-
5468
ccache-${{ inputs.workflow-id }}-${{ runner.os }}-${{ runner.arch }}-
69+
# read-only=true: restore the shared cache but never upload (PR runs).
70+
- name: Restore ccache (read-only)
71+
if: inputs.read-only == 'true'
72+
uses: actions/cache/restore@v5
73+
with:
74+
path: ~/.ccache
75+
# Same key shape as the save branch, for symmetry. This branch never
76+
# saves, so the run_id/run_attempt primary key is never an exact hit -
77+
# the restore-keys below always supply the most recent seeded cache.
78+
key: ccache-${{ inputs.workflow-id }}-${{ runner.os }}-${{ runner.arch }}-${{ inputs.config-hash }}-${{ github.run_id }}-${{ github.run_attempt }}
79+
restore-keys: |
80+
ccache-${{ inputs.workflow-id }}-${{ runner.os }}-${{ runner.arch }}-${{ inputs.config-hash }}-
81+
ccache-${{ inputs.workflow-id }}-${{ runner.os }}-${{ runner.arch }}-
5582
5683
- name: Configure ccache and PATH
5784
shell: bash
@@ -76,6 +103,18 @@ runs:
76103
echo "$CCACHE_LIBEXEC" >> "$GITHUB_PATH"
77104
echo "CCACHE_DIR=$HOME/.ccache" >> "$GITHUB_ENV"
78105
106+
# On the scheduled (cron) refresh, force every compile to miss the
107+
# cache and re-store a fresh result (CCACHE_RECACHE still writes, it
108+
# just skips lookups). This reseeds the shared cache from clean
109+
# compiles instead of only layering deltas onto whatever accumulated,
110+
# so a bad/stale entry can't live forever. The cache is still saved
111+
# (read-only is false on schedule), and PR/push runs are unaffected -
112+
# they keep their warm hits. Cost: the scheduled jobs recompile fully.
113+
- name: Force fresh compiles on scheduled reseed
114+
if: github.event_name == 'schedule'
115+
shell: bash
116+
run: echo "CCACHE_RECACHE=1" >> "$GITHUB_ENV"
117+
79118
- name: Show ccache stats (initial)
80119
shell: bash
81120
run: ccache -s

‎.github/actions/install-apt-deps/action.yml‎

Lines changed: 63 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -20,11 +20,65 @@ inputs:
2020
description: 'Cache apt archives (disable for dynamic package names)'
2121
required: false
2222
default: 'true'
23+
ghcr-debs-tag:
24+
description: >
25+
Tag of a prebuilt .deb bundle published to
26+
ghcr.io/<owner>/wolfssl-ci-debs by the ci-deps-image workflow
27+
(e.g. "ubuntu-24.04-minimal"). When set, the packages are installed
28+
offline from that bundle and the apt cache path below is skipped; on
29+
that happy path the apt mirror is not contacted. The offline install
30+
is all-or-nothing (a single --no-download install of the whole set),
31+
so any failure - bundle missing, not public, or not covering every
32+
requested package - falls back to the apt path. Always safe to set;
33+
leave empty to use apt only.
34+
required: false
35+
default: ''
2336
runs:
2437
using: 'composite'
2538
steps:
39+
# Preferred path: install from a prebuilt .deb bundle pulled from ghcr,
40+
# entirely offline (--no-download), so a flaky/timing-out apt mirror
41+
# cannot break the build. Best-effort: on any failure we leave
42+
# "satisfied" unset and the apt steps below run unchanged. The bundle
43+
# image must be PUBLIC so anonymous `docker pull` works (including from
44+
# fork PRs whose GITHUB_TOKEN cannot read private packages).
45+
- name: Install from ghcr .deb bundle (offline)
46+
id: ghcr
47+
if: inputs.ghcr-debs-tag != ''
48+
shell: bash
49+
run: |
50+
set -u
51+
command -v docker >/dev/null 2>&1 || { echo "::notice::docker unavailable; using apt"; exit 0; }
52+
# Hardcode the upstream owner: the bundle is only ever published under
53+
# ghcr.io/wolfssl by ci-deps-image (gated to the wolfssl org), so fork
54+
# PRs read the public upstream image too rather than a nonexistent
55+
# ghcr.io/<fork>/wolfssl-ci-debs.
56+
IMG="ghcr.io/wolfssl/wolfssl-ci-debs:${{ inputs.ghcr-debs-tag }}"
57+
if ! docker pull -q "$IMG" >/dev/null 2>&1; then
58+
echo "::notice::ghcr bundle $IMG unavailable; using apt"
59+
exit 0
60+
fi
61+
cid=$(docker create "$IMG" 2>/dev/null) || { echo "::notice::cannot open bundle; using apt"; exit 0; }
62+
rm -rf "$RUNNER_TEMP/ghcr-debs"; mkdir -p "$RUNNER_TEMP/ghcr-debs"
63+
docker cp "$cid:/debs/." "$RUNNER_TEMP/ghcr-debs/" >/dev/null 2>&1 || true
64+
docker rm "$cid" >/dev/null 2>&1 || true
65+
ls "$RUNNER_TEMP"/ghcr-debs/*.deb >/dev/null 2>&1 || { echo "::notice::bundle had no .debs; using apt"; exit 0; }
66+
sudo cp "$RUNNER_TEMP"/ghcr-debs/*.deb /var/cache/apt/archives/
67+
NO_REC=""
68+
if [ "${{ inputs.no-install-recommends }}" = "true" ]; then
69+
NO_REC="--no-install-recommends"
70+
fi
71+
# --no-download forbids any network fetch: if the bundle is missing
72+
# a package this fails cleanly (nothing installed) and we fall back.
73+
if sudo DEBIAN_FRONTEND=noninteractive apt-get install -y $NO_REC --no-download ${{ inputs.packages }}; then
74+
echo "satisfied=true" >> "$GITHUB_OUTPUT"
75+
echo "Installed offline from $IMG: ${{ inputs.packages }}"
76+
else
77+
echo "::notice::offline install incomplete for $IMG; using apt"
78+
fi
79+
2680
- name: Compute cache key
27-
if: inputs.cache == 'true'
81+
if: inputs.cache == 'true' && steps.ghcr.outputs.satisfied != 'true'
2882
id: cache-key
2983
shell: bash
3084
run: |
@@ -35,7 +89,7 @@ runs:
3589
echo "restore-key=apt-deps-${{ runner.os }}-${{ runner.arch }}-${OS_VERSION}-" >> $GITHUB_OUTPUT
3690
3791
- name: Restore apt cache
38-
if: inputs.cache == 'true'
92+
if: inputs.cache == 'true' && steps.ghcr.outputs.satisfied != 'true'
3993
id: apt-cache
4094
uses: actions/cache/restore@v5
4195
with:
@@ -44,7 +98,7 @@ runs:
4498
restore-keys: ${{ steps.cache-key.outputs.restore-key }}
4599

46100
- name: Pre-seed apt archives from cache
47-
if: inputs.cache == 'true' && steps.apt-cache.outputs.cache-hit == 'true'
101+
if: inputs.cache == 'true' && steps.apt-cache.outputs.cache-hit == 'true' && steps.ghcr.outputs.satisfied != 'true'
48102
shell: bash
49103
run: |
50104
if [ -d ~/apt-cache ] && ls ~/apt-cache/*.deb >/dev/null 2>&1; then
@@ -53,6 +107,7 @@ runs:
53107
fi
54108
55109
- name: Install packages
110+
if: steps.ghcr.outputs.satisfied != 'true'
56111
shell: bash
57112
env:
58113
APT_CACHE_HIT: ${{ steps.apt-cache.outputs.cache-hit }}
@@ -90,16 +145,19 @@ runs:
90145
DELAY=$((DELAY * 2))
91146
done
92147
148+
# PR runs never write the apt cache (no churn); only push/schedule runs
149+
# refresh it. The make-check family does not need it anyway - it installs
150+
# from the ghcr bundle above.
93151
- name: Collect .deb files for cache
94-
if: inputs.cache == 'true' && steps.apt-cache.outputs.cache-hit != 'true'
152+
if: inputs.cache == 'true' && github.event_name != 'pull_request' && steps.apt-cache.outputs.cache-hit != 'true' && steps.ghcr.outputs.satisfied != 'true'
95153
shell: bash
96154
run: |
97155
mkdir -p ~/apt-cache
98156
cp /var/cache/apt/archives/*.deb ~/apt-cache/ 2>/dev/null || true
99157
echo "Cached $(ls ~/apt-cache/*.deb 2>/dev/null | wc -l) .deb files"
100158
101159
- name: Save apt cache
102-
if: inputs.cache == 'true' && steps.apt-cache.outputs.cache-hit != 'true'
160+
if: inputs.cache == 'true' && github.event_name != 'pull_request' && steps.apt-cache.outputs.cache-hit != 'true' && steps.ghcr.outputs.satisfied != 'true'
103161
uses: actions/cache/save@v5
104162
with:
105163
path: ~/apt-cache

‎.github/ci-deps/packages-ubuntu-22.04-full.txt‎

Lines changed: 80 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,80 @@
1+
# make-check family + interop apt packages for ubuntu-22.04 (the '-full'
2+
# bundle: ghcr.io/<owner>/wolfssl-ci-debs:ubuntu-22.04-full). Superset of
3+
# -minimal; interop workflows install their subset offline from it.
4+
# Keep sorted; add a package when an interop workflow adds one.
5+
autoconf
6+
automake
7+
bison
8+
bridge-utils
9+
build-essential
10+
ca-certificates
11+
cargo
12+
ccache
13+
chrpath
14+
cmake
15+
cpio
16+
crossbuild-essential-arm64
17+
crossbuild-essential-armel
18+
crossbuild-essential-armhf
19+
crossbuild-essential-riscv64
20+
device-tree-compiler
21+
dfu-util
22+
diffstat
23+
dos2unix
24+
doxygen
25+
file
26+
flex
27+
g++
28+
g++-multilib
29+
gawk
30+
gcc
31+
gcc-multilib
32+
gcovr
33+
git
34+
git-core
35+
gnupg
36+
gperf
37+
gtk-sharp3
38+
help2man
39+
iproute2
40+
lcov
41+
libcairo2-dev
42+
libglib2.0-dev
43+
libgtk2.0-0
44+
liblocale-gettext-perl
45+
libmagic1
46+
libncurses5-dev
47+
libpcap-dev
48+
libpopt0
49+
libsdl1.2-dev
50+
libsdl2-dev
51+
libssl-dev
52+
libtool
53+
libtool-bin
54+
locales
55+
make
56+
net-tools
57+
ninja-build
58+
openssh-client
59+
ovmf
60+
parallel
61+
pkg-config
62+
python-is-python3
63+
python3-dev
64+
python3-pip
65+
python3-ply
66+
python3-setuptools
67+
python3-tk
68+
python3-wheel
69+
qemu-kvm
70+
qemu-user
71+
rsync
72+
socat
73+
srecord
74+
sudo
75+
texinfo
76+
uml-utilities
77+
unzip
78+
wget
79+
xz-utils
80+
zip

‎.github/ci-deps/packages-ubuntu-22.04-minimal.txt‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
# make-check family apt packages for ubuntu-22.04 (the '-minimal'
2+
# bundle: ghcr.io/<owner>/wolfssl-ci-debs:ubuntu-22.04-minimal). UNION of
3+
# every family workflow's list; superset is fine. Keep sorted.
4+
autoconf
5+
automake
6+
build-essential
7+
crossbuild-essential-arm64
8+
crossbuild-essential-armel
9+
crossbuild-essential-armhf
10+
crossbuild-essential-riscv64
11+
libtool
12+
qemu-user

‎.github/ci-deps/packages-ubuntu-24.04-full.txt‎

Lines changed: 94 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,94 @@
1+
# make-check family + interop apt packages for ubuntu-24.04 (the '-full'
2+
# bundle: ghcr.io/<owner>/wolfssl-ci-debs:ubuntu-24.04-full). Superset of
3+
# -minimal; interop workflows install their subset offline from it.
4+
# Keep sorted; add a package when an interop workflow adds one.
5+
apache2
6+
apache2-dev
7+
autoconf
8+
autoconf-archive
9+
automake
10+
autopoint
11+
bubblewrap
12+
build-essential
13+
ccache
14+
clang
15+
clang-14
16+
clang-19
17+
cmake
18+
g++-10
19+
g++-11
20+
g++-12
21+
g++-9
22+
gcc-10
23+
gcc-11
24+
gcc-12
25+
gcc-9
26+
gcc-multilib
27+
gettext
28+
gyp
29+
jq
30+
krb5-admin-server
31+
krb5-kdc
32+
krb5-otp
33+
libbz2-dev
34+
libc++-dev
35+
libcap-dev
36+
libcap-ng-dev
37+
libcmocka-dev
38+
libcppunit-dev
39+
libcunit1
40+
libcunit1-dev
41+
libcunit1-doc
42+
libcurl4-openssl-dev
43+
libdb5.3-dev
44+
libev-dev
45+
libevent-2.1-7
46+
libevent-dev
47+
libffi-dev
48+
libgdbm-dev
49+
libgtest-dev
50+
libidn2-dev
51+
libio-socket-ssl-perl
52+
libjansson-dev
53+
libkrb5-dev
54+
liblz4-dev
55+
liblzma-dev
56+
liblzo2-dev
57+
libncursesw5-dev
58+
libnghttp2-dev
59+
libnl-genl-3-200
60+
libnl-genl-3-dev
61+
libnss-wrapper
62+
libnss3-dev
63+
libp11-dev
64+
libpam-dev
65+
libpam0g-dev
66+
libpcre2-dev
67+
libpsl-dev
68+
libpsl5
69+
libreadline-dev
70+
librtlsdr-dev
71+
libsecret-1-dev
72+
libsocket-wrapper
73+
libsqlite3-dev
74+
libssl-dev
75+
libtool
76+
liburcu-dev
77+
libuv1-dev
78+
linux-libc-dev
79+
make
80+
man2html
81+
meson
82+
mono-complete
83+
nghttp2
84+
ninja-build
85+
pkg-config
86+
pkgconf
87+
psmisc
88+
python3-docutils
89+
python3-impacket
90+
python3-psutil
91+
shellcheck
92+
uuid-dev
93+
valgrind
94+
zlib1g-dev

0 commit comments

Comments
 (0)