ci: Speed up the Linux builds (parallel make, ccache, docs out of ALL) - #1408
Merged
Conversation
The `Build nsclient` step is 96-97% of every Linux package job - 43 minutes on x64, 64 on arm64 - and it ran a bare `make`, so three of the four cores on a hosted runner sat idle for the whole build. The dependency graph is wide (70-odd independent modules): a full build measured 6600 CPU-seconds over 819 translation units and sustained 1433% CPU at -j16, so it scales near-linearly with the cores available. The sanitizer job already builds this tree with `-j$(nproc)` in the same ubuntu:24.04 container, so parallel builds are known to work here. Assisted-by: Claude Code:claude-opus-5 Signed-off-by: Michael Medin <michael@medin.name>
The mkdocs site was in `ALL`, so every build on every platform rendered it -
but only the Windows installer ships it (the install() is guarded by WIN32).
The Linux package and sanitizer jobs built the site and threw it away.
Worse, being in `ALL` made mkdocs a load-bearing part of the build: a docs
failure aborted the whole thing before the binaries were done, which is not
what you want from a step whose output is discarded.
Add NSCP_BUILD_DOCS_HTML, defaulting to ON on Windows (where the installer
needs the site pre-built - it is built with /p:BuildProjectReferences=false,
so it cannot pull the docs target in as a dependency) and OFF elsewhere. The
target still exists everywhere:
cmake --build . --target build_docs_html
The Windows install() is guarded by the same option, since with the site not
built there is no directory to install and install(DIRECTORY) on a missing
source is a hard error at package time.
Assisted-by: Claude Code:claude-opus-5
Signed-off-by: Michael Medin <michael@medin.name>
Nothing was cached between runs, so every push recompiled all ~820
translation units from scratch in each of the three Linux jobs (two package
builds plus the sanitizer job) - and a push typically only touches a handful
of files.
Wire ccache in as a CMake compiler launcher and keep its directory in the
GitHub Actions cache. The launcher is only passed when ccache is actually on
PATH, so a distro that stops shipping it degrades to today's behaviour
instead of failing to configure; on Rocky the package comes from EPEL, which
the job already enables.
The cache directory sits inside the workspace, which is the one path spelled
the same way by actions/cache (which resolves `path:` relative to the
workspace) and by the build running inside the container, where
${{ github.workspace }} is the host's view and not the container's. It is
restored after checkout, since checkout prunes untracked files.
Each job writes an entry keyed by sha and restores by prefix, so a run always
starts from the most recent cache for its distro and arch, and a new branch
inherits main's. Capped at 500M per job to stay well inside the repository's
10G cache budget. Every job prints `ccache --show-stats` after building, so
the hit rate is visible in the log rather than inferred from the timings.
Assisted-by: Claude Code:claude-opus-5
Signed-off-by: Michael Medin <michael@medin.name>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Three independent changes, one per commit, all aimed at the one step that dominates every build job.
Where the time goes today
Measured on the last release build of
main(run 31686095180, commit 7acb180): 2 h 21 m wall, 10.2 runner-hours over 21 jobs, plus 5.0 h of jobs sitting in the queue waiting for a runner.Build nsclientis 96-97 % of every Linux job:Build nsclientDependency install, packaging, artifact upload and ctest are all noise by comparison.
1.
make -j$(nproc)(4d1e256b)Both Linux package workflows ran a bare
make, leaving three of the four cores on a hosted runner idle for the whole build.A full local build is 6 600 CPU-seconds over 819 translation units and sustained 1433 % CPU at
-j16- the module graph is wide enough (70-odd independent modules) to scale near-linearly.tests-sanitizers.ymlalready builds this tree with-j$(nproc)in the sameubuntu:24.04container, so parallel builds are known-good here.2. HTML docs out of
ALLoff Windows (13991a7d)build_docs_htmlwas inALLon every platform, but only the Windows installer ships the site (install()is guarded byWIN32). Linux built it and threw it away - and because it was inALL, a mkdocs failure aborted the whole build before the binaries were finished. Hit exactly that while benchmarking.New
NSCP_BUILD_DOCS_HTMLoption:ONon Windows (the installer is built with/p:BuildProjectReferences=false, so it cannot pull the docs in as a dependency),OFFelsewhere. The target still exists everywhere -cmake --build . --target build_docs_html.3. ccache on the Linux jobs (
06a7f9b9)Nothing was cached between runs. Wired in as a CMake compiler launcher with the ccache dir in
actions/cache, only when ccache is actually onPATHso a distro dropping the package degrades to today's behaviour rather than failing to configure. Every job printsccache --show-statsafter building, so the hit rate is in the log.This PR's own run is a cold cache - it shows the
-jwin only. The second push to this branch is where ccache shows up.Not in this PR
NSCP_DEF_PLUGIN_CPPand friends) are pasted into every module'sSRCS, so 36 % of all compile CPU is recompiling a file already compiled in another target - 521 of 819 TUs, 2 644 CPU-seconds. The seven worst (nscapi/settings/helper.cpp43x,net/socket/socket_helpers.cpp26x,nscapi_program_options.cpp42x, ...) are 19 % on their own. Fixing that means one object library and it pays off on Windows too.msbuildhas no/m(mitigated by/MPbeing set globally).check_tcp.cpp64 s,CheckNSCP.cpp62 s,check_http.cpp52 s - candidates for a shared PCH.build-feature.ymlruns this same 21-job matrix on every PR push (~10 runner-hours/push); gating arm64 and the legacy x86-static build to tags would halve it, but that is a support-policy call.🤖 Generated with Claude Code