refactor(cmake): reorganize dependency management into per-library files#2091
Open
refactor(cmake): reorganize dependency management into per-library files#2091
Conversation
- Introduce DependenciesVersions.cmake to centralize all versions, URLs and hashes - Split monolithic Dependencies.cmake into individual files under src/cmake/deps/ - Add helper macros in Helpers.cmake to reduce boilerplate - Ensure all libraries install into the 'lib' folder - Fix pkgconfig paths for Eigen and OpenMesh
predat
force-pushed
the
dev/refactor-build-dependencies
branch
from
6170417 to
20920d4
Compare
added 5 commits
March 20, 2026 12:19
The dependency Docker image build was broken: `cmake -P` was called on
`external/src/tiff-stamp/download-tiff.cmake` which never existed
because
`tiff` is a git-based dep. This error was a symptom of a deeper
inconsistency in `av_add_cmake_dep` where `SOURCE_DIR` was set outside
the ExternalProject `PREFIX`, causing CMake to derive `STAMP_DIR` and
`TMP_DIR` to paths it could not locate.
This PR fixes the root cause and modernizes the surrounding
infrastructure.
`SOURCE_DIR` was set to `${CMAKE_CURRENT_BINARY_DIR}/<name>` which is
outside `PREFIX` (`BUILD_DIR/external`). CMake derives `STAMP_DIR` and
`TMP_DIR` from `PREFIX`, so step scripts (`download-*.cmake`,
`gitclone-*.cmake`) were written to paths CMake could not find at
configure time.
All ExternalProject directories are now explicitly set under
`BUILD_DIR`:
- `SOURCE_DIR` → `${BUILD_DIR}/src/<name>`
- `BINARY_DIR` → `${BUILD_DIR}/<name>_build`
- `STAMP_DIR` → `${BUILD_DIR}/src/<name>-stamp`
- `TMP_DIR` → `${BUILD_DIR}/src/<name>-tmp`
`GIT_SHALLOW TRUE` added for git-based deps: faster clones, less
network.
Internal `_D_RESOLVED_*` variables are `unset()` after use to avoid
macro scope leakage.
- Replace 15 hardcoded `cmake -P` calls with a loop over URL-based deps
only. Git-based deps (`tiff`, `ceres`, ...) generate
`gitclone-*.cmake`
not `download-*.cmake` — calling `cmake -P` on them was the visible
symptom of the stamp dir bug.
- Remove cmake tarball download: the image now uses the official CMake
binary installer, no source build needed.
- Replace `test && ...` with `if/fi` for the `ColorChartDetectionModel`
clone — clearer intent, avoids the silent failure on `rm README.md`.
- Quote all variables and paths consistently.
- Add `# syntax=docker/dockerfile:1.4` to enable BuildKit features.
- Add `--mount=type=cache` on `dnf` and `pip` `RUN` steps: RPMs and
Python wheels survive layer invalidation, significantly speeding up
iterative rebuilds.
- Replace cmake source build (`bootstrap + make -j16`, ~15 min) with the
official binary installer: `cmake-${CMAKE_VERSION}-linux-x86_64.sh`.
`CMAKE_VERSION` is now an `ARG` (default `3.31.11`), overridable with
`--build-arg CMAKE_VERSION=x.y.z`.
- Re-declare `ARG CMAKE_VERSION` after `FROM` (Docker ARG scoping rule).
- Merge configure + build + cleanup into a single `RUN`: the ~10 GB of
build artefacts in `AV_BUILD` are removed in the same layer and never
committed to the final image.
- Merge 4× `COPY` + 4× `RUN echo` into 4× `COPY` + 1× `RUN` block for
env var exports.
- Replace `update-alternatives --force` (unsupported on Rocky 9
`alternatives` 1.20) with `rm -f` before `alternatives --install`.
- Replace backtick substitution `` ` `` with `$(...)` throughout.
- Sort `dnf` package list alphabetically, one package per line.
- Order `COPY` instructions from least to most frequently changed to
maximize Docker layer cache reuse.
- Enable `DOCKER_BUILDKIT=1` in `build-rocky.sh`.
- Verified full `docker/build-rocky.sh` run on Rocky Linux 9 /
CUDA 12.1.1 / GCC 12.4
- All URL-based deps pre-fetched successfully via `docker/fetch.sh`
- Git-based deps (tiff, ceres, lemon, ...) cloned correctly at build
time
…IES_PARALLEL=0 Docker build contexts have limited parallelism headroom; building deps sequentially avoids OOM kills and I/O contention on the builder host. cmake --build is updated accordingly (--parallel 1).
…oring - Add # syntax=docker/dockerfile:1.4 and BuildKit cache mounts on apt (/var/cache/apt + /var/lib/apt) and pip (/root/.cache/pip) - Merge all apt-get install + PPA setup into a single RUN layer - Merge 4x COPY+RUN echo into 4x COPY + 1x RUN block for env vars - Merge configure + build + cleanup into a single RUN to avoid committing ~10 GB of build artefacts into intermediate layers - Sort package list alphabetically, one package per line - Order COPY from least to most frequently changed for cache reuse - Replace backtick substitution with $(...) throughout - Re-declare ARG CUDA_VERSION/UBUNTU_VERSION after FROM - Add build example and section comments
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This PR reorganizes the entire dependency management system of AliceVision's
CMake superbuild to improve maintainability, readability, and long-term
scalability.
Changes
CMake / Build system
DependenciesVersions.cmake(new): single source of truth for all dependencyversions, download URLs and hashes.
src/cmake/deps/(new directory): each third-party library now has itsown dedicated
.cmakefile (alembic, boost, ceres, cuda, eigen, opencv,openexr, openimageio, onnxruntime, suitesparse, tbb, …).
Helpers.cmake: new macros to avoid code duplication across dependencyfiles.
lib/folder.Dependencies
Motivation
The previous
Dependencies.cmakewas a single large file mixing versions,URLs, hashes and build logic for all dependencies, making it hard to review,
update or extend. The new per-library structure makes each dependency
self-contained and independently versionable.