Skip to content

Cache apt package downloads in tools/e2e/Containerfile builds #474

Description

@Minipada

Current state

tools/e2e/Containerfile has several apt-get install/rosdep install RUN steps
(toolchain-base, toolchain's two rosdep installs, workspace, runtime) that each end
with rm -rf /var/lib/apt/lists/*. None of them cache the downloaded .deb archives across
builds — --cache-from/--cache-to only cover whole layers, so any RUN whose layer is
invalidated (e.g. workspace's rosdep install, which sits right after COPY . src/ros2_data_collection
and so reruns on almost every source change) re-downloads its full apt package set from
scratch every time.

The dc-ccache mount (RUN --mount=type=cache,id=dc-ccache,target=/root/.ccache, in the
workspace stage) already solves the equivalent problem for compiled object files, using
build.sh's BUILDAH_TMPDIR + an actions/cache step in CI to persist the mount across
runs (buildah cache mounts live under $TMPDIR, entirely separate from
--cache-from/--cache-to).

What to build

  • Add a RUN --mount=type=cache,id=dc-apt,target=/var/cache/apt/archives (same pattern as
    dc-ccache) to the apt-get/rosdep install RUN steps that would benefit most —
    toolchain-base, toolchain, and workspace (the heaviest, most-invalidated ones).
  • Disable apt's default "delete .deb after install" hook so the cache mount actually
    retains files (rm -f /etc/apt/apt.conf.d/docker-clean before the install, or an
    equivalent apt.conf override).
  • Drop rm -rf /var/lib/apt/lists/* in stages that get the cache mount, or otherwise decide
    whether package index caching is also wanted.
  • Extend build.sh's BUILDAH_TMPDIR persistence / CI actions/cache step to cover the new
    dc-apt cache id alongside the existing dc-ccache one.

Acceptance criteria

  • dc-apt (or similar) cache mount added to the relevant Containerfile RUN steps
  • apt's default post-install cleanup is disabled so the mount is actually populated
  • CI (build.sh / ci.yaml) persists the new cache mount the same way it already does
    for dc-ccache
  • A rebuild that only changes DC source (not the apt package set) demonstrably skips
    re-downloading already-cached .deb files
  • Existing daily APT_CACHEBUST behavior (forcing a security-patch refresh once per UTC
    day) is preserved

Blocked by

  • None — can start immediately

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    toolingScripts, CI/CD

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions