Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 38 additions & 1 deletion .github/workflows/debian.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ on:
push:
paths:
- .github/workflows/debian.yml
- docker/debian/Dockerfile
- docker/debian/Dockerfile*
- test
workflow_dispatch:

Expand Down Expand Up @@ -92,6 +92,42 @@ jobs:
echo "CONTAINER_IMAGE=${CONTAINER_REGISTRY}/${CONTAINER_REPO}/debian-${{ matrix.os.release }}" >> $GITHUB_ENV
PLATFORM=${{ matrix.architecture.platform }}
echo "PLATFORM_PAIR=${PLATFORM//\//-}" >> $GITHUB_ENV
if [ "${{ matrix.os.release == 'bullseye' }}" == "true" ]; then
echo "CONTAINER_REPOSITORY_GCC=${CONTAINER_REPO}/debian-gcc" >> $GITHUB_ENV
echo "CONTAINER_IMAGE_GCC=${CONTAINER_REGISTRY}/${CONTAINER_REPO}/debian-gcc" >> $GITHUB_ENV
fi
- name: Prepare gcc image metadata
if: ${{ env.CONTAINER_IMAGE_GCC }}
id: meta-gcc
uses: docker/metadata-action@v5
env:
DOCKER_METADATA_ANNOTATIONS_LEVELS: manifest,manifest-descriptor
with:
images: ${{ env.CONTAINER_IMAGE_GCC }}
labels: |
org.opencontainers.image.authors=For inquiries, please use https://${{ github.repository }}/issues
org.opencontainers.image.documentation=https://${{ github.repository }}
org.opencontainers.image.vendor=XRPLF
org.opencontainers.image.title=${{ env.CONTAINER_REPOSITORY_GCC }}
- name: Build gcc image
# Note, we always push this image, otherwise steps.build will fail.
# Also, because we always push it, we intentionally do not tag it.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this going to pollute the CI registry with tons of untagged images? Let's think afterwards about how to optimize this.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do not know how ghcr.io treats untagged images, I hope they are "garbage collected", also it would only happen when this workflow runs which (after the period of high initial activity) won't be hopefully that much.

However I agree this needs to be addressed, which is why I put a TODO in the next line.

I am thinking of building and pushing these images in a separate workflow, and this one would simply use tagged image e.g. ghcr.io/xrplf/ci/debian-gcc:12-bullseye. These won't be updated often (in fact, once or twice per months seem realistic) so we do not need to remember their digests etc. because, with this rare rebuilds, network race conditions won't bother us.

# TODO: Move to separate workflow or job
if: ${{ env.CONTAINER_IMAGE_GCC }}
id: build-gcc
uses: docker/build-push-action@v6
with:
build-args: |
BUILDKIT_DOCKERFILE_CHECK=skip=InvalidDefaultArgInFrom
BUILDKIT_INLINE_CACHE=1
context: .
file: docker/debian/Dockerfile.gcc-${{ matrix.os.compiler_version }}-${{ matrix.os.release }}
outputs: type=image,name=${{ env.CONTAINER_IMAGE_GCC }},push-by-digest=true,name-canonical=true,push=true
platforms: ${{ matrix.architecture.platform }}
provenance: mode=max
push: true
sbom: true
labels: ${{ steps.meta-gcc.outputs.labels }}
- name: Prepare container metadata
id: meta
uses: docker/metadata-action@v5
Expand Down Expand Up @@ -120,6 +156,7 @@ jobs:
GCOVR_VERSION=${{ env.GCOVR_VERSION }}
CMAKE_VERSION=${{ env.CMAKE_VERSION }}
DEBIAN_VERSION=${{ matrix.os.release }}
BASE_IMAGE=${{ env.CONTAINER_IMAGE_GCC && format('{0}@{1}', env.CONTAINER_IMAGE_GCC, steps.build-gcc.outputs.digest) || format('gcc:{0}-{1}', matrix.os.compiler_version, matrix.os.release) }}
context: .
file: docker/debian/Dockerfile
outputs: type=image,name=${{ env.CONTAINER_IMAGE }},push-by-digest=true,name-canonical=true,push=true
Expand Down
3 changes: 2 additions & 1 deletion docker/debian/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,8 @@
# image, even though it is not used for Clang.
ARG DEBIAN_VERSION
ARG GCC_VERSION=invalid
FROM gcc:${GCC_VERSION}-${DEBIAN_VERSION} AS gcc-src
ARG BASE_IMAGE=gcc:${GCC_VERSION}-${DEBIAN_VERSION}
FROM ${BASE_IMAGE} AS gcc-src

# ====================== BASE IMAGE ======================
FROM debian:${DEBIAN_VERSION} AS base
Expand Down
156 changes: 156 additions & 0 deletions docker/debian/Dockerfile.gcc-12-bullseye
Original file line number Diff line number Diff line change
@@ -0,0 +1,156 @@
FROM buildpack-deps:bullseye

## NOTE: EVERYTHING BELOW THIS COMMENT IS FROM
## https://github.com/docker-library/gcc/blob/7070981b23d22d3ca790f87bff26f13f3614dd4c/12/Dockerfile


RUN set -eux; \
apt-get update; \
apt-get install -y --no-install-recommends \
# install abigail-tools so we can use abidiff later to verify that we don't break Debian packages
abigail-tools \
; \
rm -rf /var/lib/apt/lists/*

# https://gcc.gnu.org/mirrors.html
ENV GPG_KEYS \

Check warning on line 16 in docker/debian/Dockerfile.gcc-12-bullseye

View workflow job for this annotation

GitHub Actions / build (linux/arm64, ubuntu-24.04-arm, bullseye, gcc, 12)

Legacy key/value format with whitespace separator should not be used

LegacyKeyValueFormat: "ENV key=value" should be used instead of legacy "ENV key value" format More info: https://docs.docker.com/go/dockerfile/rule/legacy-key-value-format/

Check warning on line 16 in docker/debian/Dockerfile.gcc-12-bullseye

View workflow job for this annotation

GitHub Actions / build (linux/amd64, ubuntu-24.04, bullseye, gcc, 12)

Legacy key/value format with whitespace separator should not be used

LegacyKeyValueFormat: "ENV key=value" should be used instead of legacy "ENV key value" format More info: https://docs.docker.com/go/dockerfile/rule/legacy-key-value-format/
# 1024D/745C015A 1999-11-09 Gerald Pfeifer <gerald@pfeifer.com>
B215C1633BCA0477615F1B35A5B3A004745C015A \
# 1024D/B75C61B8 2003-04-10 Mark Mitchell <mark@codesourcery.com>
B3C42148A44E6983B3E4CC0793FA9B1AB75C61B8 \
# 1024D/902C9419 2004-12-06 Gabriel Dos Reis <gdr@acm.org>
90AA470469D3965A87A5DCB494D03953902C9419 \
# 1024D/F71EDF1C 2000-02-13 Joseph Samuel Myers <jsm@polyomino.org.uk>
80F98B2E0DAB6C8281BDF541A7C8C3B2F71EDF1C \
# 2048R/FC26A641 2005-09-13 Richard Guenther <richard.guenther@gmail.com>
7F74F97C103468EE5D750B583AB00996FC26A641 \
# 1024D/C3C45C06 2004-04-21 Jakub Jelinek <jakub@redhat.com>
33C235A34C46AA3FFB293709A328C3A2C3C45C06 \
# 4096R/09B5FA62 2020-05-28 Jakub Jelinek <jakub@redhat.com>
D3A93CAD751C2AF4F8C7AD516C35B99309B5FA62

# https://gcc.gnu.org/mirrors.html
ENV GCC_MIRRORS \

Check warning on line 33 in docker/debian/Dockerfile.gcc-12-bullseye

View workflow job for this annotation

GitHub Actions / build (linux/arm64, ubuntu-24.04-arm, bullseye, gcc, 12)

Legacy key/value format with whitespace separator should not be used

LegacyKeyValueFormat: "ENV key=value" should be used instead of legacy "ENV key value" format More info: https://docs.docker.com/go/dockerfile/rule/legacy-key-value-format/

Check warning on line 33 in docker/debian/Dockerfile.gcc-12-bullseye

View workflow job for this annotation

GitHub Actions / build (linux/amd64, ubuntu-24.04, bullseye, gcc, 12)

Legacy key/value format with whitespace separator should not be used

LegacyKeyValueFormat: "ENV key=value" should be used instead of legacy "ENV key value" format More info: https://docs.docker.com/go/dockerfile/rule/legacy-key-value-format/
https://ftpmirror.gnu.org/gcc \
https://mirrors.kernel.org/gnu/gcc \
https://bigsearcher.com/mirrors/gcc/releases \
http://www.netgull.com/gcc/releases \
https://ftpmirror.gnu.org/gcc \
# "sourceware.org" is the canonical upstream release host (the host of "gcc.gnu.org")
https://sourceware.org/pub/gcc/releases \
# only attempt the origin FTP as a mirror of last resort
ftp://ftp.gnu.org/gnu/gcc

# Last Modified: 2025-07-11
ENV GCC_VERSION 12.5.0

Check warning on line 45 in docker/debian/Dockerfile.gcc-12-bullseye

View workflow job for this annotation

GitHub Actions / build (linux/arm64, ubuntu-24.04-arm, bullseye, gcc, 12)

Legacy key/value format with whitespace separator should not be used

LegacyKeyValueFormat: "ENV key=value" should be used instead of legacy "ENV key value" format More info: https://docs.docker.com/go/dockerfile/rule/legacy-key-value-format/

Check warning on line 45 in docker/debian/Dockerfile.gcc-12-bullseye

View workflow job for this annotation

GitHub Actions / build (linux/amd64, ubuntu-24.04, bullseye, gcc, 12)

Legacy key/value format with whitespace separator should not be used

LegacyKeyValueFormat: "ENV key=value" should be used instead of legacy "ENV key value" format More info: https://docs.docker.com/go/dockerfile/rule/legacy-key-value-format/
# Docker EOL: 2027-01-11

RUN set -ex; \
\
savedAptMark="$(apt-mark showmanual)"; \
apt-get update; \
apt-get install -y --no-install-recommends \
dpkg-dev \
flex \
gnupg \
; \
rm -r /var/lib/apt/lists/*; \
\
_fetch() { \
local fetch="$1"; shift; \
local file="$1"; shift; \
for mirror in $GCC_MIRRORS; do \
if curl -fL "$mirror/$fetch" -o "$file"; then \
return 0; \
fi; \
done; \
echo >&2 "error: failed to download '$fetch' from several mirrors"; \
return 1; \
}; \
\
_fetch "gcc-$GCC_VERSION/gcc-$GCC_VERSION.tar.xz.sig" 'gcc.tar.xz.sig'; \
_fetch "gcc-$GCC_VERSION/gcc-$GCC_VERSION.tar.xz" 'gcc.tar.xz'; \
export GNUPGHOME="$(mktemp -d)"; \
for key in $GPG_KEYS; do \
gpg --batch --keyserver keyserver.ubuntu.com --recv-keys "$key"; \
done; \
gpg --batch --verify gcc.tar.xz.sig gcc.tar.xz; \
gpgconf --kill all; \
rm -rf "$GNUPGHOME"; \
mkdir -p /usr/src/gcc; \
tar -xf gcc.tar.xz -C /usr/src/gcc --strip-components=1; \
rm gcc.tar.xz*; \
\
cd /usr/src/gcc; \
\
# "download_prerequisites" pulls down a bunch of tarballs and extracts them,
# but then leaves the tarballs themselves lying around
./contrib/download_prerequisites; \
{ rm *.tar.* || true; }; \
\
# explicitly update autoconf config.guess and config.sub so they support more arches/libcs
for f in config.guess config.sub; do \
wget -O "$f" "https://git.savannah.gnu.org/cgit/config.git/plain/$f?id=7d3d27baf8107b630586c962c057e22149653deb"; \
# find any more (shallow) copies of the file we grabbed and update them too
find -mindepth 2 -name "$f" -exec cp -v "$f" '{}' ';'; \
done; \
\
dir="$(mktemp -d)"; \
cd "$dir"; \
\
extraConfigureArgs=''; \
dpkgArch="$(dpkg --print-architecture)"; \
case "$dpkgArch" in \
# with-arch: https://salsa.debian.org/toolchain-team/gcc/-/blob/gcc-13-debian/debian/rules2#L533-573
# with-float: https://salsa.debian.org/toolchain-team/gcc/-/blob/gcc-13-debian/debian/rules2#L521-523
# with-mode: https://salsa.debian.org/toolchain-team/gcc/-/blob/gcc-13-debian/debian/rules2#L571
armel) \
extraConfigureArgs="$extraConfigureArgs --with-arch=armv5te --with-float=soft" \
;; \
armhf) \
# https://bugs.launchpad.net/ubuntu/+source/gcc-defaults/+bug/1939379/comments/2
extraConfigureArgs="$extraConfigureArgs --with-arch=armv7-a+fp --with-float=hard --with-mode=thumb" \
;; \
\
# with-arch-32: https://salsa.debian.org/toolchain-team/gcc/-/blob/gcc-13-debian/debian/rules2#L670
i386) \
extraConfigureArgs="$extraConfigureArgs --with-arch-32=i686"; \
;; \
esac; \
\
gnuArch="$(dpkg-architecture --query DEB_BUILD_GNU_TYPE)"; \
/usr/src/gcc/configure \
--build="$gnuArch" \
--disable-multilib \
--enable-languages=c,c++,fortran,go \
$extraConfigureArgs \
; \
make -j "$(nproc)"; \
make install-strip; \
\
cd ..; \
\
rm -rf "$dir" /usr/src/gcc; \
\
apt-mark auto '.*' > /dev/null; \
[ -z "$savedAptMark" ] || apt-mark manual $savedAptMark; \
apt-get purge -y --auto-remove -o APT::AutoRemove::RecommendsImportant=false

# gcc installs .so files in /usr/local/lib64 (and /usr/local/lib)...
RUN set -ex; \
# this filename needs to sort higher than all the architecture filenames ("aarch64-...", "armeabi...", etc)
{ echo '/usr/local/lib64'; echo '/usr/local/lib'; } > /etc/ld.so.conf.d/000-local-lib.conf; \
ldconfig -v; \
# the libc created by gcc might be too old for a newer Debian
# check that the Debian libstdc++ doesn't have newer requirements than the gcc one
deb="$(readlink -ve /usr/lib/*/libstdc++.so* | head -1)"; \
gcc="$(readlink -ve /usr/local/lib*/libstdc++.so | head -1)"; \
# using LD_PRELOAD to make sure "abidiff" itself doesn't fail with the exact error we're trying to test for 😂😭
LD_PRELOAD="$deb" abidiff --no-added-syms "$deb" "$gcc"

# ensure that alternatives are pointing to the new compiler and that old one is no longer used
RUN set -ex; \
dpkg-divert --divert /usr/bin/gcc.orig --rename /usr/bin/gcc; \
dpkg-divert --divert /usr/bin/g++.orig --rename /usr/bin/g++; \
dpkg-divert --divert /usr/bin/gfortran.orig --rename /usr/bin/gfortran; \
update-alternatives --install /usr/bin/cc cc /usr/local/bin/gcc 999
50 changes: 50 additions & 0 deletions docker/debian/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,30 @@ Build image for `gcc` supports packaging.
In order to build an image, run the commands below from the root directory of
the repository.

#### Note on old Debian releases

This image supports variety of releases of Debian, GCC and Clang.

The GCC binaries are sourced from [Docker "Official Image" for gcc](https://github.com/docker-library/gcc)
with an important caveat - in order to install a GCC release in older
Debian versions, we keep a local copy of `Dockerfile` from the above repository,
backported to an older Debian base image. Such dockerfiles are stored in this
directory with special file extension, e.g. `gcc-12-bullseye`. They are not altered from
the source, except for change of the base image to older Debian version. They also
show in a comment the specific `Dockerfile` they have been sourced from.

If you want to build a Docker image for GCC and an older Debian version, you should
first build GCC using an appropriate image, giving it the _exact_ name and tag as
passed later to the main `Dockerfile` as `BASE_IMAGE`. This may require significant
CPU resources and take some time (e.g. 30 minutes using 40 cores) and it's needed
to ensure that we do not use an old GCC release with known, and fixed, bugs.

For example:

```shell
docker buildx build . --progress plain --file docker/debian/Dockerfile.gcc-12-bullseye --tag localhost.localdomain/gcc:12-bullseye
```

#### Building the Docker image for GCC

Ensure you've run the login command above to authenticate with the Docker
Expand All @@ -60,6 +84,32 @@ docker buildx build . \
--tag ${CONTAINER_REGISTRY}/${CONTAINER_IMAGE}
```

If you have prepared a GCC image for an older Debian version, you also need
to explicitly set `BASE_IMAGE` build argument, e.g.

```shell
DEBIAN_VERSION=bullseye
GCC_VERSION=12
CONAN_VERSION=2.19.1
GCOVR_VERSION=8.3
CMAKE_VERSION=3.31.6
BASE_IMAGE=localhost.localdomain/gcc:12-bullseye
CONTAINER_IMAGE=xrplf/ci/debian-${DEBIAN_VERSION}:gcc-${GCC_VERSION}

docker buildx build . \
--file docker/debian/Dockerfile \
--target gcc \
--build-arg BUILDKIT_DOCKERFILE_CHECK=skip=InvalidDefaultArgInFrom \
--build-arg BUILDKIT_INLINE_CACHE=1 \
--build-arg CONAN_VERSION=${CONAN_VERSION} \
--build-arg DEBIAN_VERSION=${DEBIAN_VERSION} \
--build-arg GCC_VERSION=${GCC_VERSION} \
--build-arg GCOVR_VERSION=${GCOVR_VERSION} \
--build-arg CMAKE_VERSION=${CMAKE_VERSION} \
--build-arg BASE_IMAGE=${BASE_IMAGE} \
--tag ${CONTAINER_REGISTRY}/${CONTAINER_IMAGE}
```

#### Building the Docker image for Clang

Ensure you've run the login command above to authenticate with the Docker
Expand Down