Conversation
.github/workflows/debian.yml
Outdated
| - name: Prepare gcc image metadata | ||
| if: ${{ matrix.os.release == 'bullseye' }} | ||
| id: meta-gcc | ||
| uses: docker/metadata-action@v5 | ||
| env: | ||
| DOCKER_METADATA_ANNOTATIONS_LEVELS: manifest,manifest-descriptor | ||
| with: | ||
| # NOTE, tag and image must match base of docker/debian/Dockerfile | ||
| images: gcc | ||
| tags: | | ||
| type=raw,value=${{ matrix.os.compiler_version }}-${{ matrix.os.release }} | ||
| labels: | | ||
| org.opencontainers.image.authors=For inquiries, please use https://${{ github.repository }}/issues | ||
| org.opencontainers.image.documentation=https://${{ github.repository }} | ||
| org.opencontainers.image.vendor=XRPLF | ||
| org.opencontainers.image.title=${{ env.CONTAINER_REPOSITORY }}-gcc | ||
| - name: Build gcc image do not push | ||
| if: ${{ matrix.os.release == 'bullseye' }} | ||
| uses: docker/build-push-action@v6 | ||
| with: | ||
| build-args: | | ||
| BUILDKIT_DOCKERFILE_CHECK=skip=InvalidDefaultArgInFrom | ||
| BUILDKIT_INLINE_CACHE=1 | ||
| context: . | ||
| file: docker/debian/Dockerfile.gcc-${{ matrix.os.compiler_version }}-${{ matrix.os.release }} | ||
| outputs: type=image,name=gcc,push=false | ||
| platforms: ${{ matrix.architecture.platform }} | ||
| provenance: mode=max | ||
| push: false | ||
| sbom: true | ||
| labels: ${{ steps.meta-gcc.outputs.labels }} | ||
| tags: ${{ steps.meta-gcc.outputs.tags }} |
There was a problem hiding this comment.
Is this going to result in building this image on each future commit to the repo that affects Debian, even if the change does not impact the GCC image you're building here? These steps here seem like they should be a one-time thing.
Is it possible to:
- Build this image in a separate job and push it to our CI registry.
- Change the
FROM gcc:${GCC_VERSION}-${DEBIAN_VERSION} AS gcc-srclogic to conditionally choosegccorghcr.io/xrplf/ci/gcc-12-bullseye.
There was a problem hiding this comment.
I do not like doing it because I'm lazy, but it's an excellent point.
I guess I will do it, but it raises some questions:
- Where to keep the dockerfiles - same location
docker/debian? I'd prefer it this way, as it keeps theREADME.mdmaking sense - This will require a new workflow, what do we call it ?
debian-gcc? - How do we ensure consistency between both workflows re. the image being created (in one) and used (in other) ? At some point we will add
bookwormto "old Debian versions" and we need to ensure both workflows see it as such. Delegate to action ? - Do we want it right away in this PR, or sometime later (this PR is kind-of urgent)
There was a problem hiding this comment.
First of all, it's ok to do it as a follow-up.
Regarding where to put it, as - for now - it will be exclusively used by the Debian image, I would recommend keeping the GCC Dockerfile in the same location.
Options:
- Create a separate (reusable) workflow that can be started from the Debian workflow, so you can use
needs:to wait for it; in that case the concurrency group of the separate workflow may need to be different, however. - (easier) Create a separate job in the Debian workflow that runs first, so you can also depend on it.
- (best) Create as an action that executes as a step to force everything runs on the same machine, to ensure consistency between building the image and actually using it.
With the first two options there's a greater risk, as you already pointed out, that the Debian job will run on a different machine and pull the last known good image, rather than an updated image. The risk doesn't disappear by running it as an action however, since the runners are shared between different PRs, and presumably the Docker cache as well.
To ensure consistency, can the GCC image be tagged with the SHA or the PR number? Although that would make the Dockerfile more complicated. I also would like to avoid having tons of GCC images in our registry. Would it be possible to:
- Build but not push the GCC image using the SHA for PR commits.
- Build and push the GCC image using the real tag upon merge?
There was a problem hiding this comment.
For now I push it without a tag (at all). It is identified further down in the workflow by digest only. We will have to fix this, but it should be enough to unblock us for now.
b232b79 to
f37027a
Compare
77477e8 to
c7a4a66
Compare
c7a4a66 to
3c20c33
Compare
This reverts commit 055005a.
| org.opencontainers.image.title=${{ env.CONTAINER_REPOSITORY_GCC }} | ||
| - name: Build gcc image | ||
| # Note, we always push this image, otherwise steps.build will fail. | ||
| # Also, because we always push it, we intentionally do not tag it. |
There was a problem hiding this comment.
Is this going to pollute the CI registry with tons of untagged images? Let's think afterwards about how to optimize this.
There was a problem hiding this comment.
I do not know how ghcr.io treats untagged images, I hope they are "garbage collected", also it would only happen when this workflow runs which (after the period of high initial activity) won't be hopefully that much.
However I agree this needs to be addressed, which is why I put a TODO in the next line.
I am thinking of building and pushing these images in a separate workflow, and this one would simply use tagged image e.g. ghcr.io/xrplf/ci/debian-gcc:12-bullseye. These won't be updated often (in fact, once or twice per months seem realistic) so we do not need to remember their digests etc. because, with this rare rebuilds, network race conditions won't bother us.
This is followup to #37