Skip to content

chore: only rebuild golden image on changes #556

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
dd23 merged 10 commits into from
May 4, 2026
Merged

chore: only rebuild golden image on changes #556

Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
b9cef34
chore: build golden image only on change
dd23 Apr 29, 2026
6e790c6
chore: handle case when latest differs from current
dd23 Apr 29, 2026
02e6ef7
fix: also update latest
dd23 Apr 29, 2026
9347ed3
chore: simplify
dd23 Apr 29, 2026
5267aa5
fix: suppress zizmor warning (we do the same in main.yml)
dd23 Apr 29, 2026
92c3cf8
chore: increase test timeout
dd23 Apr 29, 2026
986b693
Merge branch 'main' into dd/chore/golden_image_cache
dd23 Apr 30, 2026
0705d2d
chore(ci): derive golden base image from docker/base/Dockerfile
dd23 Apr 30, 2026
c99a1c7
chore: remove unneeded success check
dd23 May 4, 2026
ee39763
chore: remove unneeded sha256 hash call (we still do one at the end)
dd23 May 4, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
165 changes: 156 additions & 9 deletions .github/workflows/docker-build.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -52,8 +52,8 @@ on:
required: true
outputs:
image_tag:
description: 'Docker Tag generated by this task'
value: '${{ jobs.golden-image.outputs.image_tag }}'
description: 'Docker Tag for the core-service / core-client images (per-commit SHA)'
value: '${{ jobs.core-service.outputs.image_tag }}'
enclave_pcr0:
description: 'Enclave PCR0 generated by this task'
value: '${{ jobs.enclave.outputs.enclave_pcr0 }}'
Expand All @@ -67,15 +67,103 @@ on:

permissions: {}

concurrency:
group: docker-build-rust-golden-latest
cancel-in-progress: false

jobs:
############################################################################
# Computes a content-addressed tag for the golden image and checks whether
Comment thread
dd23 marked this conversation as resolved.
# an image with that tag already exists in GHCR. The tag is derived from
Comment thread
dd23 marked this conversation as resolved.
# the inputs that actually determine the image content: the Dockerfile,
# rust-toolchain.toml, and the resolved digest of the Chainguard base
# image (which is a moving tag upstream). When `cache_hit == 'true'`, the
# content-tagged image already exists and can be retagged as `latest` before
# the downstream Dockerfiles consume their existing `rust-golden-image:latest`
# base.
golden-image-tag:
name: docker-build/golden-image-tag
runs-on: ubuntu-latest
permissions:
contents: read # Required to checkout repository code
packages: read # Required to inspect manifests on GHCR
outputs:
golden_tag: ${{ steps.compute.outputs.golden_tag }}
cache_hit: ${{ steps.lookup.outputs.cache_hit }}
steps:
- name: Checkout
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
with:
persist-credentials: false
- name: Login to Chainguard Container Registry
uses: docker/login-action@4907a6ddec9925e35a0a9e82d7399ccc52663121 # v4.1.0
with:
registry: cgr.dev
username: ${{ secrets.CGR_USERNAME }} # zizmor: ignore[secrets-outside-env] CGR creds are shared by docker-build workflows.
password: ${{ secrets.CGR_PASSWORD }} # zizmor: ignore[secrets-outside-env] CGR creds are shared by docker-build workflows.
- name: Login to GitHub Container Registry
uses: docker/login-action@4907a6ddec9925e35a0a9e82d7399ccc52663121 # v4.1.0
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Compute content hash
id: compute
run: |
set -euo pipefail
DOCKERFILE="docker/base/Dockerfile"
# Single source of truth: read the first external FROM line from the
# Dockerfile so bumping the base image there is enough to trigger a
# golden image rebuild. Skips any `--flag=...` arguments to FROM.
BASE_IMAGE=$(awk '
toupper($1) == "FROM" {
for (i = 2; i <= NF; i++) {
if ($i !~ /^--/) { print $i; exit }
}
}
' "${DOCKERFILE}")
if [[ -z "${BASE_IMAGE}" ]]; then
echo "Failed to parse base image from ${DOCKERFILE}" >&2
exit 1
fi
BASE_DIGEST=$(docker buildx imagetools inspect "${BASE_IMAGE}" --format '{{.Manifest.Digest}}')
echo "Resolved ${BASE_IMAGE} -> ${BASE_DIGEST}"
CONTENT_HASH=$(
{
echo "base=${BASE_DIGEST}"
cat "${DOCKERFILE}" rust-toolchain.toml
} | sha256sum | cut -c1-12
)
Comment thread
dd23 marked this conversation as resolved.
GOLDEN_TAG="golden-${CONTENT_HASH}"
echo "golden_tag=${GOLDEN_TAG}" >> "$GITHUB_OUTPUT"
echo "Computed golden image tag: ${GOLDEN_TAG}"
- name: Check whether golden image already exists
id: lookup
env:
GOLDEN_TAG: ${{ steps.compute.outputs.golden_tag }}
run: |
set -euo pipefail
GOLDEN_IMAGE="ghcr.io/zama-ai/kms/rust-golden-image:${GOLDEN_TAG}"

if docker manifest inspect "${GOLDEN_IMAGE}" >/dev/null 2>&1; then
echo "cache_hit=true" >> "$GITHUB_OUTPUT"
echo "Cache hit: ${GOLDEN_IMAGE} already exists."
else
echo "cache_hit=false" >> "$GITHUB_OUTPUT"
echo "Cache miss: golden image will be built and pushed as :${GOLDEN_TAG}."
fi

############################################################################
# Builds Docker image for base Rust Golden Image
#
# Skipped entirely when `golden-image-tag` reports a cache hit, so that we
# only rebuild the golden image when its content actually changes.
golden-image:
name: docker-build/golden-image (bpr)
if: github.event_name != 'workflow_dispatch' || inputs.build-golden-image
# needs:
# - start-amd64-runner
needs: [golden-image-tag]
if: |
(github.event_name != 'workflow_dispatch' || inputs.build-golden-image) &&
needs.golden-image-tag.outputs.cache_hit != 'true'
uses: zama-ai/ci-templates/.github/workflows/common-docker.yml@478ee7de5baeab4fd49edfc42c9b81a56f6cfb03 # v1.0.5
permissions:
actions: read # Required to read workflow run information
Expand All @@ -95,22 +183,78 @@ jobs:
runs-on-amd: "runs-on=${{ github.run_id }}/runner=64cpu-linux-x64/spot=false/volume=200gb/extras=s3-cache"
runs-on-arm: "runs-on=${{ github.run_id }}/runner=64cpu-linux-arm64/spot=false/volume=200gb/extras=s3-cache"
rust-toolchain-file-path: rust-toolchain.toml
image-tag: ${{ needs.golden-image-tag.outputs.golden_tag }}
secrets:
BLOCKCHAIN_ACTIONS_TOKEN: ${{ secrets.BLOCKCHAIN_ACTIONS_TOKEN }}
AWS_ACCESS_KEY_S3_USER: ${{ secrets.AWS_ACCESS_KEY_S3_USER }}
AWS_SECRET_KEY_S3_USER: ${{ secrets.AWS_SECRET_KEY_S3_USER }}
CGR_USERNAME: ${{ secrets.CGR_USERNAME }}
CGR_PASSWORD: ${{ secrets.CGR_PASSWORD }}

############################################################################
# Retags the content-addressed golden image as latest.
#
# The downstream Dockerfiles cannot receive the content tag through
# common-docker.yml@v1.0.5, so keep `latest` synchronized before they build.
############################################################################
golden-image-latest:
name: docker-build/golden-image-latest
runs-on: ubuntu-latest
needs:
- golden-image-tag
- golden-image
if: |
!cancelled() &&
needs.golden-image-tag.result == 'success' &&
(github.event_name != 'workflow_dispatch' || inputs.build-golden-image) &&
(
needs.golden-image.result == 'success' ||
needs.golden-image-tag.outputs.cache_hit == 'true'
)
permissions:
packages: write # Required to update the latest tag in GHCR
steps:
- name: Login to GitHub Container Registry
uses: docker/login-action@4907a6ddec9925e35a0a9e82d7399ccc52663121 # v4.1.0
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}

- name: Retag golden image as latest
env:
GOLDEN_TAG: ${{ needs.golden-image-tag.outputs.golden_tag }}
run: |
set -euo pipefail
GOLDEN_IMAGE="ghcr.io/zama-ai/kms/rust-golden-image:${GOLDEN_TAG}"
LATEST_IMAGE="ghcr.io/zama-ai/kms/rust-golden-image:latest"

docker buildx imagetools create \
--tag "${LATEST_IMAGE}" \
"${GOLDEN_IMAGE}"

GOLDEN_DIGEST=$(docker buildx imagetools inspect "${GOLDEN_IMAGE}" --format '{{.Manifest.Digest}}')
LATEST_DIGEST=$(docker buildx imagetools inspect "${LATEST_IMAGE}" --format '{{.Manifest.Digest}}')
echo "${GOLDEN_IMAGE} -> ${GOLDEN_DIGEST}"
echo "${LATEST_IMAGE} -> ${LATEST_DIGEST}"

if [[ "${GOLDEN_DIGEST}" != "${LATEST_DIGEST}" ]]; then
echo "Latest tag does not match ${GOLDEN_TAG} after retagging"
exit 1
fi

############################################################################
# Builds Docker image for core-client
#
core-client:
name: docker-build/core-client (bpr)
if: github.event_name != 'workflow_dispatch' || inputs.build-core-client
needs:
# - start-amd64-runner
- golden-image
- golden-image-latest
if: |
!cancelled() &&
needs.golden-image-latest.result == 'success' &&
(github.event_name != 'workflow_dispatch' || inputs.build-core-client)
uses: zama-ai/ci-templates/.github/workflows/common-docker.yml@478ee7de5baeab4fd49edfc42c9b81a56f6cfb03 # v1.0.5
permissions:
actions: read # Required to read workflow run information
Expand Down Expand Up @@ -142,10 +286,13 @@ jobs:
#
core-service:
name: docker-build/core-service (bpr)
if: github.event_name != 'workflow_dispatch' || inputs.build-core-service
needs:
# - start-amd64-runner
- golden-image
- golden-image-latest
if: |
!cancelled() &&
needs.golden-image-latest.result == 'success' &&
(github.event_name != 'workflow_dispatch' || inputs.build-core-service)
uses: zama-ai/ci-templates/.github/workflows/common-docker.yml@478ee7de5baeab4fd49edfc42c9b81a56f6cfb03 # v1.0.5
permissions:
actions: read # Required to read workflow run information
Expand Down
Loading