ci(release): sign container images + attach SLSA provenance/SBOM (cosign keyless)

fgogolli · fgogolli · commit 99262d2dc4c9 · 2026-07-15T08:48:22.000+01:00
Achieves SLSA Build L2 for all pushed container images.

Note: Scorecard's Signed-Releases check reads only GitHub Release *assets*
(tarballs/wheels), not container registries, so this does not change the
Scorecard score. The signing is for genuine supply-chain security: consumers
of the GHCR images can verify who built them, via what workflow, from what
commit.

Design choices
--------------
* Digest capture: `--metadata-file metadata.json` on `docker buildx build`;
  `jq -r '."containerimage.digest"'` extracts the immutable sha256 index
  digest. Signing is done against `IMAGE@DIGEST` — never against a mutable
  tag.
* Signature: `cosign sign --yes` (keyless/Sigstore OIDC). The Rekor
  transparency-log entry embeds the GitHub Actions workflow identity.
* Provenance: `actions/attest-build-provenance` with `push-to-registry: true`.
  This generates a SLSA provenance attestation signed with the OIDC token and
  stores it in the registry. Rationale for using both cosign + attest:
  cosign produces the OCI referrer signature object; attest-build-provenance
  produces a GitHub-native SLSA provenance attestation — they are
  complementary and independently verifiable.
* SBOM: `--sbom=true` on the BuildKit invocation injects an in-toto SBOM
  attestation into the image at build time (no separate step needed).
* `--provenance=true` on the BuildKit build also injects the BuildKit-level
  SLSA provenance attestation directly into the OCI index.

prod-containers job
-------------------
* Added `id-token: write` + `attestations: write` permissions.
* Added `--provenance=true --sbom=true --metadata-file metadata.json` to the
  buildx invocation.
* Added cosign-installer, `cosign sign`, and attest-build-provenance steps
  after the build+push step.

prod-manifests job
------------------
* Added `id-token: write` + `attestations: write` permissions.
* cosign-installer step added before the manifest-creation step.
* After each `imagetools create`, the manifest-list digest is captured via
  `docker buildx imagetools inspect --format '{{json .Manifest}}'` and
  collected; a trailing `cosign sign` step signs all unique manifest-list
  digests. The per-python images (the leaves) are already signed in
  prod-containers; this closes the chain for the convenience tags.

dev-containers job (container-pipeline)
----------------------------------------
* Applied identical signing (cosign + attest-build-provenance) gated on
  `steps.build.outputs.pushed == 'true'`, so PR builds and local-only builds
  skip signing entirely. Push paths (push to main/develop, workflow_dispatch
  with push_images=true) receive the same supply-chain guarantees as prod.
diff --git a/.github/workflows/dev-containers.yml b/.github/workflows/dev-containers.yml
@@ -103,6 +103,10 @@ jobs:
       contents: read
       packages: write
       security-events: write
+      # Required for keyless cosign signing (ambient OIDC token) and
+      # actions/attest-build-provenance on pushed images.
+      id-token: write
+      attestations: write
 
     steps:
     - name: Checkout code
@@ -134,6 +138,7 @@ jobs:
       run: make build
 
     - name: Build and scan container image
+      id: build
       shell: bash
       env:
         REGISTRY: ${{ needs.config.outputs.container-registry }}
@@ -169,12 +174,23 @@ jobs:
             --build-arg "VCS_REF=$(git rev-parse --short HEAD)" \
             --build-arg "PYTHON_VERSION=$PYTHON_VERSION" \
             --build-arg "PACKAGE_NAME_SHORT=$PACKAGE_NAME_SHORT" \
+            --provenance=true \
+            --sbom=true \
+            --metadata-file metadata.json \
             --cache-from type=gha \
             --cache-to type=gha,mode=max \
             --push \
             -t "$REGISTRY/$IMAGE_NAME:$VERSION-python$PYTHON_VERSION" \
             .
 
+          # Extract the immutable digest for the signing steps below.
+          DIGEST=$(jq -r '."containerimage.digest"' metadata.json)
+          {
+            echo "digest=${DIGEST}"
+            echo "image=${REGISTRY}/${IMAGE_NAME}"
+            echo "pushed=true"
+          } >> "$GITHUB_OUTPUT"
+
           # Pull back a single-arch image for local Trivy scanning.
           # Multi-arch manifests cannot be loaded/scanned; we use the first
           # entry of build-platforms (SCAN_PLATFORM) rather than hardcoding amd64.
@@ -195,8 +211,29 @@ jobs:
             --load \
             -t "$REGISTRY/$IMAGE_NAME:$VERSION-python$PYTHON_VERSION" \
             .
+          echo "pushed=false" >> "$GITHUB_OUTPUT"
         fi
 
+    - name: Install cosign
+      # Only needed when an image was pushed; skip on PR / local-only builds.
+      if: steps.build.outputs.pushed == 'true'
+      uses: sigstore/cosign-installer@6f9f17788090df1f26f669e9d70d6ae9567deba6  # v4.1.2
+
+    - name: Sign container image (keyless / OIDC)
+      if: steps.build.outputs.pushed == 'true'
+      env:
+        DIGEST: ${{ steps.build.outputs.digest }}
+        IMAGE: ${{ steps.build.outputs.image }}
+      run: cosign sign --yes "$IMAGE@$DIGEST"
+
+    - name: Attach SLSA provenance attestation
+      if: steps.build.outputs.pushed == 'true'
+      uses: actions/attest-build-provenance@0f67c3f4856b2e3261c31976d6725780e5e4c373  # v4.1.1
+      with:
+        subject-name: ${{ steps.build.outputs.image }}
+        subject-digest: ${{ steps.build.outputs.digest }}
+        push-to-registry: true
+
     - name: Run Trivy security scan
       uses: aquasecurity/trivy-action@ed142fd0673e97e23eac54620cfb913e5ce36c25  # v0.36.0
       with:
diff --git a/.github/workflows/prod-release.yml b/.github/workflows/prod-release.yml
@@ -95,6 +95,10 @@ jobs:
       contents: read
       packages: write
       security-events: write
+      # Required for keyless cosign signing (ambient OIDC token) and
+      # actions/attest-build-provenance.
+      id-token: write
+      attestations: write
 
     steps:
     - name: Checkout code
@@ -119,6 +123,7 @@ jobs:
         path: dist/
 
     - name: Build and push container image
+      id: build
       env:
         REGISTRY: ${{ needs.config.outputs.container-registry }}
         IMAGE_NAME: ${{ needs.config.outputs.container-image }}
@@ -134,17 +139,49 @@ jobs:
           --build-arg "VCS_REF=$(git rev-parse --short HEAD)" \
           --build-arg "PYTHON_VERSION=$PYTHON_VERSION" \
           --build-arg "PACKAGE_NAME_SHORT=$PACKAGE_NAME_SHORT" \
+          --provenance=true \
+          --sbom=true \
+          --metadata-file metadata.json \
           --cache-from type=gha \
           --cache-to type=gha,mode=max \
           --push \
           -t "$REGISTRY/$IMAGE_NAME:$VERSION-python$PYTHON_VERSION" \
           .
 
+        # Extract the immutable digest from the BuildKit metadata output.
+        # containerimage.digest is the multi-arch index digest (sha256:…).
+        DIGEST=$(jq -r '."containerimage.digest"' metadata.json)
+        echo "digest=${DIGEST}" >> "$GITHUB_OUTPUT"
+        echo "image=${REGISTRY}/${IMAGE_NAME}" >> "$GITHUB_OUTPUT"
+
         # Trivy cannot scan a multi-arch manifest — pull back a single-arch image.
         # Use the first entry of build-platforms (same approach as dev-containers.yml).
         SCAN_PLATFORM=$(echo "$BUILD_PLATFORMS" | cut -d, -f1)
         docker pull --platform "$SCAN_PLATFORM" "$REGISTRY/$IMAGE_NAME:$VERSION-python$PYTHON_VERSION"
 
+    - name: Install cosign
+      uses: sigstore/cosign-installer@6f9f17788090df1f26f669e9d70d6ae9567deba6  # v4.1.2
+
+    - name: Sign container image (keyless / OIDC)
+      # Signs the immutable digest — not a mutable tag — using the GitHub OIDC
+      # ambient credential.  The Rekor transparency-log entry records the
+      # workflow identity, giving consumers a verifiable supply-chain link.
+      env:
+        DIGEST: ${{ steps.build.outputs.digest }}
+        IMAGE: ${{ steps.build.outputs.image }}
+      run: cosign sign --yes "$IMAGE@$DIGEST"
+
+    - name: Attach SLSA provenance attestation
+      # Generates a SLSA Build L2-conformant provenance attestation signed with
+      # the GitHub OIDC token and pushes it to the registry alongside the image.
+      # Complements the cosign signature: signature proves who signed; provenance
+      # records exactly what was built and how (workflow, inputs, commit SHA).
+      uses: actions/attest-build-provenance@0f67c3f4856b2e3261c31976d6725780e5e4c373  # v4.1.1
+      with:
+        subject-name: ${{ steps.build.outputs.image }}
+        subject-digest: ${{ steps.build.outputs.digest }}
+        push-to-registry: true
+
     - name: Run Trivy security scan
       uses: aquasecurity/trivy-action@ed142fd0673e97e23eac54620cfb913e5ce36c25  # v0.36.0
       with:
@@ -171,6 +208,9 @@ jobs:
     permissions:
       contents: read
       packages: write
+      # Required for keyless cosign signing of the manifest-list digests.
+      id-token: write
+      attestations: write
 
     steps:
     - name: Checkout code
@@ -183,7 +223,11 @@ jobs:
         username: ${{ github.actor }}
         password: ${{ secrets.GITHUB_TOKEN }}
 
+    - name: Install cosign
+      uses: sigstore/cosign-installer@6f9f17788090df1f26f669e9d70d6ae9567deba6  # v4.1.2
+
     - name: Create production tag manifests
+      id: manifests
       shell: bash
       env:
         REGISTRY: ${{ needs.config.outputs.container-registry }}
@@ -206,6 +250,9 @@ jobs:
           fi
         done < <(make get-container-tags-env)
 
+        # Collect all created manifest-list digests so the signing step can sign them.
+        MANIFEST_REFS=""
+
         # Create primary tags (latest, version)
         if [[ -n "${primary_tags:-}" ]]; then
           IFS=',' read -ra TAGS <<< "$primary_tags"
@@ -214,6 +261,9 @@ jobs:
             docker buildx imagetools create \
               -t "$REGISTRY/$IMAGE:$tag" \
               "$REGISTRY/$IMAGE:$VERSION-python$DEFAULT_PYTHON"
+            # Resolve the digest of the newly-created manifest list.
+            DIGEST=$(docker buildx imagetools inspect --format '{{json .Manifest}}' "$REGISTRY/$IMAGE:$tag" | jq -r '.digest')
+            MANIFEST_REFS="${MANIFEST_REFS} $REGISTRY/$IMAGE@$DIGEST"
           done
         fi
 
@@ -226,9 +276,27 @@ jobs:
             docker buildx imagetools create \
               -t "$REGISTRY/$IMAGE:$tag" \
               "$REGISTRY/$IMAGE:$VERSION-python$py_ver"
+            DIGEST=$(docker buildx imagetools inspect --format '{{json .Manifest}}' "$REGISTRY/$IMAGE:$tag" | jq -r '.digest')
+            MANIFEST_REFS="${MANIFEST_REFS} $REGISTRY/$IMAGE@$DIGEST"
           done
         fi
 
+        # Deduplicate refs (multiple tags may resolve to the same digest).
+        UNIQUE_REFS=$(echo "$MANIFEST_REFS" | tr ' ' '\n' | sort -u | tr '\n' ' ')
+        echo "manifest-refs=${UNIQUE_REFS}" >> "$GITHUB_OUTPUT"
+
+    - name: Sign manifest-list images (keyless / OIDC)
+      # The per-python images were already signed in prod-containers; signing the
+      # manifest lists (latest, version, pythonX.Y tags) closes the chain so
+      # consumers of the convenience tags can also verify a signature.
+      env:
+        MANIFEST_REFS: ${{ steps.manifests.outputs.manifest-refs }}
+      run: |
+        for ref in $MANIFEST_REFS; do
+          echo "Signing manifest list: $ref"
+          cosign sign --yes "$ref"
+        done
+
   prod-pypi:
     name: Publish to PyPI
     needs: [config, build-wheel]
