Add unique per-build image tags via generate-tags task

container/README.md

@@ -101,9 +101,23 @@ Did you create the ECR repository [in cg-provision](https://github.com/cloud-gov
 
 ## Design choices
 
-### Why tag with the git short hash / short_ref?
+### How are images tagged?
 
-We build a variety of docker images with a variety of versioning schemes, ranging from "none" to semantic versioning. We may expand our tagging scheme in the future, but for now, the short hash — the first seven characters of the commit SHA — is a tag that works universally and functions as a reference back to the commit the image was built from, which is useful in troubleshooting.
+On merge, the `main` job pushes each image with the following tags:
+
+* `latest` — a moving pointer to the most recently built image, configured on the `image` resource.
+* `<config-digest>` — the full sha256 hex of the image **config** digest (the `sha256:` prefix stripped). The config digest is a content hash over the image's layers, environment, entrypoint, and labels, so it changes whenever any of those change. This makes the tag **guaranteed unique for any change to the image**, while identical rebuilds reuse the same tag rather than creating a redundant one. This is the tag to use when you need to pin an exact, immutable build.
+* `<short_ref>-<short_config-digest>` — a human-readable tag combining the commit short hash (the first seven characters of the commit SHA) with the first twelve characters of the config digest. It carries traceability back to the commit the image was built from while being practically unique per build. This replaces the previous standalone `<short_ref>` tag, which was not unique per build (a `base-image` bump, the weekly rebuild, or `common-dockerfiles` changes rebuild the same commit and would overwrite it).
+
+These tags are assembled by the [`generate-tags`](generate-tags.sh) task, which reads `src/.git/short_ref` and the `image/digest` file produced by the `oci-build` task, and writes a whitespace-separated list to `tags/additional_tags`. That file is fed to the `registry-image` resource's `additional_tags` param on the `put: image` step.
+
+The `staging` job pushes each image to the `staging-image` resource (the `staging` tag) using the same `generate-tags` task, but with `tag_prefix: staging-` so the unique tags become `staging-<config-digest>` and `staging-<short_ref>-<short_config-digest>`. The prefix keeps staging artifacts clearly distinguishable in the registry from the promoted tags pushed by the `main` job.
+
+The `generate-tags` task takes a `TAG_PREFIX` param that must match the resource's `tag_prefix` (empty for `main`, `staging-` for `staging`). It uses this only to validate that the final pushed tag stays within the 128-character OCI/ECR tag limit, failing closed if a tag would be too long. With the current `sha256` config digests the longest tag is `staging-` + 64 hex = 72 characters, well within the limit; the guard protects against a future change (e.g. a longer digest algorithm) silently producing an invalid tag.
+
+Note: the config digest is **not** the same value as the registry *manifest* digest (the `repo@sha256:...` that `docker pull` reports). We tag with the config digest because `oci-build` already emits it as `image/digest` with no extra registry round-trip, and it is equally unique per build for our purpose of "a new tag whenever the image changes." If a tag matching the registry manifest digest is ever required, it would need to be computed from the image tarball (or read back from the resource's push output) instead.
+
+We build a variety of docker images with a variety of versioning schemes, ranging from "none" to semantic versioning. Semantic version tags are intentionally *not* generated automatically: most of these images have no meaningful version to bump, and SemVer encodes a human judgment about API compatibility that a trigger-driven rebuild cannot make. For the subset of images whose upstream publishes real release tags, deriving a version tag from that upstream reference (via `registry-image`'s `version` / `bump_aliases`) remains a possible future opt-in.
 
 ### Why load step params from a map instead of individually?
 

container/generate-tags.sh

@@ -0,0 +1,92 @@
+#!/bin/bash
+set -euo pipefail
+
+# Generate the set of tags to apply to a built image.
+#
+# The registry-image resource's `additional_tags` param reads a single file
+# containing a whitespace-separated list of tags. This script assembles that
+# file so that every distinct image build is uniquely and traceably tagged:
+#
+#   1. <config_digest>              - the sha256 hex of the image config digest,
+#                                     guaranteed unique for any change to the image
+#   2. <short_ref>-<short_digest>   - a human-readable tag that is practically
+#                                     unique per build (commit + first 12 digest chars)
+#                                     and carries traceability back to the commit
+#
+# Inputs (Concourse resource dirs):
+#   src/.git/short_ref   - written by the git resource
+#   image/digest         - written by the oci-build task (the config digest,
+#                          e.g. "sha256:abc123..."). This is the config digest,
+#                          not the registry manifest digest, but it is equally
+#                          unique per image build for tagging purposes.
+#
+# Params:
+#   TAG_PREFIX           - optional; must match the `tag_prefix` set on the
+#                          registry-image `put` step (e.g. "staging-"). Used only
+#                          to validate the final pushed tag length; the prefix
+#                          itself is applied by the resource, not this script.
+#
+# Output:
+#   tags/additional_tags - whitespace-separated tag list
+
+SHORT_REF_FILE="src/.git/short_ref"
+DIGEST_FILE="image/digest"
+OUTPUT_FILE="tags/additional_tags"
+
+# OCI / ECR image tags are limited to 128 characters, and must match
+# [a-zA-Z0-9_][a-zA-Z0-9._-]{0,127}.
+MAX_TAG_LENGTH=128
+
+# Prefix the resource will prepend to each tag via `tag_prefix`. Defaults to
+# empty (the `main` job pushes unprefixed tags).
+TAG_PREFIX="${TAG_PREFIX:-}"
+
+if [[ ! -f "${SHORT_REF_FILE}" ]]; then
+  echo "error: ${SHORT_REF_FILE} not found" >&2
+  exit 1
+fi
+
+if [[ ! -f "${DIGEST_FILE}" ]]; then
+  echo "error: ${DIGEST_FILE} not found" >&2
+  exit 1
+fi
+
+short_ref="$(tr -d '[:space:]' < "${SHORT_REF_FILE}")"
+
+# The digest file looks like "sha256:<64 hex chars>". Strip the algorithm
+# prefix so it is usable as an OCI tag (tags may not contain ':').
+raw_digest="$(tr -d '[:space:]' < "${DIGEST_FILE}")"
+full_digest="${raw_digest#sha256:}"
+
+if [[ -z "${short_ref}" ]]; then
+  echo "error: short_ref is empty" >&2
+  exit 1
+fi
+
+if [[ -z "${full_digest}" ]]; then
+  echo "error: image digest is empty" >&2
+  exit 1
+fi
+
+# First 12 hex characters, matching the length Docker shows for short digests.
+short_digest="${full_digest:0:12}"
+readable_tag="${short_ref}-${short_digest}"
+
+# Validate that every tag, once the resource applies TAG_PREFIX, stays within
+# the OCI/ECR 128-character limit. Fail closed rather than let the push fail
+# opaquely at the registry.
+for tag in "${full_digest}" "${readable_tag}"; do
+  effective_tag="${TAG_PREFIX}${tag}"
+  tag_length="${#effective_tag}"
+  if (( tag_length > MAX_TAG_LENGTH )); then
+    echo "error: tag '${effective_tag}' is ${tag_length} characters, exceeding the ${MAX_TAG_LENGTH}-character OCI/ECR limit" >&2
+    exit 1
+  fi
+done
+
+mkdir -p tags
+printf '%s %s' "${full_digest}" "${readable_tag}" > "${OUTPUT_FILE}"
+
+echo "Generated image tags (tag_prefix='${TAG_PREFIX}'):"
+echo "  config digest:  ${TAG_PREFIX}${full_digest}"
+echo "  readable:       ${TAG_PREFIX}${readable_tag}"

container/generate-tags.yml

@@ -0,0 +1,28 @@
+---
+platform: linux
+
+image_resource:
+  type: registry-image
+  source:
+    aws_access_key_id: ((ecr_aws_key))
+    aws_secret_access_key: ((ecr_aws_secret))
+    repository: general-task
+    aws_region: us-gov-west-1
+    tag: latest
+
+inputs:
+- name: common-pipelines
+- name: src
+- name: image
+
+outputs:
+- name: tags
+
+params:
+  # Must match the `tag_prefix` on the registry-image `put` step so the tag
+  # length guard validates the actual pushed tag. Defaults to empty (the main
+  # job pushes unprefixed tags); the staging job overrides this to "staging-".
+  TAG_PREFIX: ""
+
+run:
+  path: common-pipelines/container/generate-tags.sh

container/pipeline-base.yml

@@ -139,6 +139,11 @@ jobs:
       - task: cve-check
         file: common-pipelines/container/cve-check.yml
 
+      - task: generate-tags
+        file: common-pipelines/container/generate-tags.yml
+        params:
+          TAG_PREFIX: staging-
+
       - in_parallel:
           - task: software-inventory
             file: common-pipelines/container/software-inventory.yml
@@ -148,9 +153,12 @@ jobs:
             inputs:
               - image
               - src
+              - tags
             no_get: true
             params:
               image: image/image.tar
+              additional_tags: tags/additional_tags
+              tag_prefix: staging-
 
     on_failure:
       put: slack
@@ -254,6 +262,9 @@ jobs:
       - task: cve-check
         file: common-pipelines/container/cve-check.yml
 
+      - task: generate-tags
+        file: common-pipelines/container/generate-tags.yml
+
       - in_parallel:
           - task: import-scan
             file: common-pipelines/container/import-scan.yml
@@ -278,10 +289,11 @@ jobs:
             inputs:
               - image
               - src
+              - tags
             no_get: true
             params:
               image: image/image.tar
-              additional_tags: src/.git/short_ref
+              additional_tags: tags/additional_tags
 
     on_failure:
       put: slack

container/pipeline-external.yml

@@ -67,6 +67,9 @@ jobs:
       - task: cve-check
         file: common-pipelines/container/cve-check.yml
 
+      - task: generate-tags
+        file: common-pipelines/container/generate-tags.yml
+
       - in_parallel:
           - task: import-scan
             file: common-pipelines/container/import-scan.yml
@@ -91,10 +94,11 @@ jobs:
             inputs:
               - image
               - src
+              - tags
             no_get: true
             params:
               image: image/image.tar
-              additional_tags: src/.git/short_ref
+              additional_tags: tags/additional_tags
 
     on_failure:
       put: slack
@@ -208,6 +212,11 @@ jobs:
       - task: cve-check
         file: common-pipelines/container/cve-check.yml
 
+      - task: generate-tags
+        file: common-pipelines/container/generate-tags.yml
+        params:
+          TAG_PREFIX: staging-
+
       - in_parallel:
           - task: software-inventory
             file: common-pipelines/container/software-inventory.yml
@@ -217,9 +226,12 @@ jobs:
             inputs:
               - image
               - src
+              - tags
             no_get: true
             params:
               image: image/image.tar
+              additional_tags: tags/additional_tags
+              tag_prefix: staging-
 
     on_failure:
       put: slack

container/pipeline-internal.yml

@@ -181,6 +181,11 @@ jobs:
       - task: cve-check
         file: common-pipelines/container/cve-check.yml
 
+      - task: generate-tags
+        file: common-pipelines/container/generate-tags.yml
+        params:
+          TAG_PREFIX: staging-
+
       - in_parallel:
           - task: software-inventory
             file: common-pipelines/container/software-inventory.yml
@@ -190,9 +195,12 @@ jobs:
             inputs:
               - image
               - src
+              - tags
             no_get: true
             params:
               image: image/image.tar
+              additional_tags: tags/additional_tags
+              tag_prefix: staging-
 
     on_failure:
       put: slack
@@ -312,6 +320,9 @@ jobs:
       - task: cve-check
         file: common-pipelines/container/cve-check.yml
 
+      - task: generate-tags
+        file: common-pipelines/container/generate-tags.yml
+
       - in_parallel:
           - task: import-scan
             file: common-pipelines/container/import-scan.yml
@@ -336,10 +347,11 @@ jobs:
             inputs:
               - image
               - src
+              - tags
             no_get: true
             params:
               image: image/image.tar
-              additional_tags: src/.git/short_ref
+              additional_tags: tags/additional_tags
 
     on_failure:
       put: slack

container/pipeline-pages.yml

@@ -122,6 +122,11 @@ jobs:
       - task: cve-check
         file: common-pipelines/container/cve-check.yml
 
+      - task: generate-tags
+        file: common-pipelines/container/generate-tags.yml
+        params:
+          TAG_PREFIX: staging-
+
       - in_parallel:
           - task: software-inventory
             file: common-pipelines/container/software-inventory.yml
@@ -131,9 +136,12 @@ jobs:
             inputs:
               - image
               - src
+              - tags
             no_get: true
             params:
               image: image/image.tar
+              additional_tags: tags/additional_tags
+              tag_prefix: staging-
 
     on_failure:
       put: slack
@@ -251,6 +259,9 @@ jobs:
       - task: cve-check
         file: common-pipelines/container/cve-check.yml
 
+      - task: generate-tags
+        file: common-pipelines/container/generate-tags.yml
+
       - in_parallel:
           - task: import-scan
             file: common-pipelines/container/import-scan.yml
@@ -275,10 +286,11 @@ jobs:
             inputs:
               - image
               - src
+              - tags
             no_get: true
             params:
               image: image/image.tar
-              additional_tags: src/.git/short_ref
+              additional_tags: tags/additional_tags
 
     on_failure:
       put: slack