Skip to content

[TT-16946][TT-16950][TT-16951] feat: use FIPS base image for EE and FIPS Docker builds#444

Open
buger wants to merge 36 commits intomasterfrom
feat/fips-base-image
Open

[TT-16946][TT-16950][TT-16951] feat: use FIPS base image for EE and FIPS Docker builds#444
buger wants to merge 36 commits intomasterfrom
feat/fips-base-image

Conversation

@buger
Copy link
Copy Markdown
Member

@buger buger commented Apr 7, 2026

$(gh pr view 444 --repo TykTechnologies/gromit --json body -q .body)

Related Tickets:

Add per-build DockerBaseImage field to override the default distroless
base image. When set, the Dockerfile.distroless template uses the
specified image for amd64/arm64 while keeping the default distroless
image as fallback for s390x (where no FIPS image is available).

- Add DockerBaseImage field to the build struct
- Update Dockerfile.distroless template with BASE_IMAGE/BASE_IMAGE_S390X
  ARGs and per-architecture stage selection
- Update goreleaser.gotmpl to pass BASE_IMAGE build-arg when
  DockerBaseImage is configured
- Set dockerbaseimage: tykio/dhi-busybox:1.37-fips on all EE and FIPS
  builds across master, release-5.8, and release-5.10 branches

Open source (std) builds are unaffected — they continue using the
default gcr.io/distroless base image.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@buger buger requested a review from a team as a code owner April 7, 2026 16:08
buger and others added 9 commits April 7, 2026 19:41
- Add FIPS build configuration for tyk master branch (was only on
  release-5.8)
- Add s390x architecture to FIPS builds with skipdocker: true, so
  s390x gets .deb/.rpm packages but not Docker images
- Fix GetDockerPlatforms() to respect SkipDocker flag, excluding
  architectures marked skipdocker from Docker platform list
- Apply same s390x+skipdocker pattern to release-5.8 FIPS build

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- tyk-analytics: add FIPS build to master branch, add dockerbaseimage
  and s390x+skipdocker to release-5.8 FIPS build
- tyk-pump: add dockerbaseimage and s390x+skipdocker to FIPS build
- tyk-sink: add dockerbaseimage and s390x+skipdocker to FIPS build

All FIPS builds across the stack now use tykio/dhi-busybox:1.37-fips
as Docker base image for amd64/arm64, with s390x packages built but
Docker images excluded (no FIPS base available for s390x).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Move EE and FIPS build definitions from per-branch to repo level for
tyk and tyk-analytics, leveraging gromit's build inheritance so all
branches (including new ones) automatically get these builds.

- tyk: EE and FIPS builds moved to repo level, branch configs now
  only specify buildenv and features
- tyk-analytics: FIPS build moved to repo level, same simplification
- Added release-5.11 and release-5.12 branches for both tyk and
  tyk-analytics (they inherit EE/FIPS from repo level automatically)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add a generic Feature field to the build struct. When a build has
Feature set, it is only included in branches where that feature is
present in the features list. This allows FIPS builds to be defined
once at repo level and selectively enabled per branch by adding
"fips" to the branch's features.

- Add Feature field to build struct
- Add filterBuildsByFeature() to exclude builds whose required
  feature is not present in the branch
- Set feature: fips on all FIPS build definitions (tyk, tyk-analytics,
  tyk-pump, tyk-sink)
- Add fips to features list on all branches that should have FIPS

To enable FIPS on a new branch, simply add "fips" to its features:
  release-5.13:
    features:
      - release-test
      - distroless
      - fips

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Go 1.24+ includes native FIPS 140-3 support via the built-in Go
Cryptographic Module, replacing the external BoringCrypto C library.

Changes across all FIPS builds (tyk, tyk-analytics, tyk-pump, tyk-sink):
- Replace GOEXPERIMENT=boringcrypto with GOFIPS140=v1.0.0
- Remove boringcrypto from build tags (keep fips tag for source
  conditional compilation)
- Remove skipdocker from s390x — native Go FIPS supports all
  platforms, so s390x now gets full Docker image support
- Update descriptions from "boringssl" to "FIPS 140-3 compliant
  cryptography"

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The FIPS base Docker image (tykio/dhi-busybox:1.37-fips) does not
have an s390x variant. While Go 1.24+ native FIPS supports s390x
for binary compilation, Docker images still cannot be built for
s390x without the base image. Restore skipdocker: true on s390x
for all FIPS builds.

s390x FIPS packages (.deb/.rpm) are still built and published.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
All Tyk repos (tyk, tyk-analytics, tyk-pump, tyk-sink, portal,
ai-studio, tyk-identity-broker) have already migrated to Go 1.25
in their go.mod files. Update the gromit buildenv to match:

- cgo-services group: 1.24-bullseye → 1.25-bullseye
- pgo-services group: 1.24-bookworm → 1.25-bookworm
- Per-branch overrides: updated accordingly

midsommar remains on 1.22-bullseye (unchanged).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
When DockerBaseImage is set on a build that includes platforms where
the custom base image is unavailable (e.g., s390x), the Docker build
is split into two steps:
1. Platforms with base image support (amd64/arm64) use BASE_IMAGE
2. Fallback platforms (s390x) use the default distroless base

This allows EE builds to use the FIPS base image for amd64/arm64
while still producing s390x Docker images with the standard base.

Changes:
- Add SkipBaseImage field to arch config (marks platforms where the
  custom base image is unavailable)
- Add GetBaseImagePlatforms(), GetFallbackPlatforms(),
  HasFallbackPlatforms() helper functions
- Update goreleaser.gotmpl to generate split Docker push steps
- Simplify Dockerfile.distroless to single BASE_IMAGE ARG (no more
  per-arch FROM stages)
- Set skipbaseimage: true on EE s390x

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
EE and FIPS Docker images are amd64/arm64 only (hardened base image).
s390x gets packages but no Docker images for both EE and FIPS.
OSS (std) keeps all three archs with default distroless base.

- Set skipdocker: true on EE s390x (same as FIPS s390x)
- Remove SkipBaseImage field and split-build helpers (not needed)
- Simplify Dockerfile.distroless to single BASE_IMAGE ARG
- Revert goreleaser.gotmpl to simple single-step Docker builds

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
buger and others added 4 commits April 8, 2026 18:25
- Add pull_request types filter (opened, synchronize, reopened,
  ready_for_review, labeled) to release.yml template — was added
  manually in tyk repo but never synced to gromit
- Add UI assets build fallback for tyk-analytics: continue-on-error
  on S3 fetch + Node.js fallback build steps — was added manually
  in tyk-analytics repo but never synced to gromit

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The types filter (opened, synchronize, reopened, ready_for_review,
labeled) was added manually in the tyk repo but may not be
appropriate for all repos. The default (opened, synchronize,
reopened) is safer. Keep the UI fallback fix for tyk-analytics.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
After pushing EE and FIPS Docker images (on tag releases), fetch the
VEX data from Docker Hardened Images advisories, rewrite product
references to tykio image names, and attach as OpenVEX attestation
via docker scout. This allows customers scanning with trivy --vex oci
or Docker Scout to automatically suppress CVEs that DHI has marked as
not_affected or under_investigation.

Also reverts the Dockerfile cleanup approach — the DHI base image
must not be modified to preserve its FIPS certification, SBOM, and
provenance attestations.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Switch from docker scout to cosign for attaching VEX attestations
to EE/FIPS images. Cosign produces DSSE-wrapped attestations that
trivy can discover via --vex oci flag using the OCI referrers API.

- Add sigstore/cosign-installer step
- Use cosign attest --type openvex for VEX attachment
- Cosign uses GitHub Actions OIDC for keyless signing (id-token:
  write permission already present for AWS OIDC)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Upgrade all distroless base images to Debian 13 (trixie):
- cgo-services group: base-debian12 → base-debian13
- tyk repo: base-debian12:latest → base-debian13:latest
- pgo-services group: static-debian12 → static-debian13

Debian 13 eliminates the glibc CVE-2026-0861 present in Debian 12
and aligns the OSS images with the EE/FIPS images which already
use a Debian 13 based Docker Hardened Image.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
buger and others added 2 commits April 9, 2026 12:35
For patch releases on 5.12, keep EE Docker images on the existing
distroless base to minimize change. Only FIPS images get the DHI
hardened base on 5.12. Master and future releases keep DHI for both
EE and FIPS.

Branch-level override uses dockerbaseimage: distroless as a
self-documenting sentinel value — the template treats it as
"use default distroless base."

Also fixes a pre-existing bug in mergeBuilds where repo-level build
objects were mutated across branches (shallow copy via maps.Copy).
Now deep copies before merging to prevent cross-branch contamination.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Remove the trivy/jq/cosign VEX pipeline step. Docker Scout
automatically applies VEX from the DHI base image through the
provenance chain (images are built with provenance: mode=max).

Customers using Docker Scout see VEX applied automatically.
Customers using trivy can configure the DHI VEX repository.

Also removes cosign-installer since it's no longer needed.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@probelabs probelabs bot changed the title feat: use FIPS base image for EE and FIPS Docker builds [TT-16946][TT-16950][TT-16951] feat: use FIPS base image for EE and FIPS Docker builds Apr 9, 2026
buger and others added 5 commits April 9, 2026 14:07
Use docker scout vex get to extract VEX from the DHI base image,
then cosign attest --type openvex to attach it as an OCI referrer.
This makes VEX discoverable by trivy --vex oci without any
customer-side configuration.

Pipeline: docker scout vex get → cosign attest → OCI referrer
Customer: trivy image --vex oci <image> (zero config)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add FIPS build variant for the developer portal with DHI busybox
base image, GOFIPS140=v1.0.0, and fips feature flag on master.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
When a branch defines Archs for a build, clear the repo-level Archs
before merging so they replace rather than append. This prevents
duplicate build IDs (e.g., ee-arm64 appearing twice) which causes
goreleaser to fail.

Also adds cosign-installer and docker scout VEX attachment step,
and adds portal FIPS support.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Incorporate changes from gromit#447 — adds debian:trixie to the
prod-variations upgrade test matrix and fixes missing trailing newline.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add FIPS build variant with DHI busybox base image, GOFIPS140=v1.0.0,
and fips feature flag on master.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The std build was missing -trimpath at repo level in gromit config,
causing the generated goreleaser.yml to drop it. Current tyk master
has -trimpath on all builds (added in PR #5115 for plugin compatibility).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
buger and others added 2 commits April 10, 2026 09:27
Adds dependency-guard as a reusable workflow job that runs only on
pull requests (if: github.event_name == 'pull_request'). Goreleaser
needs dep-guard, but since GitHub Actions treats skipped jobs as
satisfied, tag/push/schedule builds proceed without blocking.

- PRs: dep-guard runs, must pass before goreleaser starts
- Tags/pushes/schedule: dep-guard is skipped, goreleaser proceeds

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Pin Docker Scout CLI install to specific version instead of
  piping from main branch
- Change tyk distroless base from base-debian13:latest to
  base-debian13:nonroot for consistent nonroot default

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
buger and others added 2 commits April 14, 2026 17:51
DockerHub login was gated on tag pushes only, but the FIPS Docker
build needs to pull tykio/dhi-busybox:1.37-fips on PR builds too.
Without auth, DockerHub rate limits cause pull failures on CI runners.

The push steps retain their own tag conditions, so images are only
pushed on releases.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Remove FIPS build and feature from TIB for now.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
buger added a commit to TykTechnologies/tyk that referenced this pull request Apr 14, 2026
)

## Summary
- Use tykio/dhi-busybox:1.37-fips for FIPS Docker images
- Add FIPS builds using GOFIPS140=v1.0.0
- Update distroless to Debian 13
- EE stays on distroless (patch release)

### Gromit PR
TykTechnologies/gromit#444

Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
buger and others added 3 commits April 15, 2026 10:17
The goreleaser job depends on dep-guard, but dep-guard only runs on
PRs. On push/tag/schedule events, dep-guard is skipped, which caused
goreleaser to also be skipped — breaking all release builds.

Add explicit if condition that allows goreleaser to proceed when
dep-guard is skipped (non-PR events) while still blocking when
dep-guard fails (PR with unapproved deps).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
FIPS builds for tyk, tyk-analytics, and portal were using the same
ECR CI repo as std builds, causing image overwrites. Fixed:
- tyk FIPS: cirepo tyk → tyk-fips
- tyk-analytics FIPS: cirepo tyk-analytics → tyk-analytics-fips
- portal FIPS: cirepo portal → portal-fips

tyk-pump and tyk-sink already had correct separate repos.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add --chown=65532:65532 to COPY for the application directory.
Both distroless:nonroot and DHI busybox run as uid 65532 (nonroot).
Without this, the gateway cannot write to middleware/bundles for
plugin bundle downloads.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
buger and others added 6 commits April 16, 2026 18:12
For PRs targeting release branches (e.g., release-5.12.1), the resolver
now checks if the same branch exists in tyk-analytics and uses it
instead of always falling back to master via gromit-default.

Previously, any non-master BASE_REF immediately got strategy=gromit-default,
which mapped to tyk-analytics:master — causing test failures due to
dashboard/test version mismatch.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The actions/checkout step sets an AUTHORIZATION header with the limited
GITHUB_TOKEN which overrides ORG_GH_TOKEN credentials in git ls-remote
for cross-repo branch checking. Adding persist-credentials: false
prevents this interference.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
dep-guard only runs on PRs. Having goreleaser depend on it causes
GitHub Actions to transitively skip all downstream jobs on push/tag
events. Fix by making goreleaser independent and moving dep-guard
gate to the aggregator.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Corrected approach: keep goreleaser depending on dep-guard (preserves
PR gating on pull requests) but add !cancelled() + result checks to
all downstream job templates to prevent transitive skip propagation
on push/tag events.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Adding deps-reviewed label after dep-guard fails didn't trigger a
new run because pull_request only defaulted to opened/synchronize/
reopened. Add 'labeled' type so the workflow re-runs when the label
is added.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The build-dashboard-image step had golang-cross:1.24-bookworm
hardcoded instead of using the branch's buildenv. This caused
the dashboard to be built with Go 1.24 while everything else
uses 1.25.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant