Skip to content

[releng master] gromit: sync templates #484

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
probelabs wants to merge 1 commit into
base: master
Choose a base branch
from
Open

[releng master] gromit: sync templates #484

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions .github/workflows/drift-check.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
# Generated by: gromit policy
# This file is managed by gromit, do not edit by hand.
# The check logic lives in TykTechnologies/github-actions.
name: Drift Check
on:
pull_request:
jobs:
drift:
uses: TykTechnologies/github-actions/.github/workflows/drift-check.yml@production

Check failure on line 9 in .github/workflows/drift-check.yml

View check run for this annotation

probelabs / Visor: security

security Issue 

Reusable workflows and actions are not pinned to a commit SHA, creating a supply chain risk. The `uses` clauses reference a mutable tag '@production'. If the 'production' tag is compromised or updated with malicious code in the source repository, this workflow will execute it. This issue is present for `drift-check.yml` on line 9, and in `.github/workflows/release.yml` for `distro-matrix` (line 302) and `upgrade-tests` (line 396).
Raw output 
Pin all reusable workflows and actions to a specific, immutable commit SHA. Find the latest commit SHA for the 'production' branch/tag in the `TykTechnologies/github-actions` repository and replace `@production` with the full SHA to ensure the executed code is immutable and auditable.

Check failure on line 9 in .github/workflows/drift-check.yml

View check run for this annotation

SonarQubeCloud / SonarCloud Code Analysis

Use full commit SHA hash for this dependency. 


See more on https://sonarcloud.io/project/issues?id=TykTechnologies_tyk-identity-broker&issues=AZ9-qibXIIpXeaai_GJ6&open=AZ9-qibXIIpXeaai_GJ6&pullRequest=484
229 changes: 114 additions & 115 deletions .github/workflows/release.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -117,6 +117,40 @@
key: ${{ runner.os }}-go-${{ hashFiles('**/go.sum') }}
restore-keys: |
${{ runner.os }}-go-${{ hashFiles('**/go.sum') }}
- name: Validate Go version
run: |
GO_VERSION_STR=$(docker run --rm tykio/golang-cross:${{ matrix.golang_cross }} go version 2>&1 || true)
if [[ ! "$GO_VERSION_STR" =~ go[0-9]+ ]]; then
echo "::warning::Could not determine Go version from container: $GO_VERSION_STR"
exit 1
fi
GO_VERSION=$(echo "$GO_VERSION_STR" | awk '{print $3}' | sed 's/^go//')
echo "Go version in container: $GO_VERSION"
MINOR_VERSION=$(echo "$GO_VERSION" | cut -d. -f1,2)
echo "Minor version: $MINOR_VERSION"

# Fetch latest patches
LATEST_PATCH_STR=$(curl -s 'https://go.dev/dl/?mode=json&include=all' || true)
if [ -z "$LATEST_PATCH_STR" ]; then
echo "::warning::Could not fetch Go releases from API. Skipping validation."
exit 0

Check warning on line 136 in .github/workflows/release.yml

View check run for this annotation

probelabs / Visor: performance

performance Issue 

The "Validate Go version" step introduces a network request to `go.dev` during the build process to fetch the list of all Go releases. This adds a dependency on an external service, which can increase job execution time due to network latency and introduce a potential point of failure if the service is slow or unavailable.
Raw output 
To improve performance and reliability, consider caching the response from the Go releases API. GitHub Actions provides caching mechanisms that could store the result for a period (e.g., daily), reducing the number of live API calls across workflow runs. This would make the job faster and more resilient to temporary network or API issues.
fi
LATEST_PATCH=$(echo "$LATEST_PATCH_STR" | jq -r --arg minor "go${MINOR_VERSION}" '.[] | select(.stable == true and (.version | startswith($minor + ".") or . == $minor)) | .version' | head -n 1 | sed 's/^go//')

if [ -n "$LATEST_PATCH" ]; then
echo "Latest patch version for Go $MINOR_VERSION is $LATEST_PATCH"
if [ "$GO_VERSION" != "$LATEST_PATCH" ]; then
echo "::error::Go version validation failed!"
echo "Used version: $GO_VERSION (in tykio/golang-cross:${{ matrix.golang_cross }})"
echo "Latest patch: $LATEST_PATCH"
echo "Please update the buildenv Go version to the latest patch version."
exit 1
else
echo "Go version is up to date: $GO_VERSION"
fi
else
echo "::warning::Could not determine latest patch version for Go $MINOR_VERSION from API. Skipping validation."
fi
- name: Build
env:
NFPM_PASSPHRASE: ${{ secrets.SIGNING_KEY_PASSPHRASE }}
Expand All @@ -127,7 +161,7 @@
ci/bin/unlock-agent.sh
git config --global url."https://x-access-token:${{ steps.app-token.outputs.token }}@github.com/".insteadOf "https://github.com/"
git config --global --add safe.directory /go/src/github.com/TykTechnologies/tyk-identity-broker
goreleaser release --clean -f ${{ matrix.goreleaser }} ${{ !startsWith(github.ref, 'refs/tags/') && ' --snapshot --skip=sign,docker' || '--skip=docker' }}' | tee /tmp/build.sh
goreleaser release --timeout 120m --clean -f ${{ matrix.goreleaser }} ${{ !startsWith(github.ref, 'refs/tags/') && ' --snapshot --skip=sign,docker' || '--skip=docker' }}' | tee /tmp/build.sh
chmod +x /tmp/build.sh
docker run --rm --privileged -e GITHUB_TOKEN=${{ github.token }} \
-e GOPRIVATE=github.com/TykTechnologies \
Expand All @@ -150,47 +184,76 @@
tykio/golang-cross:${{ matrix.golang_cross }} /tmp/build.sh
- name: Docker metadata for std CI
id: ci_metadata_std
if: ${{ matrix.golang_cross == '1.25-bookworm' }}
uses: docker/metadata-action@c299e40c65443455700f0fdfc63efafe5b349051 # v5
with:
images: |
${{ steps.ecr.outputs.registry }}/tyk-identity-broker
flavor: |
latest=false
tags: |
type=ref,event=branch
type=ref,event=pr
type=sha,format=long
type=semver,pattern={{major}},prefix=v
type=semver,pattern={{major}}.{{minor}},prefix=v
type=semver,pattern={{version}},prefix=v
- name: push std image to CI
if: ${{ matrix.golang_cross == '1.25-bookworm' }}
uses: docker/build-push-action@10e90e3645eae34f1e60eeb005ba3a3d33f178e8 # v6
with:
context: "dist"
platforms: ${{ github.event_name == 'pull_request' && 'linux/amd64' || 'linux/amd64,linux/arm64,linux/s390x' }}
file: ci/Dockerfile.distroless
provenance: mode=max
outputs: type=image,push=true,compression=gzip,force-compression=true,oci-mediatypes=true
cache-from: type=gha
cache-to: type=gha,mode=max
tags: ${{ steps.ci_metadata_std.outputs.tags }}
labels: ${{ steps.ci_metadata_std.outputs.labels }}
build-args: |
BUILD_PACKAGE_NAME=tyk-identity-broker
- name: Verify compression format for std CI image
if: ${{ matrix.golang_cross == '1.25-bookworm' }}
run: |
# Resolve the first tag for verification
TAG=$(echo "${{ steps.ci_metadata_std.outputs.tags }}" | tr ',' '\n' | head -n1)

Check warning on line 220 in .github/workflows/release.yml

View check run for this annotation

probelabs / Visor: architecture

architecture Issue 

The shell function `verify_manifest` and its surrounding logic for checking Docker layer compression is duplicated. The exact same script block is used for the `std` image (lines 187-220) and the `distroless` image (lines 224-257).
Raw output 
To adhere to the DRY (Don't Repeat Yourself) principle and improve maintainability, this duplicated script logic should be extracted into a reusable component. A composite GitHub Action within the local repository (e.g., in a `.github/actions/` directory) would be an ideal solution. The workflow could then call this action twice with the respective image tags as input, centralizing the verification logic.
echo "Inspecting layer compression for tag: $TAG"

Check warning on line 221 in .github/workflows/release.yml

View check run for this annotation

probelabs / Visor: quality

architecture Issue 

The shell script logic for verifying Docker image layer compression is duplicated. The same script block appears in the `build-distroless` job (lines 187-221) and the `build-std` job (lines 234-268). This violates the DRY principle, making the workflow harder to maintain as any changes to the logic must be synchronized in both places.
Raw output 
To improve maintainability, extract the duplicated script into a reusable composite action within the `.github/actions/` directory. This action can then be called from both jobs, passing the image tag as an input parameter.

docker buildx imagetools inspect --raw -- "$TAG" > manifest.json
if grep -q "tar+zstd" manifest.json; then
echo "::error::Image contains zstd-compressed layers! This is incompatible with legacy registries."
cat manifest.json | grep mediaType
exit 1

# Extract child digests (excluding attestation manifests) if it's a multi-arch manifest list / index
DIGESTS=$(jq -r '.manifests[] | select((.platform.architecture // "") != "unknown" and (.annotations["vnd.docker.reference.type"] // "") != "attestation-manifest") | .digest' manifest.json 2>/dev/null || true)

verify_manifest() {
local file=$1
# Check if any layer does NOT end in gzip/tar+gzip/tar.gzip
local invalid_layers
invalid_layers=$(jq -r '.layers[].mediaType' "$file" | grep -v 'tar+gzip' | grep -v 'tar.gzip' || true)
if [ -n "$invalid_layers" ]; then
echo "::error::Found layers that are not gzip-compressed:"
echo "$invalid_layers"
exit 1
fi
# Also verify we found at least one layer
local layer_count
layer_count=$(jq '.layers | length' "$file" 2>/dev/null || echo 0)
if [ "$layer_count" -eq 0 ]; then
echo "::error::No layers found in manifest!"
exit 1
fi
}

if [ -n "$DIGESTS" ]; then
echo "Multi-arch image list detected. Checking each platform manifest..."
for digest in $DIGESTS; do
echo "Inspecting child platform manifest: $digest"
docker buildx imagetools inspect --raw -- "$TAG@$digest" > child_manifest.json
verify_manifest child_manifest.json
done
else
echo "Single-arch manifest detected. Checking manifest..."
verify_manifest manifest.json
fi
echo "Success: All layers are uniformly compressed using gzip."
- name: Docker metadata for std tag push
Expand Down Expand Up @@ -234,10 +297,39 @@
echo "Inspecting layer compression for tag: $TAG"

docker buildx imagetools inspect --raw -- "$TAG" > manifest.json
if grep -q "tar+zstd" manifest.json; then
echo "::error::Image contains zstd-compressed layers! This is incompatible with legacy registries."
cat manifest.json | grep mediaType
exit 1

# Extract child digests (excluding attestation manifests) if it's a multi-arch manifest list / index
DIGESTS=$(jq -r '.manifests[] | select((.platform.architecture // "") != "unknown" and (.annotations["vnd.docker.reference.type"] // "") != "attestation-manifest") | .digest' manifest.json 2>/dev/null || true)

verify_manifest() {
local file=$1
# Check if any layer does NOT end in gzip/tar+gzip/tar.gzip
local invalid_layers
invalid_layers=$(jq -r '.layers[].mediaType' "$file" | grep -v 'tar+gzip' | grep -v 'tar.gzip' || true)
if [ -n "$invalid_layers" ]; then
echo "::error::Found layers that are not gzip-compressed:"
echo "$invalid_layers"
exit 1
fi
# Also verify we found at least one layer
local layer_count
layer_count=$(jq '.layers | length' "$file" 2>/dev/null || echo 0)
if [ "$layer_count" -eq 0 ]; then
echo "::error::No layers found in manifest!"
exit 1
fi
}

if [ -n "$DIGESTS" ]; then
echo "Multi-arch image list detected. Checking each platform manifest..."
for digest in $DIGESTS; do
echo "Inspecting child platform manifest: $digest"
docker buildx imagetools inspect --raw -- "$TAG@$digest" > child_manifest.json
verify_manifest child_manifest.json
done
else
echo "Single-arch manifest detected. Checking manifest..."
verify_manifest manifest.json
fi
echo "Success: All layers are uniformly compressed using gzip."
- name: save deb
Expand Down Expand Up @@ -301,113 +393,20 @@
outputs:
deb: ${{ steps.params.outputs.deb }}
rpm: ${{ steps.params.outputs.rpm }}
rpm_amd64: ${{ steps.params.outputs.rpm_amd64 }}
steps:
- name: set params
- name: Set distro matrix
uses: TykTechnologies/github-actions/.github/actions/tests/distro-matrix@production
id: params
shell: bash
env:
# startsWith covers pull_request_target too
BASE_REF: ${{startsWith(github.event_name, 'pull_request') && github.base_ref || github.ref_name}}
run: |
set -eo pipefail
curl -s --retry 5 --retry-delay 10 --fail-with-body "http://tui.internal.dev.tyk.technology/v2/$VARIATION/tyk-identity-broker/$BASE_REF/${{ github.event_name}}/api/Distros.gho" | tee -a "$GITHUB_OUTPUT"
if ! [[ $VARIATION =~ prod ]];then
echo "::warning file=.github/workflows/release.yml,line=24,col=1,endColumn=8::Using test variation"
fi
upgrade-deb:
services:
httpbin.org:
image: kennethreitz/httpbin
runs-on: ${{ vars.DEFAULT_RUNNER }}
needs:
- test-controller-distros
strategy:
fail-fast: true
matrix:
arch:
- amd64
- arm64
distro: ${{ fromJson(needs.test-controller-distros.outputs.deb) }}
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
with:
fetch-depth: 1
sparse-checkout: ci
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4
with:
name: deb
- uses: docker/setup-qemu-action@c7c53464625b32c7a7e944ae62b3e17d2b600130 # v3
- uses: docker/setup-buildx-action@8d2750c68a42422c14e847fe6c8ac0403b4cbd6f # v3
- name: generate dockerfile
run: |
echo 'FROM ${{ matrix.distro }}
ARG TARGETARCH
COPY tyk-identity-broker*_${TARGETARCH}.deb /tyk-identity-broker.deb
RUN apt-get update && apt-get install -y curl
# TODO(security): curl|bash - consider fetching script and verifying checksum before execution
RUN curl -fsSL https://packagecloud.io/install/repositories/tyk/tyk-identity-broker/script.deb.sh | bash || echo "Repository setup failed, but continuing" # SECURITY: accepted risk, see TODO above
RUN apt-get install -y tyk-identity-broker=1.1.0 || echo "Previous version not found, testing fresh install"
RUN dpkg -i /tyk-identity-broker.deb

' | tee Dockerfile
- name: install on ${{ matrix.distro }}
uses: docker/build-push-action@10e90e3645eae34f1e60eeb005ba3a3d33f178e8 # v6
with:
context: "."
platforms: linux/${{ matrix.arch }}
build-args: |
RHELARCH=${{ startsWith(matrix.arch, 'arm64') && 'aarch64' || 'x86_64' }}
cache-from: type=gha
cache-to: type=gha,mode=max
file: Dockerfile
push: false
upgrade-rpm:
services:
httpbin.org:
image: kennethreitz/httpbin
runs-on: ${{ vars.DEFAULT_RUNNER }}
upgrade-tests:
needs:
- test-controller-distros
strategy:
fail-fast: true
matrix:
arch:
- amd64
- arm64
distro: ${{ fromJson(needs.test-controller-distros.outputs.rpm) }}
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
with:
fetch-depth: 1
sparse-checkout: ci
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4
with:
name: rpm
- uses: docker/setup-qemu-action@c7c53464625b32c7a7e944ae62b3e17d2b600130 # v3
- uses: docker/setup-buildx-action@8d2750c68a42422c14e847fe6c8ac0403b4cbd6f # v3
- name: generate dockerfile
run: |
echo 'FROM ${{ matrix.distro }}
ARG RHELARCH
COPY tyk-identity-broker*.${RHELARCH}.rpm /tyk-identity-broker.rpm
RUN command -v curl || yum install -y curl
RUN command -v useradd || yum install -y shadow-utils
# TODO(security): curl|bash - consider fetching script and verifying checksum before execution
RUN curl -fsSL https://packagecloud.io/install/repositories/tyk/tyk-identity-broker/script.rpm.sh | bash || echo "Repository setup failed, but continuing" # SECURITY: accepted risk, see TODO above
RUN yum install -y tyk-identity-broker-1.1.0-1 || echo "Previous version not found, testing fresh install"
RUN curl https://keyserver.tyk.io/tyk.io.rpm.signing.key.2020 -o tyk-identity-broker.key && rpm --import tyk-identity-broker.key
RUN rpm --checksig /tyk-identity-broker.rpm
RUN rpm -Uvh --force /tyk-identity-broker.rpm

' | tee Dockerfile
- name: install on ${{ matrix.distro }}
uses: docker/build-push-action@10e90e3645eae34f1e60eeb005ba3a3d33f178e8 # v6
with:
context: "."
platforms: linux/${{ matrix.arch }}
build-args: |
RHELARCH=${{ startsWith(matrix.arch, 'arm64') && 'aarch64' || 'x86_64' }}
cache-from: type=gha
cache-to: type=gha,mode=max
file: Dockerfile
push: false
uses: TykTechnologies/github-actions/.github/workflows/upgrade-tests.yml@production

Check failure on line 404 in .github/workflows/release.yml

View check run for this annotation

SonarQubeCloud / SonarCloud Code Analysis

Use full commit SHA hash for this dependency. 


See more on https://sonarcloud.io/project/issues?id=TykTechnologies_tyk-identity-broker&issues=AZ9L4QLjOFFB0Wg3DHz2&open=AZ9L4QLjOFFB0Wg3DHz2&pullRequest=484
with:
deb: ${{ needs.test-controller-distros.outputs.deb }}
rpm: ${{ needs.test-controller-distros.outputs.rpm }}
rpm_amd64: ${{ needs.test-controller-distros.outputs.rpm_amd64 }}
package_name: tyk-identity-broker
upgrade_repo: tyk-identity-broker
runs_on: ${{ vars.DEFAULT_RUNNER }}
run_gateway_tests: false
4 changes: 1 addition & 3 deletions ci/Dockerfile.distroless
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,10 +11,8 @@ ENV DEBIAN_FRONTEND=noninteractive
# The _ after the pkg name is to match tyk-gateway strictly and not tyk-gateway-fips (for example)
COPY ${BUILD_PACKAGE_NAME}_*${TARGETARCH}.deb /
ARG NONROOT_CHOWN=false
RUN dpkg -i /${BUILD_PACKAGE_NAME}_*${TARGETARCH}.deb && rm /*.deb \
RUN dpkg -i /"${BUILD_PACKAGE_NAME}"_*"${TARGETARCH}".deb && rm /*.deb \
&& chmod -R a+rX /opt/tyk-identity-broker/ \
&& mkdir -p /opt/tyk-identity-broker/middleware/bundles \
&& chmod a+rwX /opt/tyk-identity-broker/middleware /opt/tyk-identity-broker/middleware/bundles \
&& if [ "$NONROOT_CHOWN" = "true" ]; then chown -R 65532:65532 /opt/tyk-identity-broker/; fi

FROM ${BASE_IMAGE}
Expand Down
2 changes: 1 addition & 1 deletion ci/Dockerfile.std
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ RUN rm -fv /usr/bin/passwd /usr/sbin/adduser || true

# Comment this to test in dev
COPY dist/${BUILD_PACKAGE_NAME}_*_${TARGETARCH}.deb /
RUN dpkg -i /${BUILD_PACKAGE_NAME}_*_${TARGETARCH}.deb && find / -maxdepth 1 -name "*.deb" -delete
RUN dpkg -i /"${BUILD_PACKAGE_NAME}"_*_"${TARGETARCH}".deb && find / -maxdepth 1 -name "*.deb" -delete

# Clean up caches, unwanted .a and .o files
RUN rm -rf /root/.cache \
Expand Down
Loading