Skip to content
Merged
Changes from 20 commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
f6a9d6f
chore: add binary caching
agarvin-nr Oct 2, 2025
eeea943
chore: build binaries into docker images on cache hit
agarvin-nr Oct 2, 2025
36b3c6a
chore: clean up binary file in act
agarvin-nr Oct 2, 2025
9f8201a
chore: un-comment trivy check
agarvin-nr Oct 2, 2025
18113a8
chore: add step to validate existence of common goreleaser build files
agarvin-nr Oct 2, 2025
539ae82
chore: fix syntax to build and load docker images only on cache hit
agarvin-nr Oct 3, 2025
0c72637
chore: validate that there are binary directories and that binaries e…
agarvin-nr Oct 3, 2025
ce5d3a7
chore: ensure windows executables can be found
agarvin-nr Oct 3, 2025
7a34191
chore: change docker context to .tmp folder for easier cleanup
agarvin-nr Oct 6, 2025
4a52f47
chore: uncomment trivy
agarvin-nr Oct 6, 2025
51728cb
chore: copy binary into an appropriately-named file
agarvin-nr Oct 6, 2025
5594c3e
chore: copy all extra files defined by goreleaser into docker context
agarvin-nr Oct 7, 2025
869ae2e
chore: fix syntax on if for .tmp/docker subdirectory generation
agarvin-nr Oct 7, 2025
97bb441
chore: add validation of archives
agarvin-nr Oct 7, 2025
c85e67a
chore: add package validation
agarvin-nr Oct 7, 2025
9c08a9d
chore: simplify binary validation by using artifacts.json
agarvin-nr Oct 7, 2025
9742c81
chore: remove erroneous exit 1
agarvin-nr Oct 7, 2025
3bcbd42
chore: fix binary path in copy to docker context
agarvin-nr Oct 7, 2025
a1741b7
chore: fix syntax issues and improve readability
agarvin-nr Oct 7, 2025
cd06266
chore: add alert or fail conditions on jq returning empty strings
agarvin-nr Oct 7, 2025
d50ea54
chore: split validate-goreleaser-build and validate-source-files into…
agarvin-nr Oct 9, 2025
46bb153
chore: actually add script files
agarvin-nr Oct 9, 2025
c2d9ad7
chore: uncomment necessary checks
agarvin-nr Oct 9, 2025
75128e8
chore: replace env.tmp with runner.temp
agarvin-nr Oct 9, 2025
de87b60
Merge remote-tracking branch 'origin' into agarvin/binaryCache
agarvin-nr Oct 9, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
161 changes: 160 additions & 1 deletion .github/workflows/ci-base.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@ permissions:

env:
caching_enabled: ${{ !inputs.nightly }}
tmp: ${{ github.workspace }}/.tmp
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That sounds much nicer, I'll look into that, thank you!


jobs:
build:
Expand Down Expand Up @@ -174,17 +175,42 @@ jobs:
run: |
if [ ${{ inputs.nightly }} = "true" ] && [ ${{ inputs.fips }} = "true" ]; then
echo "goreleaser_args=--snapshot --clean --skip=publish,validate --timeout 2h --config .goreleaser-nightly-fips.yaml" >> $GITHUB_ENV
echo "goreleaser_file=.goreleaser-nightly-fips.yaml" >> $GITHUB_ENV
elif [ ${{ inputs.nightly }} = "true" ]; then
echo "goreleaser_args=--snapshot --clean --skip=publish,validate --timeout 2h --config .goreleaser-nightly.yaml" >> $GITHUB_ENV
echo "goreleaser_file=.goreleaser-nightly.yaml" >> $GITHUB_ENV
elif [ ${{ inputs.fips }} = "true" ]; then
echo "goreleaser_args=--snapshot --clean --skip=publish,validate --timeout 2h --config .goreleaser-fips.yaml" >> $GITHUB_ENV
echo "goreleaser_file=.goreleaser-fips.yaml" >> $GITHUB_ENV
elif [ ${{github.event.pull_request.user.login == 'dependabot[bot]' }} ]; then
echo "goreleaser_args=--snapshot --clean --skip=publish,validate,sign --timeout 2h" >> $GITHUB_ENV
echo "goreleaser_file=.goreleaser.yaml" >> $GITHUB_ENV
else
echo "goreleaser_args=--snapshot --clean --skip=publish,validate --timeout 2h" >> $GITHUB_ENV
echo "goreleaser_file=.goreleaser.yaml" >> $GITHUB_ENV
fi

- name: Generate binary cache key
id: generate_binary_key
run: |
BINARY_HASH="${{ hashFiles(
format('distributions/{0}/.goreleaser*.yaml', inputs.distribution),
format('distributions/{0}/_build*/*', inputs.distribution)
) }}"
ARGS_HASH=$(echo "${{ env.goreleaser_args }}" | sha256sum | cut -d' ' -f1)
echo binary_key=goreleaser-build-${{ inputs.distribution }}-${ARGS_HASH}-${BINARY_HASH} >> $GITHUB_OUTPUT

- name: Cache goreleaser build
id: cache-goreleaser
if: ${{ env.caching_enabled }}
uses: actions/cache@v4
with:
path: |
distributions/${{ inputs.distribution }}/dist
key: ${{ steps.generate_binary_key.outputs.binary_key }}

- name: Build binaries & packages with GoReleaser
if: steps.cache-goreleaser.outputs.cache-hit != 'true'
id: goreleaser
uses: goreleaser/goreleaser-action@v6
env:
Expand All @@ -197,10 +223,104 @@ jobs:
version: '2.11.2'
args: ${{ env.goreleaser_args }}
workdir: distributions/${{ inputs.distribution }}

- name: Skip GoReleaser build (cached)
if: steps.cache-goreleaser.outputs.cache-hit == 'true'
run: echo "✅ GoReleaser build skipped - using cached binaries"

- name: Validate GoReleaser build artifacts
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What are your thoughts on moving this to its own bash script? Since it's just a sanity check if might make sense to not clutter the workflow.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Putting it in its own bash script sounds much better to me! We might be able to re-use it for other workflows in the future that way.

I can do the same for the Verify source files exist step too, since it's a similar sanity check.

run: |
cd "distributions/${{ inputs.distribution }}"
if [ ! -d "dist" ]; then
echo "❌ dist directory not found!"
exit 1
fi

echo "📋 Verifying metadata files exist..."
files=("dist/artifacts.json" "dist/metadata.json")
missing_files=()
for file in "${files[@]}"; do
if [ ! -f "$file" ]; then
missing_files+=("$file")
else
echo "Found: $file"
fi
done
if [ ${#missing_files[@]} -ne 0 ]; then
echo "❌ files not found: ${missing_files[*]}"
exit 1
else
echo "✅ All common build files found!"
fi

echo "📋 Verifying Binaries exist..."
binaries=$( jq -r '.[] | select(.type == "Binary") | .path' dist/artifacts.json )
if [ -z "${binaries}" ]; then
echo "❌ No binaries found in artifacts.json"
exit 1
else
for binary in $binaries; do
if [ ! -f "${binary}" ]; then
echo "❌ ${binary} not found!"
exit 1
else
echo "Found: ${binary}"
fi
done
fi
echo "✅ All binaries found!"

echo "📋 Validating Archives and Packages..."
artifacts=$( jq -r '.[] | select(.type == "Archive" or .type == "Linux Package") | .path' dist/artifacts.json )
if [ -z "${artifacts}" ]; then
echo "⚠️ No archives or packages found in artifacts.json"
else
for artifact in $artifacts; do
echo "Validating ${artifact}"
# Verify the artifact file exists
if [ ! -f "${artifact}" ]; then
echo "❌ ${artifact} not found!"
exit 1
else
echo " Found artifact: ${artifact}"
fi
# Search for the corresponding checksum file and verify it exists
sum_file=$( jq -r ".[] | select(.type == \"Checksum\" and .extra.ChecksumOf == \"${artifact}\") | .path" dist/artifacts.json )
if [ -z "${sum_file}" ]; then
echo "❌ Checksum not defined for ${artifact} in artifacts.json"
exit 1
fi
if [ ! -f "${sum_file}" ]; then
echo "❌ ${sum_file} not found!"
exit 1
else
echo " Found checksum: ${sum_file}"
fi
# Compare checksums to ensure file integrity
artifact_sum=$(sha256sum ${artifact} | cut -d' ' -f1)
expected_sum=$(cat ${sum_file})
if [ "${artifact_sum}" != "${expected_sum}" ]; then
echo "❌ Checksums do not match!"
echo "Checksum: ${artifact_sum}"
echo "Expected: ${expected_sum}"
exit 1
else
echo " Checksum validated"
fi
done
echo "✅ Archives and Packages validated!"
fi

- name: Extract relevant metadata
run: |
VERSION=$(echo '${{ steps.goreleaser.outputs.metadata }}' | jq -r '.version')
if [ "${{ steps.cache-goreleaser.outputs.cache-hit }}" = "true" ]; then
VERSION=$(jq -r '.version' distributions/${{ inputs.distribution }}/dist/metadata.json)
echo "Using cached version: $VERSION"
else
# Extract from fresh GoReleaser build
VERSION=$(echo '${{ steps.goreleaser.outputs.metadata }}' | jq -r '.version')
echo "Using fresh build version: $VERSION"
fi
ARCH=$(echo '${{ runner.arch }}' | sed 's/X/amd/g')
ARCH=${ARCH@L}
echo "version=$VERSION" >> $GITHUB_ENV
Expand All @@ -215,6 +335,45 @@ jobs:
echo "image_tag=$VERSION-$ARCH" >> $GITHUB_ENV
fi

- name: Copy GoReleaser binary to Docker context
id: copy-binary
if: steps.cache-goreleaser.outputs.cache-hit == 'true'
run: |
cd distributions/${{ inputs.distribution }}
BINARY_PATH="$(find dist -name "${{ inputs.distribution }}*_linux_${{ env.arch }}*" -type d)/${{ inputs.distribution }}"
if [ ${{ inputs.fips }} = "true" ]; then
BINARY_PATH+="-fips"
fi
if [ ! -f "$BINARY_PATH" ]; then
echo "❌ Error: Binary not found at $BINARY_PATH"
find dist -name "*${{ inputs.distribution }}*" -type f
exit 1
fi
# move dockerfile dependencies to .tmp for easy cleanup when running in Act
BINARY_TMP="${{ env.tmp }}/docker/${{ inputs.distribution }}"
if [ ${{ inputs.fips }} = "true" ]; then
BINARY_TMP+="-fips"
fi
mkdir -p ${BINARY_TMP}
cp "$BINARY_PATH" "${BINARY_TMP}/${{ inputs.distribution }}"
cp Dockerfile ${BINARY_TMP}
for file in $(yq ".dockers[] | select(.goarch == \"${{ env.arch }}\") | .extra_files[]" ${{ env.goreleaser_file }}); do
cp "${file}" ${BINARY_TMP}
done
echo "✅ Binary copied: $(ls -la ${BINARY_TMP})"
echo "binary_tmp=${BINARY_TMP}" >> $GITHUB_OUTPUT

- name: Build and load Docker image
uses: docker/build-push-action@v5
if: steps.cache-goreleaser.outputs.cache-hit == 'true'
with:
context: ${{ steps.copy-binary.outputs.binary_tmp }}
platforms: linux/${{ env.arch }}
push: false
load: true
tags: |
${{ secrets.registry }}/${{ inputs.distribution }}:${{ env.image_tag }}

- name: Validate Usage of BoringCrypto
if: inputs.fips
run: |
Expand Down
Loading