-
Notifications
You must be signed in to change notification settings - Fork 24
chore: add binary caching #408
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 8 commits
Commits
Show all changes
25 commits
Select commit
Hold shift + click to select a range
f6a9d6f
chore: add binary caching
agarvin-nr eeea943
chore: build binaries into docker images on cache hit
agarvin-nr 36b3c6a
chore: clean up binary file in act
agarvin-nr 9f8201a
chore: un-comment trivy check
agarvin-nr 18113a8
chore: add step to validate existence of common goreleaser build files
agarvin-nr 539ae82
chore: fix syntax to build and load docker images only on cache hit
agarvin-nr 0c72637
chore: validate that there are binary directories and that binaries e…
agarvin-nr ce5d3a7
chore: ensure windows executables can be found
agarvin-nr 7a34191
chore: change docker context to .tmp folder for easier cleanup
agarvin-nr 4a52f47
chore: uncomment trivy
agarvin-nr 51728cb
chore: copy binary into an appropriately-named file
agarvin-nr 5594c3e
chore: copy all extra files defined by goreleaser into docker context
agarvin-nr 869ae2e
chore: fix syntax on if for .tmp/docker subdirectory generation
agarvin-nr 97bb441
chore: add validation of archives
agarvin-nr c85e67a
chore: add package validation
agarvin-nr 9c08a9d
chore: simplify binary validation by using artifacts.json
agarvin-nr 9742c81
chore: remove erroneous exit 1
agarvin-nr 3bcbd42
chore: fix binary path in copy to docker context
agarvin-nr a1741b7
chore: fix syntax issues and improve readability
agarvin-nr cd06266
chore: add alert or fail conditions on jq returning empty strings
agarvin-nr d50ea54
chore: split validate-goreleaser-build and validate-source-files into…
agarvin-nr 46bb153
chore: actually add script files
agarvin-nr c2d9ad7
chore: uncomment necessary checks
agarvin-nr 75128e8
chore: replace env.tmp with runner.temp
agarvin-nr de87b60
Merge remote-tracking branch 'origin' into agarvin/binaryCache
agarvin-nr File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -184,7 +184,27 @@ jobs: | |
| echo "goreleaser_args=--snapshot --clean --skip=publish,validate --timeout 2h" >> $GITHUB_ENV | ||
| fi | ||
|
|
||
| - name: Generate binary cache key | ||
| id: generate_binary_key | ||
| run: | | ||
| BINARY_HASH="${{ hashFiles( | ||
| format('distributions/{0}/.goreleaser*.yaml', inputs.distribution), | ||
| format('distributions/{0}/_build*/*', inputs.distribution) | ||
| ) }}" | ||
| ARGS_HASH=$(echo "${{ env.goreleaser_args }}" | sha256sum | cut -d' ' -f1) | ||
| echo binary_key=goreleaser-build-${{ inputs.distribution }}-${ARGS_HASH}-${BINARY_HASH} >> $GITHUB_OUTPUT | ||
|
|
||
| - name: Cache goreleaser build | ||
| id: cache-goreleaser | ||
| if: ${{ env.caching_enabled }} | ||
| uses: actions/cache@v4 | ||
| with: | ||
| path: | | ||
| distributions/${{ inputs.distribution }}/dist | ||
| key: ${{ steps.generate_binary_key.outputs.binary_key }} | ||
|
|
||
| - name: Build binaries & packages with GoReleaser | ||
| if: steps.cache-goreleaser.outputs.cache-hit != 'true' | ||
| id: goreleaser | ||
| uses: goreleaser/goreleaser-action@v6 | ||
| env: | ||
|
|
@@ -197,10 +217,72 @@ jobs: | |
| version: '2.11.2' | ||
| args: ${{ env.goreleaser_args }} | ||
| workdir: distributions/${{ inputs.distribution }} | ||
|
|
||
| - name: Skip GoReleaser build (cached) | ||
| if: steps.cache-goreleaser.outputs.cache-hit == 'true' | ||
| run: echo "✅ GoReleaser build skipped - using cached binaries" | ||
|
|
||
| - name: Validate GoReleaser build files exist | ||
| run: | | ||
| DIST_PATH="distributions/${{ inputs.distribution }}/dist" | ||
| if [ ! -d "$DIST_PATH" ]; then | ||
| echo "❌ $DIST_PATH not found!" | ||
| exit 1 | ||
| fi | ||
| cd "$DIST_PATH" | ||
| echo "📋 Checking for common files..." | ||
| files=("artifacts.json" "config.yaml" "metadata.json") | ||
| missing_files=() | ||
| for file in "${files[@]}"; do | ||
| if [ ! -f "$file" ]; then | ||
| missing_files+=("$file") | ||
| else | ||
| echo "Found: $file" | ||
| fi | ||
| done | ||
| if [ ${#missing_files[@]} -ne 0 ]; then | ||
| echo "❌ files not found: ${missing_files[*]}" | ||
| exit 1 | ||
| else | ||
| echo "✅ All common build files found!" | ||
| fi | ||
| echo "📋 Checking if binaries / executables exist..." | ||
| binary_dirs=($(find . -name "${{ inputs.distribution }}*_*" -type d)) | ||
| if [ ${#binary_dirs[@]} -eq 0 ]; then | ||
| echo "❌ No binary directories found!" | ||
| exit 1 | ||
| fi | ||
| BINARY_FILE="${{ inputs.distribution }}" | ||
|
||
| if [ ${{ inputs.fips }} = "true" ]; then | ||
| BINARY_FILE="${BINARY_FILE}-fips" | ||
| fi | ||
| missing_binaries=() | ||
| for dir in "${binary_dirs[@]}"; do | ||
| binary="$(find $dir -name "${BINARY_FILE}*" -type f)" | ||
| if [ -z "$binary" ]; then | ||
| missing_binaries+=("$binary") | ||
| else | ||
| echo "Found: $binary" | ||
| fi | ||
| done | ||
| if [ ${#missing_binaries[@]} -ne 0 ]; then | ||
| echo "❌ Binaries / Executables not found: ${missing_binaries[*]}" | ||
| exit 1 | ||
| else | ||
| echo "✅ All found directories contain binaries!" | ||
| fi | ||
| echo "binary_file=$BINARY_FILE" >> $GITHUB_ENV | ||
|
|
||
| - name: Extract relevant metadata | ||
| run: | | ||
| VERSION=$(echo '${{ steps.goreleaser.outputs.metadata }}' | jq -r '.version') | ||
| if [ "${{ steps.cache-goreleaser.outputs.cache-hit }}" = "true" ]; then | ||
| VERSION=$(jq -r '.version' distributions/${{ inputs.distribution }}/dist/metadata.json) | ||
| echo "Using cached version: $VERSION" | ||
| else | ||
| # Extract from fresh GoReleaser build | ||
| VERSION=$(echo '${{ steps.goreleaser.outputs.metadata }}' | jq -r '.version') | ||
| echo "Using fresh build version: $VERSION" | ||
| fi | ||
| ARCH=$(echo '${{ runner.arch }}' | sed 's/X/amd/g') | ||
| ARCH=${ARCH@L} | ||
| echo "version=$VERSION" >> $GITHUB_ENV | ||
|
|
@@ -215,6 +297,34 @@ jobs: | |
| echo "image_tag=$VERSION-$ARCH" >> $GITHUB_ENV | ||
| fi | ||
|
|
||
| - name: Copy GoReleaser binary to Docker context | ||
| if: steps.cache-goreleaser.outputs.cache-hit == 'true' | ||
| run: | | ||
| cd distributions/${{ inputs.distribution }} | ||
| BINARY_PATH="$(find dist -name "${{ inputs.distribution }}*_linux_${{ env.arch }}*" -type d)/${{ env.binary_file }}" | ||
| if [ ! -f "$BINARY_PATH" ]; then | ||
| echo "❌ Error: Binary not found at $BINARY_PATH" | ||
| find dist -name "*${{ inputs.distribution }}*" -type f | ||
| exit 1 | ||
| fi | ||
| cp "$BINARY_PATH" ./${{ inputs.distribution }} | ||
| echo "✅ Binary copied: $(ls -la ./${{ inputs.distribution }})" | ||
|
|
||
| - name: Build and load Docker image | ||
| uses: docker/build-push-action@v5 | ||
| if: steps.cache-goreleaser.outputs.cache-hit == 'true' | ||
| with: | ||
| context: distributions/${{ inputs.distribution }} | ||
| platforms: linux/${{ env.arch }} | ||
| push: false | ||
| load: true | ||
| tags: | | ||
| ${{ secrets.registry }}/${{ inputs.distribution }}:${{ env.image_tag }} | ||
|
|
||
| - name: Clean up binary file (Act) | ||
agarvin-nr marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| if: ${{ env.ACT && steps.cache-goreleaser.outputs.cache-hit == 'true' }} | ||
| run: rm distributions/${{ inputs.distribution }}/${{ inputs.distribution }} | ||
|
|
||
| - name: Validate Usage of BoringCrypto | ||
| if: inputs.fips | ||
| run: | | ||
|
|
||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Trying a hard stop if metadata isn't found in the cached build, following this discussion on the original draft.