-
Notifications
You must be signed in to change notification settings - Fork 15.9k
feat: update bazel central registry publish workflow #23465
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Closed
+147
−1
Closed
Changes from all commits
Commits
Show all changes
6 commits
Select commit
Hold shift + click to select a range
05d441e
feat: add BCR publish workflow
alexeagle 3de1277
Update release_bazel_module.yaml
alexeagle 9bae102
Update release_bazel_module.yaml
alexeagle d6b429a
Update prebuilt_tool_integrity.bzl
alexeagle 0c9dc69
extract the jq filter to Heredoc
alexeagle 05314db
Update release_bazel_module.yaml
alexeagle File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,5 +1,5 @@ | ||
{ | ||
"integrity": "**leave this alone**", | ||
"strip_prefix": "{REPO}-{VERSION}", | ||
"url": "https://github.com/{OWNER}/{REPO}/releases/download/{TAG}/{REPO}-{VERSION}.zip" | ||
"url": "https://github.com/{OWNER}/{REPO}/releases/download/{TAG}/{REPO}-{VERSION}.tar.gz" | ||
} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,4 @@ | ||
# Configuration for 'git archive' | ||
# see https://git-scm.com/docs/git-archive/2.40.0#ATTRIBUTES | ||
# Don't include compatibility folder in the distribution artifact, just to reduce size | ||
compatibility/ export-ignore |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,36 @@ | ||
# Publish new releases to Bazel Central Registry. | ||
name: Publish to BCR | ||
on: | ||
# Run the publish workflow after a successful release | ||
# Will be triggered from the release.yaml workflow | ||
workflow_call: | ||
inputs: | ||
tag_name: | ||
required: true | ||
type: string | ||
secrets: | ||
# This token should be owned by https://github.com/protobuf-team-bot | ||
BCR_PUBLISH_TOKEN: | ||
required: true | ||
# In case of problems, let release engineers retry by manually dispatching | ||
# the workflow from the GitHub UI | ||
workflow_dispatch: | ||
inputs: | ||
tag_name: | ||
description: git tag being released | ||
required: true | ||
type: string | ||
jobs: | ||
publish: | ||
uses: bazel-contrib/publish-to-bcr/.github/workflows/[email protected] | ||
with: | ||
tag_name: ${{ inputs.tag_name }} | ||
# GitHub repository which is a fork of the upstream where the Pull Request will be opened. | ||
registry_fork: protocolbuffers/bazel-central-registry | ||
permissions: | ||
attestations: write | ||
contents: write | ||
id-token: write | ||
secrets: | ||
# Necessary to push to the BCR fork, and to open a pull request against a registry | ||
publish_token: ${{ secrets.BCR_PUBLISH_TOKEN }} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,33 @@ | ||
# Prepare a release specifically for Bazel users, including a pre-built protoc. | ||
name: Bazel Release | ||
on: | ||
# Can be triggered from the GitHub Actions ui, using the "Run workflow" button on | ||
# https://github.com/protocolbuffers/protobuf/actions/workflows/release_bazel_module.yaml | ||
# TODO(alexeagle): consider automating the trigger from a new release being created | ||
workflow_dispatch: | ||
inputs: | ||
tag_name: | ||
description: git tag that has the protoc release artifact | ||
required: true | ||
type: string | ||
permissions: | ||
id-token: write | ||
attestations: write | ||
contents: write | ||
jobs: | ||
release: | ||
uses: bazel-contrib/.github/.github/workflows/[email protected] | ||
with: | ||
release_files: protobuf-*.tar.gz | ||
prerelease: false | ||
tag_name: ${{ inputs.tag_name }} | ||
# The release was already created by Google-internal mechanism, | ||
# so there aren't any useful assertions to make here. | ||
bazel_test_command: 'true' | ||
publish: | ||
needs: release | ||
uses: ./.github/workflows/publish_to_bcr.yaml | ||
with: | ||
tag_name: ${{ inputs.tag_name }} | ||
secrets: | ||
BCR_PUBLISH_TOKEN: ${{ secrets.BCR_PUBLISH_TOKEN }} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,53 @@ | ||
#!/usr/bin/env bash | ||
# NB: this file must be named release_prep.sh because the attestation generation doesn't trust user control. | ||
mkruskal-google marked this conversation as resolved.
Show resolved
Hide resolved
|
||
# see https://github.com/bazel-contrib/.github/blob/v7.2.3/.github/workflows/release_ruleset.yaml#L33-L45 | ||
set -o errexit -o nounset -o pipefail | ||
|
||
# Argument provided by reusable workflow caller, see | ||
# https://github.com/bazel-contrib/.github/blob/v7.2.3/.github/workflows/release_ruleset.yaml#L104 | ||
TAG=$1 | ||
PREFIX="protobuf-${TAG:1}" | ||
ARCHIVE="$PREFIX.tar.gz" | ||
ARCHIVE_TMP=$(mktemp) | ||
INTEGRITY_FILE=${PREFIX}/bazel/private/prebuilt_tool_integrity.bzl | ||
|
||
# NB: configuration for 'git archive' is in /.gitattributes | ||
git archive --format=tar --prefix=${PREFIX}/ ${TAG} > $ARCHIVE_TMP | ||
############ | ||
# Patch up the archive to have integrity hashes for built binaries that we downloaded in the GHA workflow. | ||
# Now that we've run `git archive` we are free to pollute the working directory. | ||
|
||
# Delete the placeholder file | ||
tar --file $ARCHIVE_TMP --delete $INTEGRITY_FILE | ||
|
||
# Use jq to translate GitHub Releases json into a Starlark object | ||
filter_releases=$(cat <<'EOF' | ||
# Read the file assets already present on the release | ||
reduce .assets[] as $a ( | ||
# Start with an empty dictionary, and for each asset, add | ||
{}; . + { | ||
# The format required in starlark, i.e. "release-name": "deadbeef123" | ||
($a.name): ($a.digest | sub("^sha256:"; "")) | ||
} | ||
) | ||
EOF | ||
) | ||
|
||
mkdir -p ${PREFIX}/bazel/private | ||
cat >${INTEGRITY_FILE} <<EOF | ||
"Generated during release by release_prep.sh" | ||
RELEASED_BINARY_INTEGRITY = $( | ||
curl -s https://api.github.com/repos/protocolbuffers/protobuf/releases/tags/${TAG} \ | ||
| jq -f <(echo "$filter_releases") | ||
) | ||
EOF | ||
|
||
# Append that generated file back into the archive | ||
tar --file $ARCHIVE_TMP --append ${INTEGRITY_FILE} | ||
|
||
# END patch up the archive | ||
############ | ||
|
||
gzip < $ARCHIVE_TMP > $ARCHIVE | ||
SHA=$(shasum -a 256 $ARCHIVE | awk '{print $1}') |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,20 @@ | ||
"""Release binary integrity hashes. | ||
This file contents are entirely replaced during release publishing, by .github/workflows/release_prep.sh | ||
so that the integrity of the prebuilt tools is included in the release artifact. | ||
The checked in content is only here to allow load() statements in the sources to resolve. | ||
""" | ||
|
||
# Create a mapping for every tool name to the hash of /dev/null | ||
NULLSHA = "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855" | ||
RELEASED_BINARY_INTEGRITY = { | ||
"-".join([ | ||
"protoc", | ||
os, | ||
arch, | ||
]): NULLSHA | ||
for [os, arch] in { | ||
"linux": ["aarch_64", "x86_64"], | ||
} | ||
} |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.