-
Notifications
You must be signed in to change notification settings - Fork 279
Add reproducible-build CI workflows (build + release PR) #3573
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
italo-sampaio
wants to merge
4
commits into
master
Choose a base branch
from
is/reproducible-build-ci
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+462
−0
Open
Changes from all commits
Commits
Show all changes
4 commits
Select commit
Hold shift + click to select a range
cc925c9
Add reproducible-build CI workflow
italo-sampaio fbad91a
Add reproducible-build PR workflow
italo-sampaio b14391b
Harden reproducible-build concurrency per review feedback
italo-sampaio 56155b8
Use RSK_CORE_GH_APP secrets for reproducible-builds App token
italo-sampaio File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,34 @@ | ||
| FROM eclipse-temurin:17-jdk@sha256:08295ab0f5007a37cbcc6679a8447a7278d9403f9f82acd80ed08cd10921e026 AS builder | ||
|
|
||
| RUN apt-get update -y && \ | ||
| apt-get install -y -qq --no-install-recommends git curl gnupg && \ | ||
| rm -rf /var/lib/apt/lists/* | ||
|
|
||
| WORKDIR /code/rskj | ||
|
|
||
| RUN gitrev=__REF__ && \ | ||
| git init && \ | ||
| git remote add origin https://github.com/rsksmart/rskj.git && \ | ||
| git fetch --depth 1 origin "$gitrev" && \ | ||
| git checkout FETCH_HEAD | ||
|
|
||
| RUN curl -sSL https://secchannel.rsk.co/SUPPORT.asc | gpg --import && \ | ||
| gpg --verify --output SHA256SUMS SHA256SUMS.asc && \ | ||
| sha256sum --check SHA256SUMS && \ | ||
| ./configure.sh && \ | ||
| ./gradlew --no-daemon clean build -x test -x checkstyleMain -x checkstyleTest -x checkstyleIntegrationTest && \ | ||
| ./gradlew publishRskjPublicationToMavenLocal | ||
|
|
||
|
|
||
| FROM eclipse-temurin:17-jre@sha256:f1515395c0695910a3ca665e973cc11013d1f50d265e61cb8c9156e999d914b4 AS runner | ||
|
|
||
| RUN useradd -m rsk | ||
|
|
||
| WORKDIR /home/rsk | ||
|
|
||
| USER rsk | ||
| COPY --from=builder --chown=rsk:rsk /code/rskj/rskj-core/build/libs/rskj-core-* ./ | ||
| COPY --from=builder --chown=rsk:rsk /code/rskj/rskj-core/build/rskj-core-*.pom ./ | ||
| COPY --from=builder --chown=rsk:rsk /root/.m2/repository/co/rsk/rskj-core/__VERSION__-__MODIFIER__/*.module ./ | ||
|
|
||
| CMD ["java", "-cp", "rskj-core-__VERSION__-__MODIFIER__-all.jar", "co.rsk.Start"] | ||
|
italo-sampaio marked this conversation as resolved.
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,32 @@ | ||
| # rskj __TAG__ | ||
|
|
||
| * Source: https://github.com/rsksmart/rskj | ||
| * Tag: `__TAG__` | ||
|
|
||
| ## Build | ||
|
|
||
| ``` | ||
| $ docker build -t rskj/__VERSION__-__MODIFIER_LC__ . | ||
| ``` | ||
|
|
||
| ## Verify | ||
|
|
||
| Run the following command to verify the sha256sum of the built artifacts matches the expected values: | ||
|
|
||
| ``` | ||
| $ docker run --rm rskj/__VERSION__-__MODIFIER_LC__ sh -c 'sha256sum * | grep -v javadoc.jar' | ||
|
|
||
| __HASHES__ | ||
| ``` | ||
|
|
||
| ## (Optional) Run RSK Node | ||
| ``` | ||
| $ docker run -d rskj/__VERSION__-__MODIFIER_LC__ | ||
| ``` | ||
|
|
||
| ## (Optional) Extract JAR from image | ||
|
|
||
| ``` | ||
| $ cid=$(docker run -d rskj/__VERSION__-__MODIFIER_LC__ /bin/true) | ||
|
italo-sampaio marked this conversation as resolved.
|
||
| $ docker cp "$cid":/home/rsk/ ./libs/ | ||
| $ docker rm "$cid" | ||
| ``` | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,196 @@ | ||
| name: Reproducible build PR | ||
|
|
||
| # Builds the canonical jars via the build-only `reproducible-build.yml`, | ||
| # then open a PR against rsksmart/reproducible-builds adding the rskj/<version>-<modifier> | ||
| # directory (Dockerfile + README with the hashes). | ||
| # | ||
| # Trust model: CI is the *first builder / PR author* only. The hashes it emits | ||
| # are informational — merging the generated PR still REQUIRES an independent | ||
| # rebuild confirming the same hashes. CI is intentionally not the sole witness. | ||
|
|
||
| on: | ||
| # GA release tags look like NAME-1.2.3. | ||
| push: | ||
| tags: | ||
| - "*-[0-9]*.[0-9]*.[0-9]*" | ||
|
|
||
| # Only read permission is needed. The cross-repo PR is opened with the | ||
| # scoped App token, NOT GITHUB_TOKEN. | ||
| permissions: | ||
| contents: read | ||
|
|
||
| concurrency: | ||
| group: reproducible-build-pr-${{ github.ref }} | ||
| cancel-in-progress: false | ||
|
|
||
| jobs: | ||
| # GA-only gate. push.tags globs match pre-release tags too, so re-check the | ||
| # tag here and skip (green, not red) anything that isn't a clean GA release. | ||
| guard: | ||
| runs-on: ubuntu-24.04 | ||
| outputs: | ||
| is_ga: ${{ steps.check.outputs.is_ga }} | ||
| steps: | ||
| - name: Classify the pushed tag | ||
| id: check | ||
| env: | ||
| TAG: ${{ github.ref_name }} | ||
| run: | | ||
| set -euo pipefail | ||
| is_ga=false | ||
| # All-caps release name + three-part numeric version, nothing trailing. | ||
| if printf '%s' "$TAG" | grep -Eq '^[A-Z]+-[0-9]+\.[0-9]+\.[0-9]+$'; then | ||
| is_ga=true | ||
| fi | ||
| # Belt-and-suspenders deny-list for pre-release markers (the regex above | ||
| # already excludes these; this makes the intent explicit and survives a | ||
| # future regex loosening). | ||
| case "$TAG" in | ||
| *PREVIEW*|*TESTNET*|*SNAPSHOT*|*-rc|*-rc[0-9]*|*-RC*|*-alpha*|*-beta*) | ||
| is_ga=false ;; | ||
| esac | ||
| echo "is_ga=$is_ga" >> "$GITHUB_OUTPUT" | ||
| if [ "$is_ga" = "true" ]; then | ||
| echo "Tag '$TAG' is a GA release — proceeding." | ||
| else | ||
| echo "::notice::Tag '$TAG' is not a GA release — skipping reproducible-build PR." | ||
| fi | ||
|
|
||
| # Build the canonical jars + hashes. NO `secrets:` passed → the build half can | ||
| # never see this workflow's App-token secrets. Runs only for GA tags. | ||
| build: | ||
| needs: guard | ||
| if: needs.guard.outputs.is_ga == 'true' | ||
| uses: ./.github/workflows/reproducible-build.yml | ||
| with: | ||
| ref: ${{ github.ref_name }} | ||
|
|
||
| # The only privileged job: mint the scoped App token and open the PR. | ||
| open-pr: | ||
| needs: [guard, build] | ||
| if: needs.guard.outputs.is_ga == 'true' | ||
| runs-on: ubuntu-24.04 | ||
| timeout-minutes: 15 | ||
| env: | ||
| TAG: ${{ github.ref_name }} | ||
| VERSION: ${{ needs.build.outputs.version }} | ||
| MODIFIER: ${{ needs.build.outputs.modifier }} | ||
| TARGET_REPO: rsksmart/reproducible-builds | ||
| steps: | ||
| - name: Checkout rskj at the tag (templates only) | ||
| uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0 | ||
| with: | ||
| ref: ${{ github.ref_name }} | ||
| sparse-checkout: | | ||
| .github/reproducible-build | ||
|
|
||
| - name: Download canonical build artifacts | ||
| uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1 | ||
| with: | ||
| name: ${{ needs.build.outputs.artifact_name }} | ||
| path: artifacts | ||
|
|
||
| - name: Render Dockerfile and README for the release directory | ||
| id: render | ||
| run: | | ||
| set -euo pipefail | ||
| # modifier_lc derivation lives HERE (the build half is modifier-case | ||
| # agnostic); the published dir + README use the lowercase form. | ||
| modifier_lc=$(printf '%s' "$MODIFIER" | tr '[:upper:]' '[:lower:]') | ||
| echo "modifier_lc=$modifier_lc" >> "$GITHUB_OUTPUT" | ||
|
|
||
| # Hashes come from the artifact's hashes.txt (authoritative copy the | ||
| # build job wrote), so README == artifact == a verifier's local check. | ||
| if [ ! -s artifacts/hashes.txt ]; then | ||
| echo "::error::artifacts/hashes.txt missing or empty"; exit 1 | ||
| fi | ||
| hashes=$(cat artifacts/hashes.txt) | ||
|
|
||
| mkdir -p out | ||
| # Dockerfile pins the immutable tag as the build ref. | ||
| sed -e "s|__REF__|${TAG}|g" \ | ||
| -e "s|__VERSION__|${VERSION}|g" \ | ||
| -e "s|__MODIFIER__|${MODIFIER}|g" \ | ||
| .github/reproducible-build/Dockerfile.tmpl > out/Dockerfile | ||
|
|
||
| # README: tag/version/modifier_lc via sed, then the multi-line hash | ||
| # block via awk (sed chokes on multi-line / slashed replacements). | ||
| sed -e "s|__TAG__|${TAG}|g" \ | ||
| -e "s|__VERSION__|${VERSION}|g" \ | ||
| -e "s|__MODIFIER_LC__|${modifier_lc}|g" \ | ||
| .github/reproducible-build/README.md.tmpl > out/README.partial.md | ||
| awk -v hashes="$hashes" '{ gsub(/__HASHES__/, hashes); print }' \ | ||
| out/README.partial.md > out/README.md | ||
|
|
||
| - name: Mint scoped GitHub App token | ||
| id: app-token | ||
| uses: actions/create-github-app-token@bcd2ba49218906704ab6c1aa796996da409d3eb1 # v3.2.0 | ||
| with: | ||
| app-id: ${{ secrets.RSK_CORE_GH_APP_ID }} | ||
| private-key: ${{ secrets.RSK_CORE_GH_APP_PRIVATE_KEY }} | ||
| owner: rsksmart | ||
| repositories: reproducible-builds | ||
|
|
||
| - name: Open the reproducible-builds PR | ||
| env: | ||
| GH_TOKEN: ${{ steps.app-token.outputs.token }} | ||
| TOKEN: ${{ steps.app-token.outputs.token }} | ||
| MODIFIER_LC: ${{ steps.render.outputs.modifier_lc }} | ||
| run: | | ||
| set -euo pipefail | ||
| rel_dir="rskj/${VERSION}-${MODIFIER_LC}" | ||
| branch="reproducible-build/${TAG}" | ||
|
|
||
| # Clone over x-access-token (devportal-update.yml pattern). The clone | ||
| # lands on the default branch — the reference for the existing-dir check. | ||
| git clone "https://x-access-token:${TOKEN}@github.com/${TARGET_REPO}.git" target | ||
| cd target | ||
| default_branch=$(git rev-parse --abbrev-ref HEAD) | ||
|
|
||
| # Fail-loud: never overwrite an already-published release directory. | ||
| if [ -e "$rel_dir" ]; then | ||
| echo "::error::${rel_dir} already exists in ${TARGET_REPO} — refusing to overwrite a published release." | ||
| exit 1 | ||
| fi | ||
|
|
||
| # Dedicated per-tag branch; -B so a re-run resets cleanly off default. | ||
| git checkout -B "$branch" | ||
| mkdir -p "$rel_dir" | ||
| cp ../out/Dockerfile "$rel_dir/Dockerfile" | ||
| cp ../out/README.md "$rel_dir/README.md" | ||
|
|
||
| git config user.name "github-actions[bot]" | ||
| git config user.email "github-actions[bot]@users.noreply.github.com" | ||
| git add "$rel_dir" | ||
| git commit -m "Add reproducible build for ${TAG}" | ||
| # Force-push: the branch is ours alone (per-tag), so a re-run overwrites | ||
| # its own prior push rather than failing. | ||
| git push -u -f origin "$branch" | ||
|
|
||
| # gh (unlike peter-evans) won't reconcile an existing PR, so check first. | ||
| existing=$(gh pr list --repo "$TARGET_REPO" --head "$branch" --state open \ | ||
| --json url -q '.[0].url' || true) | ||
| if [ -n "$existing" ]; then | ||
| echo "::notice::PR already open for ${branch}: ${existing}" | ||
| exit 0 | ||
| fi | ||
|
|
||
| body=$(cat <<EOF | ||
| Automated reproducible build for rskj tag \`${TAG}\`, opened by the | ||
| \`Reproducible build PR\` workflow in rskj. | ||
|
|
||
| **Before merging:** independently rebuild and confirm the hashes match. | ||
| CI is the first builder only — it is not the sole witness, and this PR | ||
| must not be auto-merged. | ||
|
|
||
| \`\`\` | ||
| $(cat ../artifacts/hashes.txt) | ||
| \`\`\` | ||
| EOF | ||
| ) | ||
| gh pr create \ | ||
| --repo "$TARGET_REPO" \ | ||
| --base "$default_branch" \ | ||
| --head "$branch" \ | ||
| --title "Add reproducible build for ${TAG}" \ | ||
| --body "$body" |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.