Skip to content

build: add support for ARM architecture #1056

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
87 changes: 62 additions & 25 deletions .github/workflows/_build.yaml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright (c) 2022 - 2023, Oracle and/or its affiliates. All rights reserved.
# Copyright (c) 2022 - 2025, Oracle and/or its affiliates. All rights reserved.
# Licensed under the Universal Permissive License v 1.0 as shown at https://oss.oracle.com/licenses/upl/.

# This is a trusted builder implemented as a reusable workflow that can be called by other
Expand All @@ -25,32 +25,48 @@
name: Build the package
on:
workflow_call:
outputs:
artifacts-sha256:
description: The hash of the artifacts
value: ${{ jobs.build.outputs.artifacts-sha256 }}
permissions:
contents: read
env:
ARTIFACT_OS: ubuntu-latest # The default OS for release.
ARTIFACT_PYTHON: '3.11' # The default Python version for release.
PACKAGE_PATH: src/macaron # The relative Python package path to the repo.
RELEASE_OS_X86_64: ubuntu-24.04 # Default OS for x86_64-compatible release artifacts.
RELEASE_OS_ARM64: ubuntu-24.04-arm # Default OS for ARM64-compatible release artifacts.
RELEASE_PYTHON_VERSION: '3.11' # Default Python version used for release artifacts.
PACKAGE_PATH: src/macaron # The relative Python package path to the repo.

jobs:
build:
outputs:
artifacts-sha256: ${{ steps.compute-hash.outputs.artifacts-sha256 }}
name: Build Macaron
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
# It is recommended to pin a Runner version specifically:
# https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners
os: [ubuntu-latest]
os: [ubuntu-24.04, ubuntu-24.04-arm]
python: ['3.11']

outputs:
arch-env: ${{ steps.set-arch-env.outputs.arch_env }}

steps:

# Create a GitHub Actions environment variable that maps a matrix.os value to a more descriptive environment
# value (e.g., ubuntu-x86-64 or ubuntu-arm64).
- name: Determine architecture label
id: set-arch-env
shell: bash
run: |
if [[ "${{ matrix.os }}" == "ubuntu-24.04" ]]; then
echo "arch_env=ubuntu-x86-64" >> "$GITHUB_OUTPUT"
elif [[ "${{ matrix.os }}" == "ubuntu-24.04-arm" ]]; then
echo "arch_env=ubuntu-arm64" >> "$GITHUB_OUTPUT"
else
echo "arch_env=unknown" >> "$GITHUB_OUTPUT"
fi

- name: Test the env variable
run: echo "Architecture-specific value ${{ steps.set-arch-env.outputs.arch_env }}"

- name: Check out repository
uses: actions/checkout@d632683dd7b4114ad314bca15554477dd762a938 # v4.2.0
with:
Expand Down Expand Up @@ -91,24 +107,33 @@ jobs:
GITHUB_TOKEN: ${{ github.token }}

# Generate the requirements.txt that contains the hash digests of the dependencies and
# generate the SBOM using CycloneDX SBOM generator.
# generate the SBOM using CyclonDX SBOM generator for the release Python version and
# supported release OS targets.
- name: Generate requirements.txt and SBOM
if: matrix.os == env.ARTIFACT_OS && matrix.python == env.ARTIFACT_PYTHON
if: >
matrix.python == env.RELEASE_PYTHON_VERSION &&
(matrix.os == env.RELEASE_OS_X86_64 || matrix.os == env.RELEASE_OS_ARM64)
run: make requirements sbom

# Remove the old requirements.txt file (which includes _all_ packages) and generate a
# new one for the package and its actual and required dependencies only.
# new one for the package and its actual and required dependencies only. Run this step
# for the release Python version and supported release OS targets only.
- name: Prune packages and generate required requirements.txt
if: matrix.os == env.ARTIFACT_OS && matrix.python == env.ARTIFACT_PYTHON
if: >
matrix.python == env.RELEASE_PYTHON_VERSION &&
(matrix.os == env.RELEASE_OS_X86_64 || matrix.os == env.RELEASE_OS_ARM64)
run: |
rm requirements.txt
make prune requirements

# Find the paths to the artifact files that will be included in the release, compute
# the SHA digest for all the release files and encode them using Base64, and export it
# from this job.
# the SHA digest for all the release files and encode them using Base64, and upload it
# from this job. Run this step for the release Python version and supported release
# OS targets only.
- name: Compute package hash
if: matrix.os == env.ARTIFACT_OS && matrix.python == env.ARTIFACT_PYTHON
if: >
matrix.python == env.RELEASE_PYTHON_VERSION &&
(matrix.os == env.RELEASE_OS_X86_64 || matrix.os == env.RELEASE_OS_ARM64)
id: compute-hash
shell: bash
run: |
Expand All @@ -123,19 +148,32 @@ jobs:
DIGEST=$(sha256sum "$TARBALL_PATH" "$WHEEL_PATH" "$REQUIREMENTS_PATH" "$SBOM_PATH" \
"$SBOM_GO_PATH" "$HTML_DOCS_PATH" "$BUILD_EPOCH_PATH" | base64 -w0)
echo "Digest of artifacts is $DIGEST."
echo "artifacts-sha256=$DIGEST" >> "$GITHUB_OUTPUT"
echo "$DIGEST" > artifacts-sha256-file-${{ steps.set-arch-env.outputs.arch_env }}

# For now only generate artifacts for the specified OS and Python version in env variables.
# Currently reusable workflows do not support setting strategy property from the caller workflow.
- name: Upload the package artifact for debugging and release
if: matrix.os == env.ARTIFACT_OS && matrix.python == env.ARTIFACT_PYTHON
if: >
matrix.python == env.RELEASE_PYTHON_VERSION &&
(matrix.os == env.RELEASE_OS_X86_64 || matrix.os == env.RELEASE_OS_ARM64)
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
with:
name: artifact-${{ matrix.os }}-python-${{ matrix.python }}
path: dist
name: artifacts-${{ steps.set-arch-env.outputs.arch_env }}
path: ./dist*/
if-no-files-found: error
retention-days: 7

# Run this step for the release Python version and supported release OS targets only.
- name: Upload artifacts sha256
if: >
matrix.python == env.RELEASE_PYTHON_VERSION &&
(matrix.os == env.RELEASE_OS_X86_64 || matrix.os == env.RELEASE_OS_ARM64)
uses: actions/upload-artifact@6f51ac03b9356f520e9adb1b1b7802705f340c2b # v4.5.0
with:
name: artifacts-sha256-file-${{ steps.set-arch-env.outputs.arch_env }}
path: artifacts-sha256-file-${{ steps.set-arch-env.outputs.arch_env }}
retention-days: 7

# This job calls the reusable workflow _build_docker.yaml to build and test
# the Docker image. Note that the built image is not pushed to ghcr.io here.
build_docker_image:
Expand All @@ -145,7 +183,6 @@ jobs:
packages: read
uses: ./.github/workflows/_build_docker.yaml
with:
artifact-sha256: ${{ needs.build.outputs.artifacts-sha256 }}
# TODO: use ${{ env.ARTIFACT_OS }} and ${{ env.ARTIFACT_PYTHON }}
# TODO: use ${{ env.RELEASE_OS_X86_64 }}
# when this issue is addressed: https://github.com/actions/runner/issues/2394.
artifact-name: artifact-ubuntu-latest-python-3.11
artifact-architecture: ubuntu-x86-64
36 changes: 25 additions & 11 deletions .github/workflows/_build_docker.yaml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright (c) 2023 - 2024, Oracle and/or its affiliates. All rights reserved.
# Copyright (c) 2023 - 2025, Oracle and/or its affiliates. All rights reserved.
# Licensed under the Universal Permissive License v 1.0 as shown at https://oss.oracle.com/licenses/upl/.

# This is a reuseable workflow to build and test the Docker image. Note that this workflow does not
Expand All @@ -10,10 +10,7 @@ name: Build and push Docker image
on:
workflow_call:
inputs:
artifact-name:
required: true
type: string
artifact-sha256:
artifact-architecture:
required: true
type: string
permissions:
Expand All @@ -40,18 +37,35 @@ jobs:
- name: Download artifact
uses: actions/download-artifact@65a9edc5881444af0b9093a5e628f2fe47ea3b2e # v4.1.7
with:
name: ${{ inputs.artifact-name }}
path: dist
path: downloads

# Verify hashes by first computing hashes for the artifacts and then comparing them
# against the hashes for the artifact.
- name: Verify the artifact hash
env:
ARTIFACT_HASH: ${{ inputs.artifact-sha256 }}
run: |
set -euo pipefail
echo "Hash of package should be $ARTIFACT_HASH."
echo "$ARTIFACT_HASH" | base64 --decode | sha256sum --strict --check --status || exit 1
cd downloads
ARCH=${{ inputs.artifact-architecture }}
HASH_DIR="artifacts-sha256-file-${ARCH}"
ARTIFACT_DIR="artifacts-${ARCH}"
HASH_FILE="${HASH_DIR}/artifacts-sha256-file-${ARCH}"

echo "Verifying artifacts for ${ARCH}"
echo "Decoding expected SHA256 digest:"
DECODED_HASH=$(base64 --decode "${HASH_FILE}")
echo "$DECODED_HASH"

pushd "${ARTIFACT_DIR}"
echo "$DECODED_HASH" | sha256sum --strict --check --status || {
echo "Hash verification failed for ${ARCH}!"
exit 1
}
popd

# Copy the target dist folder to the repo directory for the subsequent steps.
cp -r "${ARTIFACT_DIR}"/dist ../

echo "Hash verified successfully for ${ARCH}."

# Build the Docker image without pushing it.
- name: Build the Docker image
Expand Down
39 changes: 26 additions & 13 deletions .github/workflows/_deploy-github-pages.yaml
Original file line number Diff line number Diff line change
@@ -1,19 +1,15 @@
# Copyright (c) 2023 - 2023, Oracle and/or its affiliates. All rights reserved.
# Copyright (c) 2023 - 2025, Oracle and/or its affiliates. All rights reserved.
# Licensed under the Universal Permissive License v 1.0 as shown at https://oss.oracle.com/licenses/upl/.

# This workflow deploys the documentations to GitHub Pages.
name: Deploy static content to Pages
on:
workflow_call:
inputs:
artifact-name:
type: string
artifact-architecture:
required: true
description: The artifact name that contains docs content
artifact-sha256:
type: string
required: true
description: The sha of the artifact that contains docs content
description: The artifact distribution that contains docs content.
# Sets permissions of the GITHUB_TOKEN to allow deployment to GitHub Pages
permissions:
contents: read
Expand All @@ -37,18 +33,35 @@ jobs:
- name: Download artifact
uses: actions/download-artifact@65a9edc5881444af0b9093a5e628f2fe47ea3b2e # v4.1.7
with:
name: ${{ inputs.artifact-name }}
path: dist
path: downloads

# Verify hashes by first computing hashes for the artifacts and then comparing them
# against the hashes for the artifact.
- name: Verify the artifact hash
env:
ARTIFACT_HASH: ${{ inputs.artifact-sha256 }}
run: |
set -euo pipefail
echo "Hash of package should be $ARTIFACT_HASH."
echo "$ARTIFACT_HASH" | base64 --decode | sha256sum --strict --check --status || exit 1
cd downloads
ARCH=${{ inputs.artifact-architecture }}
HASH_DIR="artifacts-sha256-file-${ARCH}"
ARTIFACT_DIR="artifacts-${ARCH}"
HASH_FILE="${HASH_DIR}/artifacts-sha256-file-${ARCH}"

echo "Verifying artifacts for ${ARCH}"
echo "Decoding expected SHA256 digest:"
DECODED_HASH=$(base64 --decode "${HASH_FILE}")
echo "$DECODED_HASH"

pushd "${ARTIFACT_DIR}"
echo "$DECODED_HASH" | sha256sum --strict --check --status || {
echo "Hash verification failed for ${ARCH}!"
exit 1
}
popd

# Copy the target dist folder to the repo directory for the subsequent steps.
cp -r "${ARTIFACT_DIR}"/dist ../

echo "Hash verified successfully for ${ARCH}."

# Prepare the docs content.
- name: Prepare docs for release
Expand Down
46 changes: 45 additions & 1 deletion .github/workflows/pr-change-set.yaml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright (c) 2022 - 2023, Oracle and/or its affiliates. All rights reserved.
# Copyright (c) 2022 - 2025, Oracle and/or its affiliates. All rights reserved.
# Licensed under the Universal Permissive License v 1.0 as shown at https://oss.oracle.com/licenses/upl/.

# This workflow checks and tests the package code, builds all package
Expand All @@ -23,3 +23,47 @@ jobs:
permissions:
contents: read
packages: read

verify_artifacts:
needs: [build]
name: Verify artifacts
runs-on: ubuntu-latest
permissions:
contents: read

steps:

# Download all uploaded artifacts in the build job into the 'downloads' directory.
# This includes built package distributions and SHA256 hash files from some matrix jobs.
# The `path` input ensures all artifacts are placed under the 'downloads/' folder while
# maintaining their respective artifact subdirectory structure.
- name: Download artifact
uses: actions/download-artifact@65a9edc5881444af0b9093a5e628f2fe47ea3b2e # v4.1.7
with:
path: downloads

# Verify hashes by first computing hashes for the artifacts and then comparing them
# against the hashes computed by the build job.
- name: Verify the artifact hash
run: |
set -euo pipefail
cd downloads
for ARCH in "ubuntu-x86-64" "ubuntu-arm64"; do
HASH_DIR="artifacts-sha256-file-${ARCH}"
ARTIFACT_DIR="artifacts-${ARCH}"
HASH_FILE="${HASH_DIR}/artifacts-sha256-file-${ARCH}"

echo "Verifying artifacts for ${ARCH}"
echo "Decoding expected SHA256 digest:"
DECODED_HASH=$(base64 --decode "${HASH_FILE}")
echo "$DECODED_HASH"

pushd "${ARTIFACT_DIR}"
echo "$DECODED_HASH" | sha256sum --strict --check --status || {
echo "Hash verification failed for ${ARCH}!"
exit 1
}
popd

echo "Hash verified successfully for ${ARCH}"
done
38 changes: 27 additions & 11 deletions .github/workflows/release.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,6 @@ on:
permissions:
contents: read
env:
ARTIFACT_NAME: artifact-ubuntu-latest-python-3.11
# This is the username and email for the user who commits and pushes the release
# commit. In an organisation that should be a dedicated devops account.
USER_NAME: behnazh-w
Expand Down Expand Up @@ -133,18 +132,36 @@ jobs:
- name: Download artifact
uses: actions/download-artifact@65a9edc5881444af0b9093a5e628f2fe47ea3b2e # v4.1.7
with:
name: ${{ env.ARTIFACT_NAME }}
path: dist
path: downloads

# Verify hashes by first computing hashes for the artifacts and then comparing them
# against the hashes computed by the build job.
- name: Verify the artifact hash
env:
ARTIFACT_HASH: ${{ needs.build.outputs.artifacts-sha256 }}
run: |
set -euo pipefail
echo "Hash of package should be $ARTIFACT_HASH."
echo "$ARTIFACT_HASH" | base64 --decode | sha256sum --strict --check --status || exit 1
cd downloads
for ARCH in "ubuntu-x86-64" "ubuntu-arm64"; do
HASH_DIR="artifacts-sha256-file-${ARCH}"
ARTIFACT_DIR="artifacts-${ARCH}"
HASH_FILE="${HASH_DIR}/artifacts-sha256-file-${ARCH}"

echo "Verifying artifacts for ${ARCH}"
echo "Decoding expected SHA256 digest:"
DECODED_HASH=$(base64 --decode "${HASH_FILE}")
echo "$DECODED_HASH"

pushd "${ARTIFACT_DIR}"
echo "$DECODED_HASH" | sha256sum --strict --check --status || {
echo "Hash verification failed for ${ARCH}!"
exit 1
}
popd

# Copy the target dist folder to the repo directory for the subsequent steps.
cp -r "${ARTIFACT_DIR}"/dist ../

echo "Hash verified successfully for ${ARCH}"
done

# Log in to ghcr.io to push the Docker image.
- name: Log in to GitHub Container Registry
Expand Down Expand Up @@ -329,10 +346,9 @@ jobs:
pages: write
id-token: write
with:
# TODO: use ${{ env.ARTIFACT_NAME }} when this issue is addressed:
# https://github.com/actions/runner/issues/2394.
artifact-name: artifact-ubuntu-latest-python-3.11
artifact-sha256: ${{ needs.build.outputs.artifacts-sha256 }}
# TODO: use ${{ env.RELEASE_OS_X86_64 }}
# when this issue is addressed: https://github.com/actions/runner/issues/2394.
artifact-architecture: ubuntu-x86-64

# Send out release notifications after the Release was published on GitHub.
# Uncomment the `if` to disable sending release notifications.
Expand Down
Loading
Loading