Skip to content

Commit 3c6d32e

Browse files
Nightly Release of Torch-MLIR python wheels for linux_x86_64 builds (#1)
Reuses [build_linux_packages.sh](https://github.com/llvm/torch-mlir/blob/main/build_tools/python_deploy/build_linux_packages.sh) script to build the project and generate python wheels. I however chose to not port the original github actions workflows for snapshot releases, which seemed to do a lot more than what I initially wanted. Here's what's roughly different in this trimmed down version: - html patching for hosting pip wheels is no longer used - instead went with the [expand_assets trick](https://fzakaria.com/2024/01/15/abusing-github-as-a-pypi-server.html), similar to [openxla/stablehlo](https://github.com/openxla/stablehlo/blob/main/.github/workflows/publishWheelRelease.yml) - only release linux x86_64 prebuilt wheels (not macos, windows, aarch64 builds) - this may be updated later by interested parties I used a standalone fork which was useful for the initial bringup - it took a good 30-ish iterations on `main` branch to nail the github workflow right, and some playing with auth tokens. Here's how the release page looks like: - https://github.com/sjain-stanford/torch-mlir-release/releases - https://github.com/sjain-stanford/torch-mlir-release/releases/expanded_assets/dev-wheels (it follows the same naming convention as existing wheels, but updates the same `dev-wheels` release in place to avoid changing the expand_assets link required for pip-style hosting) Here's the workflow: - https://github.com/sjain-stanford/torch-mlir-release/actions/runs/7825872338 @stellaraccident and I took a call to avoid using Personal Access Tokens (PAT) here for release permissions, and instead elevating permissions on [GITHUB_TOKEN](https://docs.github.com/en/actions/security-guides/automatic-token-authentication#modifying-the-permissions-for-the-github_token) for `contents` scope (required for an action to make a release). We think this is reasonable considering this repo is locked down in terms of number of contributors with write access, and that this is scope specific and not a write-all.
1 parent fd84f68 commit 3c6d32e

File tree

2 files changed

+92
-0
lines changed

2 files changed

+92
-0
lines changed
+19
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
name: "Setup build environment"
2+
description: "Setup python, ninja-build etc."
3+
4+
# Make top-level permissions on GITHUB_TOKEN more restrictive
5+
permissions:
6+
contents: read
7+
8+
runs:
9+
using: "composite"
10+
11+
steps:
12+
- name: Set up Python
13+
uses: actions/setup-python@v5
14+
with:
15+
python-version: '3.11'
16+
17+
- name: Install prerequisites (Linux)
18+
run: sudo apt-get install --yes ninja-build
19+
shell: bash
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,73 @@
1+
name: Build Release and Publish
2+
3+
on:
4+
workflow_dispatch:
5+
schedule:
6+
# Runs at 11:00 AM UTC, which is 3:00 AM PST (UTC-8)
7+
- cron: '0 11 * * *'
8+
9+
# Ensure that only a single job or workflow using the same
10+
# concurrency group will run at a time. This would cancel
11+
# any in-progress jobs in the same github workflow and github
12+
# ref (e.g. refs/heads/main or refs/pull/<pr_number>/merge).
13+
concurrency:
14+
group: ${{ github.workflow }}-${{ github.ref }}
15+
cancel-in-progress: true
16+
17+
jobs:
18+
build_linux:
19+
name: Linux x86_64 Build
20+
runs-on: ubuntu-latest
21+
# Elevates permissions for `GITHUB_TOKEN`'s content scope (to allow this action to make a release)
22+
permissions:
23+
contents: write
24+
# Don't run this in everyone's forks.
25+
if: github.repository == 'llvm/torch-mlir-release'
26+
strategy:
27+
matrix:
28+
package: [torch-mlir]
29+
py_version: [cp310-cp310, cp311-cp311]
30+
31+
steps:
32+
- name: Checkout torch-mlir
33+
uses: actions/checkout@v4
34+
with:
35+
repository: llvm/torch-mlir
36+
ref: refs/heads/main
37+
submodules: 'true'
38+
39+
- name: Setup workspace
40+
uses: ./.github/actions/setup-build
41+
42+
- name: Build Python wheels and smoke test
43+
run: |
44+
cd $GITHUB_WORKSPACE
45+
tm_package_version="$(printf '%(%Y%m%d)T.${{ github.run_number }}')"
46+
echo "tm_package_version=${tm_package_version}" >> $GITHUB_ENV
47+
printf "TORCH_MLIR_PYTHON_PACKAGE_VERSION=%s\n" ${tm_package_version} > ./torch_mlir_package_version
48+
TM_PYTHON_VERSIONS=${{ matrix.py_version }} TM_PACKAGES=${{ matrix.package }} ./build_tools/python_deploy/build_linux_packages.sh
49+
50+
- name: Make assets available in dist
51+
run: |
52+
mkdir dist
53+
cp build_tools/python_deploy/wheelhouse/torch*.whl dist/
54+
55+
- name: Upload python wheels
56+
uses: actions/upload-artifact@v4
57+
with:
58+
if-no-files-found: error
59+
name: snapshot-${{ matrix.package }}-${{ matrix.py_version }}-${{ env.tm_package_version }}
60+
path: dist
61+
62+
- name: Release python wheels
63+
uses: ncipollo/[email protected]
64+
with:
65+
artifacts: dist/*.whl
66+
token: "${{ secrets.GITHUB_TOKEN }}"
67+
tag: "dev-wheels"
68+
name: "dev-wheels"
69+
body: "Automatic snapshot release of torch-mlir python wheels."
70+
removeArtifacts: false
71+
allowUpdates: true
72+
replacesArtifacts: true
73+
makeLatest: true

0 commit comments

Comments
 (0)