Skip to content

Commit ad93dca

Browse files
mckornfieldkendrickb-nvidia
mckornfield
and
kendrickb-nvidia
authored
chore(ci): refactor wheel release process (#312)
* The NeMoFW release job currently pushes commits, which doesn't make sense with uv's tagging conventions, switched us to a local version that just builds the wheel and * Also creates a github release --------- Signed-off-by: Matt Kornfield <mkornfield@nvidia.com> Signed-off-by: Matt Kornfield <mckornfield@gmail.com> Co-authored-by: Kendrick Boyd <kendrickb@nvidia.com>
1 parent bd044b0 commit ad93dca

4 files changed

Lines changed: 153 additions & 131 deletions

File tree

‎.github/workflows/README.md‎

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -15,8 +15,7 @@ All workflows that use `.github/actions/setup-python-env` now default to the ver
1515
| [gpu-tests.yml](gpu-tests.yml) | Push to `main`/`pull-request/*`, manual | GPU smoke tests (required) and E2E tests (A100) |
1616
| [conventional-commit.yml](conventional-commit.yml) | PRs | Validates PR titles follow conventional commit format |
1717
| [docs.yml](docs.yml) | Push to `main` (docs paths) | Builds and deploys documentation to GitHub Pages |
18-
| [internal-release.yml](internal-release.yml) | Tag push (`v[0-9]*`), manual dispatch | Builds and publishes wheel to Artifactory or PyPI |
19-
| [release.yml](release.yml) | Manual dispatch | Builds and publishes package to PyPI (production) |
18+
| [release.yml](release.yml) | Manual dispatch | Builds and publishes package to Test PyPI or PyPI (production) |
2019
| [secrets-detector.yml](secrets-detector.yml) | PRs | Scans for accidentally committed secrets |
2120

2221
## Pull Request Testing (copy-pr-bot)

‎.github/workflows/internal-release.yml‎

Lines changed: 0 additions & 102 deletions
This file was deleted.

‎.github/workflows/release.yml‎

Lines changed: 106 additions & 27 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,13 @@
1212
# See the License for the specific language governing permissions and
1313
# limitations under the License.
1414

15+
# ---------------------------------------------------------------------------
16+
# Release: build wheel, publish to PyPI (or Test PyPI on dry-run), and
17+
# optionally create a GitHub release.
18+
#
19+
# No version-bump PR, no docs upload, no dummy branch push.
20+
# ---------------------------------------------------------------------------
21+
1522
name: "Release NeMo Safe Synthesizer"
1623

1724
on:
@@ -22,7 +29,7 @@ on:
2229
required: true
2330
type: string
2431
dry-run:
25-
description: "Dry Run: only publish to test PyPI"
32+
description: "Dry Run: only publish to Test PyPI"
2633
required: true
2734
default: true
2835
type: boolean
@@ -31,34 +38,106 @@ on:
3138
required: true
3239
default: true
3340
type: boolean
34-
version-bump-branch:
35-
description: Branch to push the version bump
36-
required: true
37-
type: string
41+
42+
defaults:
43+
run:
44+
shell: bash -x -e -u -o pipefail {0}
3845

3946
permissions:
4047
contents: write
41-
pull-requests: write
4248

4349
jobs:
44-
release:
45-
uses: NVIDIA-NeMo/FW-CI-templates/.github/workflows/_release_library.yml@v0.81.1
46-
with:
47-
release-ref: ${{ inputs.release-ref }}
48-
python-package: nemo_safe_synthesizer
49-
library-name: NeMo Safe Synthesizer
50-
dry-run: ${{ inputs.dry-run }}
51-
version-bump-branch: ${{ inputs.version-bump-branch }}
52-
create-gh-release: ${{ inputs.create-gh-release }}
53-
packaging: uv
54-
has-src-dir: true
55-
app-id: ${{ vars.BOT_ID }}
56-
secrets:
57-
TWINE_USERNAME: ${{ secrets.TWINE_USERNAME }}
58-
TWINE_PASSWORD: ${{ secrets.TWINE_PASSWORD }}
59-
SLACK_WEBHOOK_ADMIN: ${{ secrets.SLACK_WEBHOOK_ADMIN }}
60-
SLACK_WEBHOOK: ${{ secrets.SLACK_RELEASE_ENDPOINT }}
61-
PAT: ${{ secrets.PAT }}
62-
SSH_KEY: ${{ secrets.SSH_KEY }}
63-
SSH_PWD: ${{ secrets.SSH_PWD }}
64-
BOT_KEY: ${{ secrets.BOT_KEY }}
50+
publish-wheel:
51+
name: Publish wheel (${{ inputs.dry-run && 'Test PyPI' || 'PyPI' }})
52+
runs-on: linux-amd64-cpu4
53+
outputs:
54+
version: ${{ steps.build.outputs.version }}
55+
steps:
56+
- name: Checkout at release ref
57+
uses: actions/checkout@v6
58+
with:
59+
ref: ${{ inputs.release-ref }}
60+
fetch-depth: 0
61+
fetch-tags: true
62+
63+
- name: Install uv
64+
uses: astral-sh/setup-uv@v6
65+
with:
66+
enable-cache: true
67+
68+
- name: Set up Python
69+
uses: actions/setup-python@v6
70+
with:
71+
python-version-file: ".python-version"
72+
73+
- name: Build wheel
74+
id: build
75+
run: |
76+
make build-wheel
77+
WHEEL=$(ls dist/*.whl)
78+
VERSION=$(echo "$WHEEL" | sed -n 's/.*-\([0-9][^-]*\)-.*/\1/p')
79+
echo "version=$VERSION" >> "$GITHUB_OUTPUT"
80+
echo "Built: $WHEEL (version $VERSION)"
81+
82+
- uses: actions/upload-artifact@v7
83+
with:
84+
name: nemo-safe-synthesizer-${{ steps.build.outputs.version }}
85+
path: dist/*.whl
86+
retention-days: 90
87+
88+
- name: Publish to Test PyPI (always, as pre-flight)
89+
env:
90+
TWINE_USERNAME: __token__
91+
TWINE_PASSWORD: ${{ secrets.TEST_PYPI_PERSONAL_TOKEN }}
92+
run: |
93+
uvx twine upload \
94+
--repository-url https://test.pypi.org/legacy/ \
95+
--skip-existing \
96+
--non-interactive \
97+
--verbose \
98+
dist/*.whl
99+
100+
- name: Publish to PyPI
101+
if: ${{ !inputs.dry-run }}
102+
env:
103+
TWINE_USERNAME: __token__
104+
TWINE_PASSWORD: ${{ secrets.PYPI_PERSONAL_TOKEN }}
105+
run: |
106+
uvx twine upload \
107+
--non-interactive \
108+
--verbose \
109+
dist/*.whl
110+
111+
create-gh-release:
112+
name: Create GitHub release
113+
needs: publish-wheel
114+
if: ${{ inputs.create-gh-release }}
115+
runs-on: ubuntu-latest
116+
steps:
117+
- name: Checkout at release ref
118+
uses: actions/checkout@v6
119+
with:
120+
ref: ${{ inputs.release-ref }}
121+
fetch-depth: 0
122+
123+
- name: Download wheel artifact
124+
uses: actions/download-artifact@v4
125+
with:
126+
name: nemo-safe-synthesizer-${{ needs.publish-wheel.outputs.version }}
127+
path: dist/
128+
129+
- name: Create GitHub release
130+
env:
131+
GH_TOKEN: ${{ github.token }}
132+
VERSION: ${{ needs.publish-wheel.outputs.version }}
133+
run: |
134+
PRERELEASE_FLAG=""
135+
if echo "$VERSION" | grep -qE '(rc|alpha|beta|\.dev)'; then
136+
PRERELEASE_FLAG="--prerelease"
137+
fi
138+
139+
gh release create "v${VERSION}" \
140+
dist/*.whl \
141+
--title "v${VERSION}" \
142+
--notes-file CHANGELOG.md \
143+
$PRERELEASE_FLAG

‎CONTRIBUTING.md‎

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,7 @@ Please read our [Code of Conduct](CODE_OF_CONDUCT.md) before contributing.
2020
- [Code Style](#code-style)
2121
- [Documentation](#documentation)
2222
- [AI Agents](#ai-agents)
23+
- [Releasing](#releasing)
2324
- [NMP Integration](#nmp-integration)
2425

2526
## Getting Started
@@ -582,6 +583,51 @@ Conventions defined in `AGENTS.md` (code style, markdown style, testing, etc.) a
582583

583584
Before contributing, run `make format` and `make check`. See `AGENTS.md` for full conventions.
584585

586+
## Releasing
587+
588+
Releases are published to PyPI via the **Release NeMo Safe Synthesizer** GitHub Actions workflow. The workflow builds the wheel from a git tag, publishes to Test PyPI as a pre-flight check, and optionally publishes to the real PyPI and creates a GitHub release.
589+
590+
### 1. Create and push a tag
591+
592+
Tags must follow [Semantic Versioning](#semantic-versioning-for-tags). For release candidates, use the `rcN` pre-release suffix:
593+
594+
```bash
595+
# Stable release
596+
git tag 0.1.0 <commit-sha>
597+
598+
# Release candidate
599+
git tag 0.1.0rc1 <commit-sha>
600+
601+
git push origin <tag>
602+
```
603+
604+
### 2. Run the workflow
605+
606+
Trigger the workflow from the GitHub UI or via `gh`:
607+
608+
**GitHub UI:** Go to **Actions → Release NeMo Safe Synthesizer → Run workflow** and fill in the inputs.
609+
610+
**CLI:**
611+
612+
```bash
613+
gh workflow run release.yml \
614+
--field release-ref=<tag> \
615+
--field dry-run=true \
616+
--field create-gh-release=false
617+
```
618+
619+
### 3. Inputs
620+
621+
| Input | Description | Default |
622+
|---|---|---|
623+
| `release-ref` | Full SHA or tag to build from | required |
624+
| `dry-run` | Publish to Test PyPI only; skip real PyPI | `true` |
625+
| `create-gh-release` | Create a GitHub release and attach the wheel | `true` |
626+
627+
**Dry-run mode** (default): builds the wheel and publishes it to [Test PyPI](https://test.pypi.org/) only. Use this to verify the package before a real release. The workflow always publishes to Test PyPI regardless of `dry-run`.
628+
629+
**Real release**: uncheck `dry-run` (or pass `--field dry-run=false`) to also publish to the real [PyPI](https://pypi.org/). Pre-release tags (`rc`, `alpha`, `beta`, `.dev`) are automatically marked as pre-releases on both PyPI and GitHub.
630+
585631
## NMP Integration
586632

587633
NeMo Safe Synthesizer is developed as a standalone package and published to PyPI and optionally published to the internal NVIDIA Artifactory. The NeMo Platform (NMP) consumes it as an external dependency.

0 commit comments

Comments
 (0)