Skip to content

Commit b6a01b3

Browse files
committed
Update OnnxRelease document
Signed-off-by: Andreas Fehlner <[email protected]>
1 parent 6808589 commit b6a01b3

File tree

1 file changed

+61
-42
lines changed

1 file changed

+61
-42
lines changed

docs/OnnxReleases.md

Lines changed: 61 additions & 42 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,8 @@ SPDX-License-Identifier: Apache-2.0
88

99
The ONNX project, going forward, will plan to release roughly on a four month cadence. We follow the [Semver](https://semver.org/) versioning approach and will make decisions as a community on a release by release basis on whether to do a major or minor release.
1010

11+
12+
1113
## Preparation
1214
* Determine version (X.Y.Z) for the new release
1315
* Discuss in Slack channel for Releases (https://lfaifoundation.slack.com/archives/C018VGGJUGK)
@@ -30,52 +32,14 @@ The ONNX project, going forward, will plan to release roughly on a four month ca
3032
1. Make sure all tests pass on the new branch.
3133

3234
* After cutting the release branch:
33-
1. Create PR to set [VERSION_NUMBER](/VERSION_NUMBER) file in `main` to the next future releas, `X.Y+1.0`.
35+
1. Create PR to set [VERSION_NUMBER](/VERSION_NUMBER) file in `main` to the next future release, `X.Y+1.0`.
3436
1. Create PR to set `VERSION_NUMBER` file in the new release's branch to `X.Y.Zrc1`.
3537
* For example the first release candidate for 1.16.0 would be `1.16.0rc1`
3638
1. Bump opset version for ai.onnx domain in `onnx/defs/operator_sets.h` and `onnx/defs/schema.h` for use by future operator additions and changes.
3739
* For example, this [demo PR](https://github.com/onnx/onnx/pull/6001).
3840

39-
## Upload release candidate to TestPyPI
40-
41-
**Important**
42-
* **WAIT** for PR to set the release branch's `VERSION_NUMBER` to merge and build before continuing.
43-
* To push files to TestPyPI or PyPI, install `twine` if you don't already have it: `pip install twine`
44-
* When prompted for a password by `twine` commands, use an API token. Your password will not work.
45-
* Note: TestPyPI and PyPI are separate accounts so make sure you are using the correct one depending where you are uploading.
46-
* [Generate a TestPYI API token](https://test.pypi.org/manage/account/token/)
47-
* [Generate a PyPI API token](https://pypi.org/manage/account/token/)
48-
* Like PyPI, A release version can only be pushed to TestPyPI **ONCE**.
49-
* To update an already pushed file, you must increase the `VERSION_NUMBER`, rebuild, and push a new X.Y.Zrc2, etc.
50-
* To test push commands, you can use docker or podman to create a local pypi server
51-
1. Start server `docker run --rm -it --platform linux/amd64 -p 80:8080 pypiserver/pypiserver:latest run -a . -P .`
52-
* This starts a local pypiserver that does not require authentication (any user/password will work on it).
53-
* The container does not save state. Stopping and starting it again will let you push the same version multiple times.
54-
1. To push files:
55-
* wheels: `twine upload --repository-url http://127.0.0.1:80 --verbose -u fake -p fake *.whl`
56-
* source: `twine upload --repository-url http://127.0.0.1:80 --verbose -u fake -p fake dist/*`
57-
1. To pull and install from your test server:
58-
* `pip uninstall -y onnx && pip install --index-url http://127.0.0.1:80/simple/ --pre onnx`
59-
60-
**Push Wheels**
61-
1. Gather the wheel files from the ONNX Github Actions for the release candidate.
62-
* For each ONNX GitHub Action:
63-
* ONNX GitHub Action
64-
* [LinuxRelease_x86_64](https://github.com/onnx/onnx/actions/workflows/release_linux_x86_64.yml)
65-
* [LinuxRelease_aarch64](https://github.com/onnx/onnx/actions/workflows/release_linux_aarch64.yml)
66-
* [MacRelease](https://github.com/onnx/onnx/actions/workflows/release_mac.yml)
67-
* [WindowsRelease](https://github.com/onnx/onnx/actions/workflows/release_win.yml)
68-
69-
* Find the run for the release branch
70-
* Or start a run by clicking "Run workflow", pick the release branch, Click "Run Workflow"
71-
* Click the completed run, scroll to the "Artifacts" section (bottom), and click "wheels" to download the files
72-
* Extract the wheels.zip files and combine their contents into a single folder
73-
74-
2. Upload the produced wheels manually to TestPyPI: `twine upload --repository testpypi --verbose -u <YOUR_TESTPYPI_USER> <extracted_wheel.zip_folder>/*.whl`.
75-
* A current owner of the ONNX project will need to give you access to the project before you can push files.
76-
* The project name and version built into the files.
77-
78-
**Source Distribution**
41+
42+
** Source Distribution (TODO: this has to also made by a step of the release pipeline **
7943
1. Make sure all the git submodules are updated
8044
* ``git submodule update --init``
8145
1. Make sure the git checkout is clean –
@@ -133,7 +97,54 @@ The ONNX project, going forward, will plan to release roughly on a four month ca
13397
* If issues are found, the bugs are to be fixed in the onnx `main` branch and then cherry-picked into the release branch.
13498
* Follow up with reporter to ensure issues are resolved (and validated in a new rc) or deferred to a new release.
13599

136-
# Official Release
100+
101+
# Official Release-Workflow
102+
103+
104+
```mermaid
105+
graph TB
106+
subgraph GitHub GUI
107+
A[Trigger: Draft a new Release in Github GUI and select the 1.X.X Branch] --> B[SaveDraft or Publish Release?]
108+
end
109+
110+
subgraph CI Pipeline
111+
B --> D1[create_release.yml is called]
112+
D1 --> D{All 4 different OSPipelines were successful?}
113+
D -->|Yes| E[Job 3: Call prepare-release job]
114+
D -->|No| F[Job: Send Failure Notification]
115+
E --> G1{Collect artifacts}
116+
G1 --> G[Job 4: Prepare Release Workflow]
117+
end
118+
119+
subgraph prepare-release Workflow
120+
G --> H[Job 5: Download all artifacts]
121+
H --> I[Job 6: Sign all artifacts with sigstore]
122+
end
123+
124+
subgraph Integration Testing
125+
I --> J[Job 7: External Integration Tests]
126+
J --> K{Integration Tests Passed?}
127+
K -->|Yes| L[Job 8: Manual Approval for Release]
128+
K -->|No| M[Job: Rollback & Investigate]
129+
M --> F
130+
end
131+
132+
subgraph Release
133+
L --> N{Is it an official release or a rc-candidate?}
134+
N --> |Yes| O1[Publish to PyPI]
135+
N --> |No| O2[Publish to alpha/beta to TestPyPI]
136+
O1 --> O[Job 10: Add signature files to GithubRelease]
137+
O --> P[Job 10: Tag Release in GitHub]
138+
P --> Q1[Job 11: Create Release Notes]
139+
Q1 --> Q[Job 12: Add artifacts to github release]
140+
end
141+
142+
143+
Q --> R[End Pipeline]
144+
F --> R
145+
M --> R
146+
```
147+
137148

138149
Validation steps must be completed before this point! This is the point of new return.
139150

@@ -157,6 +168,14 @@ Validation steps must be completed before this point! This is the point of new r
157168
* Add any additional commits that merged into the release in since wiki was written.
158169
* .tar.gz and .zip will be auto-generated after publishing the release.
159170

171+
* a) The pipelines for different os variants start to run and save the wheels for all OS Version. Only if all all 4 release pipelines were successful the next preparation step is started
172+
* b) During the preparation step of the pipeline, the artifacts are cryptographically signed with sigstore for supply chain security reasons
173+
* c) The step is paused (until the deployment was reviewed and the defined reviewer approve the release). At this step the whl could also be downloaded for offline-testing. We use the feature of [deployment environment](https://docs.github.com/en/actions/deployment/targeting-different-environments/using-environments-for-deployment) here.
174+
* d) After the The final step is about
175+
i) publishing the wheels and tar.gz files to official pypi
176+
ii) adding the signing information to the official github release
177+
178+
160179
## Upload to Official PyPI
161180
### NOTES:
162181
* Once the packages are uploaded to PyPI, **you cannot overwrite it on the same PyPI instance**.

0 commit comments

Comments
 (0)