Skip to content

Commit 152f0b9

Browse files
u-simhau-simha
authored
Merge branch 'main' into fix/graph-root-module-weight
2 parents f17384f + a979d66 commit 152f0b9

55 files changed

Lines changed: 3409 additions & 668 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/workflows/release.yml‎

Lines changed: 154 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,154 @@
1+
name: Release
2+
3+
run-name: 'Release ${{ github.ref_name }} · @${{ github.actor }}'
4+
5+
# Publishing to PyPI uses Trusted Publishing (OIDC) — no API token is stored.
6+
# A release manager pushes a `vMAJOR.MINOR.PATCH` tag; the `build` and
7+
# `smoke-test` jobs run automatically, then the `publish` job pauses on the
8+
# `pypi` GitHub environment until a reviewer approves it. That environment's
9+
# protection rules (required reviewers, prevent-self-review so the tag pusher
10+
# can't approve their own release, and a wait timer) live in the repo's
11+
# Environment settings, not in this file.
12+
on:
13+
push:
14+
tags:
15+
# Strictly vMAJOR.MINOR.PATCH with numeric parts (e.g. v0.2.2). This is a
16+
# glob, not a regex: `.` is a literal dot and `[0-9]` a digit range. The
17+
# filter must match the entire tag, so pre-releases (v1.2.3rc1 — the
18+
# trailing `rc1` is left unmatched) and other non-release tags never start
19+
# the release run. The `pypi` environment tag rule and approval gate are
20+
# secondary controls; the version guard below is the final backstop.
21+
- 'v[0-9]+.[0-9]+.[0-9]+'
22+
# Manual dry run: builds and smoke-tests the current ref but never publishes
23+
# (the publish job is gated to tag pushes). Trigger from the Actions tab
24+
# ("Release" -> "Run workflow") or `gh workflow run release.yml --ref <branch>`.
25+
workflow_dispatch:
26+
27+
# Least privilege by default; the publish job opts into `id-token: write`.
28+
permissions:
29+
contents: read
30+
31+
concurrency:
32+
# Serialize releases per tag and never cancel an in-flight publish.
33+
group: release-${{ github.ref }}
34+
cancel-in-progress: false
35+
36+
jobs:
37+
# ── Build the exact wheel + sdist that will be smoke-tested and published. ──
38+
build:
39+
name: Build distributions
40+
runs-on: ubuntu-latest
41+
timeout-minutes: 15
42+
steps:
43+
- name: Check out repository
44+
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
45+
with:
46+
persist-credentials: false
47+
- name: Install uv
48+
uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
49+
with:
50+
enable-cache: false
51+
- name: Verify the tag matches the package version
52+
# The published version comes from src/coreai_opt/_about.py, not the tag.
53+
# Fail early if they disagree so we never publish a mismatched/duplicate
54+
# version (PyPI uploads are immutable and cannot be overwritten).
55+
# `print_version.py --release` computes the version exactly as the
56+
# `make build` step below does, so this guard can't drift from what
57+
# actually gets published.
58+
# Skipped on manual dry runs, where the ref is a branch, not a vX.Y.Z tag.
59+
# Run via uv (installed above) so the interpreter satisfies
60+
# requires-python whatever the runner image ships; see the Makefile's
61+
# `version` target.
62+
if: github.event_name == 'push'
63+
run: |
64+
tag="${GITHUB_REF_NAME}"
65+
version="$(uv run --no-config --no-project --python '>=3.11' scripts/make/print_version.py --release)"
66+
echo "tag=${tag} package version=${version}"
67+
if [ "${tag}" != "v${version}" ]; then
68+
echo "::error::Tag ${tag} does not match package version v${version} (src/coreai_opt/_about.py). Update latest_released_version so the release it implies matches the tag."
69+
exit 1
70+
fi
71+
- name: Build wheel and sdist
72+
run: make build
73+
- name: Upload distributions
74+
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
75+
with:
76+
name: dist
77+
path: dist/
78+
if-no-files-found: error
79+
80+
# ── Smoke test the exact built wheel and sdist via `make test-smoke`. ──
81+
# Reuses the repo's smoke suite (ci/nox/noxfile.py → tests/test_smoke.py) but
82+
# points it at the pre-built artifact instead of rebuilding, so we test the
83+
# bytes we are about to publish. `make test-smoke` runs across every supported
84+
# Python version internally.
85+
smoke-test:
86+
name: Smoke test (${{ matrix.format }}, ${{ matrix.torch_group }})
87+
needs: build
88+
runs-on: ubuntu-latest
89+
timeout-minutes: 60
90+
env:
91+
INSTALL_PRECOMMIT: 'false'
92+
strategy:
93+
fail-fast: false
94+
matrix:
95+
# Test both distribution formats against every supported torch version,
96+
# mirroring the PR CI smoke matrix (ci.yaml).
97+
format: [wheel, sdist]
98+
torch_group: [torch_2_8, torch_2_9, torch_2_10, torch_2_11]
99+
steps:
100+
- name: Check out repository
101+
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
102+
with:
103+
persist-credentials: false
104+
- name: Install uv
105+
uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
106+
with:
107+
enable-cache: false
108+
- name: Download distributions
109+
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
110+
with:
111+
name: dist
112+
path: dist/
113+
- name: Smoke test the built ${{ matrix.format }} against ${{ matrix.torch_group }}
114+
run: |
115+
# Expand the glob to the single built artifact; the `test -f` below
116+
# fails the job if it's missing or if more than one matched.
117+
case "${{ matrix.format }}" in
118+
wheel) dist="$(echo dist/*.whl)" ;;
119+
sdist) dist="$(echo dist/*.tar.gz)" ;;
120+
esac
121+
test -f "${dist}" || { echo "::error::Expected exactly one ${{ matrix.format }} in dist/"; exit 1; }
122+
echo "Smoke testing ${dist} against ${{ matrix.torch_group }}"
123+
make test-smoke SMOKE_TEST_DIST="${dist}" TORCH_GROUP="${{ matrix.torch_group }}"
124+
125+
# ── Publish to PyPI via Trusted Publishing. Only this job holds `id-token`. ──
126+
# It builds nothing and runs no project code: it just downloads the vetted
127+
# artifact and uploads it, keeping build/test dependencies out of the
128+
# OIDC-privileged job.
129+
publish:
130+
name: Publish to PyPI
131+
needs: [build, smoke-test]
132+
# Publish only on a tag push (never on a manual dry run) and never from forks.
133+
if: github.event_name == 'push' && github.repository == 'apple/coreai-optimization'
134+
runs-on: ubuntu-latest
135+
timeout-minutes: 15
136+
environment:
137+
name: pypi
138+
url: https://pypi.org/p/coreai-opt
139+
permissions:
140+
id-token: write # mint the OIDC token PyPI validates for Trusted Publishing
141+
contents: read
142+
steps:
143+
- name: Install uv
144+
uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
145+
with:
146+
enable-cache: false
147+
- name: Download distributions
148+
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
149+
with:
150+
name: dist
151+
path: dist/
152+
- name: Publish to PyPI
153+
# `always` requires Trusted Publishing (OIDC) — no fallback to tokens.
154+
run: uv publish --trusted-publishing always

‎.pre-commit-config.yaml‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -305,6 +305,34 @@ repos:
305305
types: [python]
306306
exclude: ^tests/
307307

308+
- repo: local
309+
hooks:
310+
- id: check-internal-import-aliases
311+
name: Check `_`-prefix aliasing of internal imports
312+
description: |
313+
Enforce code_style_guide.md §3.3: public modules must alias symbols
314+
imported from private modules with a `_` prefix; private modules must
315+
not. Fails the commit and reports each violation with a suggested fix;
316+
it does not edit files — run the script with `--fix` to apply fixes.
317+
entry: python scripts/pre_commit/check_internal_import_aliases.py
318+
language: system
319+
types: [python]
320+
exclude: ^tests/
321+
322+
- repo: local
323+
hooks:
324+
- id: check-about-version
325+
name: Check _about.py version fields
326+
description: |
327+
Check that _about.py's latest_released_version matches the repo's
328+
latest release tag, and that __version__ is its last number plus
329+
one, plus .dev0. Catches a release candidate that looks like a
330+
release has already shipped when it hasn't.
331+
entry: python scripts/pre_commit/check_about_version.py
332+
language: system
333+
files: (^|/)_about\.py$
334+
pass_filenames: false
335+
308336
- repo: local
309337
hooks:
310338
- id: towncrier-check

‎CONTRIBUTING.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -22,8 +22,8 @@ The API surface is intentionally limited. This keeps the library reliable, well-
2222
Set up the environment as described in [README.md](README.md#getting-started). Then, from the activated venv:
2323

2424
```shell
25-
# Build the package.
26-
make build
25+
# Build the package (development build).
26+
make build-dev
2727

2828
# Build the documentation.
2929
make docs

‎Makefile‎

Lines changed: 64 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33
# Use of this source code is governed by a BSD-3-Clause license that can
44
# be found in the LICENSE file or at https://opensource.org/licenses/BSD-3-Clause
55

6-
.PHONY: _maybe_patch_pyproject all api-list build check clean distclean distclean-all docs docs-clean docs-open env env-all env-docs env-highest-torch env-lowest-torch env-tutorial render-api-index set-auto-venv test test-cov test-fast test-highest-pytorch test-lowest-pytorch test-slow test-smoke test-tutorials version
6+
.PHONY: _maybe_patch_pyproject all api-list build build-dev check clean distclean distclean-all docs docs-clean docs-open env env-all env-docs env-highest-torch env-lowest-torch env-tutorial render-api-index set-auto-venv test test-cov test-fast test-highest-pytorch test-lowest-pytorch test-slow test-smoke test-tutorials version
77

88
SHELL := /bin/bash
99

@@ -64,6 +64,13 @@ LOWEST_TORCH_GROUP := torch_2_8
6464
TORCH_GROUP ?= $(HIGHEST_TORCH_GROUP)
6565
export TORCH_GROUP
6666

67+
# Optional path to a pre-built distribution (wheel or sdist) for `test-smoke` to
68+
# install instead of building from source; consumed by the nox smoke session via
69+
# $SMOKE_TEST_DIST (empty = build from source). Exported like TORCH_GROUP so it
70+
# reaches the nox subprocess.
71+
SMOKE_TEST_DIST ?=
72+
export SMOKE_TEST_DIST
73+
6774
# Documentation directory. Defaults to $(MAKEFILE_DIR)docs so the same recipe
6875
# works in both contexts:
6976
#
@@ -88,6 +95,25 @@ SHELL_RC ?=
8895
# internal-only venv defaults.
8996
ENV_ALL_EXTRAS ?= true
9097

98+
# Extra `uv pip install` arguments, applied to a venv after it is synced and to
99+
# each nox session venv (see ci/nox/noxfile.py). Use it to swap a dependency
100+
# version for one run without editing pyproject.toml or the lockfile — for
101+
# example a scheduled job testing against newer upstream builds. Empty (the
102+
# default) changes nothing.
103+
POST_INSTALL_PIP_ARGS ?=
104+
export POST_INSTALL_PIP_ARGS
105+
106+
# Re-apply POST_INSTALL_PIP_ARGS to the venv at $(1).
107+
#
108+
# setup_env.sh already does this, so only use it when a recipe installs more
109+
# packages afterwards that could pull the old version back in (see env-all).
110+
# Expands to `true` when unset. The empty check uses `$(if ...)` instead of a
111+
# shell `[ -n "..." ]` test because the args contain their own quotes.
112+
# Usage: $(call post_install_pip,VENV_PATH)
113+
define post_install_pip
114+
$(if $(POST_INSTALL_PIP_ARGS),echo "Applying POST_INSTALL_PIP_ARGS to $(1)" && source $(1)/bin/activate && uv pip install $(POST_INSTALL_PIP_ARGS),true)
115+
endef
116+
91117
# Local wheelhouse for pre-release wheels not yet on an index.
92118
# Exported so every recipe-level uv invocation (uv lock, uv sync, uv venv)
93119
# resolves matching packages from disk without an index lookup. The wildcard
@@ -174,7 +200,7 @@ endif
174200
# =============================================================================
175201

176202
# Default target - run full workflow
177-
all: clean distclean-all env-all check test-lowest-pytorch test-highest-pytorch build
203+
all: clean distclean-all env-all check test-lowest-pytorch test-highest-pytorch build-dev
178204

179205
# =============================================================================
180206
# Environment Setup
@@ -205,14 +231,29 @@ env-all: _maybe_patch_pyproject
205231
@$(SETUP_ENV) --venv $(VENV) --python-version $(PYTHON_VERSION) --all-groups
206232
@$(call write_active_venv,$(VENV))
207233
@$(ENV_ALL_EXTRAS)
234+
@$(call post_install_pip,$(VENV))
208235

209236
# =============================================================================
210237
# Build
211238
# =============================================================================
212239

213-
# Build package
240+
# Build the canonical, publishable distribution (wheel + sdist): the on-tree
241+
# version with any `.dev` suffix stripped (e.g. 0.2.2.dev0 -> 0.2.2), via
242+
# `uv build --no-sources`. `--no-sources` ignores [tool.uv.sources], so the
243+
# artifact doesn't depend on uv-specific index overrides — the recommended way
244+
# to build for publication. This is what the release workflow runs. Routed
245+
# through build.py (like build-dev) so both targets share one code path; set
246+
# COREAI_OPT_VERSION_EXTENSION to insert an extra release segment (see
247+
# RELEASE.md).
214248
build:
215-
@$(call use_env,VENV) && uv run --no-sync --active python $(SCRIPTS)/make/build.py
249+
@$(call use_env,VENV) && uv run --no-sync --active python $(SCRIPTS)/make/build.py --no-sources
250+
251+
# Build a development distribution with build.py: the release base with a
252+
# unique, timestamped PEP 440 dev suffix (e.g. 0.2.2.dev202607231430+abc1234).
253+
# Used by contributors, the smoke tests, and the nightly pipeline. Set
254+
# DEV_VERSION=... to use an exact version instead.
255+
build-dev:
256+
@$(call use_env,VENV) && uv run --no-sync --active python $(SCRIPTS)/make/build.py --dev
216257

217258
# =============================================================================
218259
# Code Quality
@@ -252,25 +293,33 @@ test-slow:
252293

253294
# Run smoke tests only (pass PYTEST_ARGS for custom flags, e.g., make test-smoke PYTEST_ARGS="--junitxml=results.xml").
254295
# Pass TORCH_GROUP to smoke test against a specific torch version (default: HIGHEST_TORCH_GROUP).
296+
# Pass SMOKE_TEST_DIST=<path to a .whl or .tar.gz> to smoke test a pre-built
297+
# distribution instead of building one from source (used by the release
298+
# workflow to test the exact artifact being published).
255299
test-smoke:
256300
@$(call use_env,VENV) && \
257301
echo "Running smoke tests..." && \
258302
uv run --no-sync --active nox -f $(MAKEFILE_DIR)ci/nox/noxfile.py -s smoke_tests -- $(PYTEST_ARGS) && \
259303
echo "All smoke tests passed!"
260304

261-
# Run tests on lowest supported PyTorch version (pass PYTEST_ARGS for custom flags)
262-
test-lowest-pytorch: env-lowest-torch
305+
# Run tests on lowest supported PyTorch version (pass PYTEST_ARGS for custom flags).
306+
# TORCH_GROUP is already exported, so setting it per target is enough for
307+
# use_env to pick the right torch build. Use `=`, not `:=`: an including
308+
# Makefile may change HIGHEST_TORCH_GROUP after this file is read.
309+
test-lowest-pytorch: TORCH_GROUP = $(LOWEST_TORCH_GROUP)
310+
test-lowest-pytorch:
263311
@echo "Running tests on lowest PyTorch version supported..."
264-
@source $(VENV_LOWEST_TORCH)/bin/activate && \
312+
@$(call use_env,VENV_LOWEST_TORCH) && \
265313
echo "Testing with lowest supported PyTorch versions" && \
266314
uv run --no-sync --active python $(SCRIPTS)/make/log_versions.py && \
267315
$(RUN_TESTS) $(PYTEST_ARGS) && \
268316
echo "All tests passed!"
269317

270318
# Run tests on highest supported PyTorch version (pass PYTEST_ARGS for custom flags)
271-
test-highest-pytorch: env-highest-torch
319+
test-highest-pytorch: TORCH_GROUP = $(HIGHEST_TORCH_GROUP)
320+
test-highest-pytorch:
272321
@echo "Running tests on highest PyTorch version supported..."
273-
@source $(VENV_HIGHEST_TORCH)/bin/activate && \
322+
@$(call use_env,VENV_HIGHEST_TORCH) && \
274323
echo "Testing with latest supported PyTorch versions" && \
275324
uv run --no-sync --active python $(SCRIPTS)/make/log_versions.py && \
276325
$(RUN_TESTS) $(PYTEST_ARGS) && \
@@ -307,9 +356,13 @@ distclean-all:
307356
set-auto-venv:
308357
@$(SCRIPTS)/make/set_auto_venv.sh $(DEFAULT_VENV) $(SHELL_RC)
309358

310-
# Show current version
359+
# Show the development version carried on the tree (e.g. 0.2.2.dev0), including
360+
# any COREAI_OPT_VERSION_EXTENSION (e.g. 0.2.2.1.dev0). Reads _about.py as plain
361+
# text, so no venv is needed — but `uv run --no-project` is still what guarantees
362+
# a >= 3.11 interpreter (a bare `python3` is 3.9 on stock macOS, and `python` may
363+
# not exist at all) without requiring `make env` first.
311364
version:
312-
@python -c "exec(open('$(MAKEFILE_DIR)src/coreai_opt/_about.py').read()); print(__version__)"
365+
@uv run --no-config --no-project --python '>=3.11' $(SCRIPTS)/make/print_version.py
313366

314367
# =============================================================================
315368
# Documentation

‎RELEASE.md‎

Lines changed: 28 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,37 @@
11
# Package Release Guide
22

3-
The OSS release process for Core AI Optimization is being defined. This page will document the workflow for publishing to PyPI once the public release infrastructure is finalized.
3+
The OSS release process for Core AI Optimization is still being defined. This page will document the workflow for publishing to PyPI once the public release infrastructure is finalized.
44

5-
Available locally:
5+
The following commands are available locally:
66

77
```bash
8-
make build # build the package wheel
9-
make version # show current version
8+
make build # build the canonical, publishable wheel + sdist (uv build --no-sources)
9+
make build-dev # build a timestamped dev wheel (e.g. 0.2.2.dev202607231430+abc1234)
10+
make version # show the development version carried on the tree (e.g. 0.2.2.dev0)
1011
make clean # remove build artifacts
1112
```
1213

14+
## Version scheme
15+
16+
`main` always carries the version planned for the _next_ release. This ensures that ongoing development is never mistaken for an already-published version, and that a release can be stabilized, tested, and published on its own branch, independently of later changes on `main`. (The release-branch workflow itself — branch naming, tagging, and backporting fixes to `main` — will be documented separately in the release schedule doc; this section covers only the version-string mechanics.)
17+
18+
`src/coreai_opt/_about.py` stores `latest_released_version` (the last tagged release, e.g. `"0.2.1"`) and computes `__version__` from it by incrementing its last number by one and adding a `.dev0` suffix (e.g. `"0.2.2.dev0"`). A pre-commit hook (`check-about-version`) verifies that `__version__` always follows this rule and that `latest_released_version` matches the repo's latest release tag. As a result, `__version__` can never look as though a release has shipped when it hasn't. The `.dev0` suffix is only a marker on the tree; it never appears in a built wheel.
19+
20+
- `make build` builds the release that `__version__` implies, e.g. `0.2.2`. A release is cut by tagging it (`v0.2.2`); `latest_released_version` is then hard-coded to `"0.2.2"`, which bumps `__version__` to the next candidate (`0.2.3.dev0`).
21+
- `make build-dev` builds that same release but with a unique `.dev<UTC-timestamp>+<short-sha>` suffix instead. It is used by contributors, smoke tests, and the nightly pipeline. `DEV_VERSION=<version>` uses that version exactly instead.
22+
23+
Sorting is preserved: `0.2.2.dev0 < 0.2.2.dev202607231430+abc1234 < 0.2.2`.
24+
25+
### Extending the scheme downstream
26+
27+
A repo that uses this one as a submodule and includes this `Makefile` — building one combined wheel from both trees — can add its own 4th number. Set `COREAI_OPT_VERSION_EXTENSION` to the number it's about to release next (e.g. `"1"` for its first release off a given OSS release, then `"2"` for the one after that). Then call `make build`, `make build-dev`, or `make version` unchanged:
28+
29+
- `latest_released_version` `"0.2.1"` + extension `"1"` -> candidate: `0.2.1.1.dev0`
30+
- `make build` -> `0.2.1.1`
31+
- `make build-dev` -> `0.2.1.1.dev<UTC-timestamp>+<short-sha>`
32+
33+
The extra number is used exactly as given (`scripts/release/release_utils.apply_version_extension`); `latest_released_version`'s own last number is only bumped for OSS's own `main`, when no extension is set.
34+
35+
There is still only one `_about.py` (this package's own); the extra number is a plain string handled entirely in `scripts/release/release_utils.next_release_base` — no other file or package is involved.
36+
1337
<!-- TODO: Document the chosen OSS release workflow (PyPI trusted publishing, twine upload, or uv publish). -->

0 commit comments

Comments
 (0)