Skip to content

Commit 6e42740

Browse files
jeanschmidtmalfet
andauthored
Add PyPI wheel cache build workflow, version bumps, and tooling improvements (#7912)
**Impact:** PyTorch CI (ARC OSDC wheel cache), release versions, ClickHouse ingestion, blast remote execution CLI **Risk:** medium ## What Adds a daily GitHub Actions workflow that pre-builds PyPI wheels for all supported Python versions (3.10–3.14, including free-threaded) across CUDA variants and CPU, uploading them to an S3-backed cache (`pytorch-pypi-wheel-cache`). ## Why PyTorch ARC OSDC CI jobs depend on third-party PyPI packages that require compilation (native extensions). Building these from source on every job is expensive and slow. A shared S3 wheel cache that is rebuilt daily eliminates redundant compilation, reduces CI wall-clock time, and avoids transient build failures from upstream PyPI changes mid-run. ## System design / Workflow 1. Capture — A wants-collector pod (in pytorch/ci-infra) scans EFS access logs on each OSDC cluster, identifies packages missing pre-built wheels, and uploads package==version lists to s3://pytorch-pypi-wheel-cache/wants/. 2. Build — A daily GitHub Actions workflow downloads all wants files, deduplicates them, and builds every package across the full matrix: 7 Python versions (3.10–3.14, including free-threaded) × 3 CUDA variants + CPU × 2 architectures (x86_64, aarch64) = 8 parallel jobs. Runs inside manylinux_2_28 containers that ship with all Python versions pre-installed. Wheels are uploaded to s3://pytorch-pypi-wheel-cache/{variant}/ (e.g. cu128/, cpu/). Known-incompatible combos are skipped via a declarative skip list. Failed builds warn but don't fail the workflow — the cache is best-effort. 3. Serve — The OSDC pypi-server module syncs wheels from S3 to a local PVC, where pypiserver serves them to CI jobs. > Note: PyPI upstream is always preferred over locally-built wheels. The local cache only fills gaps where PyPI doesn't have a compatible wheel (e.g. specific CUDA/arch/free-threaded combinations). ## How - The workflow generates a build matrix (CUDA 12.8, 13.0 × x86_64, aarch64 + CPU variants) via a Python script, then runs builds inside `manylinux`-compatible containers on Meta-hosted runners. - `build.sh` downloads a merged wants list from S3 (`s3://bucket/wants/*.txt`), checks which wheels already exist in the cache, builds only what's missing, repairs `linux_*` platform tags to `manylinux_2_28_*`, and uploads the results. - A skip list (`skip_python_versions.txt`) prevents known-incompatible package/Python combos from being attempted, even under `force_rebuild`. - The `repair_manylinux_2_28.sh` script was hardened: uses `mktemp -d` with trap cleanup, discovers `.dist-info` via `find` instead of filename parsing, and properly quotes all paths. ## Changes ### PyPI wheel cache (primary) - New workflow `.github/workflows/build-pypi-cache.yml` — daily cron + manual dispatch with force-rebuild support - New `generate_matrix.py` — produces the CUDA × arch × CPU build matrix JSON - New `build.sh` — downloads wants lists, deduplicates, skips cached wheels, builds, repairs platform tags, uploads to S3 - New `skip_python_versions.txt` — declares known-unsupported package/Python combos (numpy on 3.10, numba on 3.13+, scipy free-threaded, etc.) - New `build_requirements.txt` and `system_packages.txt` — build-time Python and OS dependencies - Improved `repair_manylinux_2_28.sh` — safer temp dir handling, robust `.dist-info` discovery, proper quoting ### CI infrastructure - Added `m8g.metal-24xl` ARM64 runner type across all four scale-config files (canary, lf-canary, lf, prod) ### Release versions - `FBGEMM_GPU_VERSION`: 1.5.0 → 1.6.0 - `TORCHAO_VERSION`: 0.16.0 → 0.17.0 - `TORCH_TENSORRT_VERSION`: 2.10.0 → 2.11.0 ## Notes - The workflow contains two `TODO` comments for removing PR-specific triggers and concurrency keys before final merge to main. - The `AWS_ROLE` ARN in the workflow has a double-colon (`iam::308535385114`) which should be verified as intentional or a typo. - The wants list mechanism (`s3://bucket/wants/*.txt`) implies a separate process populates these files — this PR does not include that producer. ## Testing - The workflow currently has a `pull_request` trigger scoped to its own paths, so opening this PR will exercise the full matrix build on CI. https://github.com/pytorch/test-infra/actions/runs/23877221276 --------- Signed-off-by: Jean Schmidt <contato@jschmidt.me> Co-authored-by: Nikita Shulga <2453524+malfet@users.noreply.github.com>
1 parent 9ca5018 commit 6e42740

7 files changed

Lines changed: 858 additions & 0 deletions

File tree

0 commit comments

Comments
 (0)