Skip to content

Commit 6958756

Browse files
authored
s3_management: keep PyPI-sourced NVIDIA indexes from being overwritten (#8483)
Follow-up to #8473, which switched `cuda-bindings`, `cuda-pathfinder`, `cuda-python` and `nvidia-ml-py` to be sourced from PyPI (`pypi.org/simple/`) instead of `pypi.nvidia.com`. `update_dependencies.py` writes those indexes **per accelerator target** — `main()` loops `PACKAGES_PER_PROJECT` and calls `upload_package_using_simple_index(pkg_name, f"{prefix}/{target}")`, producing `whl/nightly/cu130/cuda-bindings/index.html` etc. with absolute `files.pythonhosted.org` links. `cuda-bindings` is declared for cu126, cu128, cu129, cu130, cu132 and cu134. `manage_v2.py` separately regenerates per-package indexes from the wheels it finds in S3. `PACKAGE_LINKS_ALLOW_LIST` is what stops it from overwriting an externally-sourced index with an S3-relative one (see `index_has_external_links()`, #8115). All four packages need to be in that list now, or the PyPI-sourced indexes get clobbered on the next `manage_v2` run. Adds `cuda-bindings`, `cuda-pathfinder`, `cuda-python` and `nvidia-ml-py`. ### Why both spellings Membership is tested against the name from `obj_to_package_name()` — the wheel filename up to the first `-` — which uses **underscores** (`cuda_bindings`). A hyphen-only entry can never match, which is why the list already carries both `spmd-types` and `spmd_types`. `cuda-bindings` was already present hyphen-only and therefore inert. Verified against the live nightly indexes (link counts, root vs `cu128`): | entry | spelling | root | cu128 | | | --- | --- | --- | --- | --- | | `numpy` | single word | 4188 | 4188 | identical — active | | `spmd-types` | both listed | 11 | 11 | identical — active | | `typing-extensions` | hyphen only | 116 | 6 | regenerated — inert | | `cuda-bindings` | hyphen only | 159 | 39 | regenerated — inert | The other ~40 hyphen-only entries in the list are dormant for the same reason. Not touched here — activating them en masse is its own change. ### Upstream availability `cuda-bindings` is not on the NVIDIA CDN, so PyPI is the correct source: | index | `pypi.nvidia.com` | `pypi.org/simple` | | --- | --- | --- | | `cuda-bindings` | 404 | 200 (393 files) | | `cuda-python` | 404 | 200 (335) | | `cuda-pathfinder` | 404 | 200 (27) | | `nvidia-ml-py` | 404 | 200 (67) | | `nvidia-cublas-cu12` | 200 | 200 | Every CUDA major is on the PyPI CDN, so a single upstream index serving all targets is fine — cu126/cu128 pull the 12.9 files and cu130+ the 13.0 files from the same index. ### Test plan `cuda-python`, `cuda-pathfinder` and `nvidia-ml-py` each publish one `py3-none-any` wheel and currently resolve identically at root and in all 14 live accelerator subdirectories (`cpu`, `cu118`–`cu134`, `rocm6.3`–`rocm7.1`, `xpu`), so no index should change shape. `cuda-bindings` is the one that moves: today `manage_v2` regenerates it per subdirectory from S3 wheels (cu126/cu128/cu129 → 12.9.x, cu130/cu132/cu134 → 13.0.3); after this it keeps the PyPI-sourced index that `update_dependencies.py` writes. Confirmed the set literal evaluates with all four underscore names present. `lintrunner` PYFMT and `ruff` clean; remaining `flake8` E501 hits are pre-existing (7 on `main`, unchanged). cc @tinglvv
1 parent da70907 commit 6958756

1 file changed

Lines changed: 8 additions & 0 deletions

File tree

s3_management/manage_v2.py

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -491,7 +491,15 @@
491491
"spmd-types",
492492
"spmd_types",
493493
"cuda-bindings",
494+
"cuda_bindings",
494495
"cuda-toolkit",
496+
# Underscore spelling is the one that matches; see "spmd_types" above.
497+
"cuda-pathfinder",
498+
"cuda_pathfinder",
499+
"cuda-python",
500+
"cuda_python",
501+
"nvidia-ml-py",
502+
"nvidia_ml_py",
495503
"nvidia-cuda-nvrtc-cu12",
496504
"nvidia-cuda-nvrtc",
497505
"nvidia-cuda-runtime-cu12",

0 commit comments

Comments
 (0)