Commit 90a18ed
authored
Re-add nvidia-cudnn-cu12 9.1.0.70 to the mirrored cudnn index (#8532)
Fixes #193491
`nvidia-cudnn-cu12==9.1.0.70` disappeared from the cudnn index around
2026-08-12, which breaks every `pip install torch --index-url
https://download.pytorch.org/whl/cu124` on Linux x86_64 with
`ResolutionImpossible`. Every torch 2.4.0-2.6.0 build pins that exact
version.
**Cause:** NVIDIA stopped publishing 9.1.0.70 on
`pypi.nvidia.com/nvidia-cudnn-cu12/` (the listing now jumps `9.0.0.312`
-> `9.1.1.17`). `upload_package_using_simple_index` mirrors that index
verbatim, so the next run silently dropped the entry even though the
wheel itself is still served fine from S3/R2.
**Fix:** a `RETAINED_WHEELS` table plus `append_retained_wheels()`,
following the existing `append_preview_numpy_wheels()` idiom, re-injects
links for wheels we still host but upstream no longer lists. Skipped if
upstream ever restores the version, so there are no duplicate links.
The injected href is a **root-relative path**
(`/whl/cu124/nvidia_cudnn_cu12-9.1.0.70-...whl`), matching the form
`manage_v2.py` already uses for its libtorch and source-code indexes. It
therefore resolves against whichever host serves the index -- the wheel
is present and identical on both `download.pytorch.org` and
`download-r2.pytorch.org` -- and, being root-relative rather than
directory-relative, stays correct wherever the index is copied.
Injection consequently runs *after*
`replace_relative_links_with_absolute`, which would otherwise resolve
the path against `pypi.nvidia.com`.
**Injection is limited to the channel roots** (`whl/nightly`,
`whl/test`, `whl`). `manage_v2.py` copies a root package index down into
every accepted subdirectory (`cu[0-9]+`, `rocm*`, `cpu`, `xpu`) for
packages in its `PACKAGE_LINKS_ALLOW_LIST`, which includes
`nvidia-cudnn-cu12`, so the CUDA targets inherit the link automatically.
That keeps `cu124` and the other conserved targets conserved -- nothing
re-mirrors them from upstream, so no other version NVIDIA removes can be
dropped from them either. `PACKAGES_PER_PROJECT` and `manage_v2.py` are
both unchanged.
`cu118` is unaffected: torch 2.6.0+cu118 pins
`nvidia-cudnn-cu11==9.1.0.70`, and `nvidia_cudnn_cu11` is deliberately
absent from `PACKAGE_LINKS_ALLOW_LIST` --
`whl/cu118/nvidia-cudnn-cu11/index.html` is a hand-curated 4-link index
that still lists the version and is untouched here.
## Verification
- sha256 `165764f4...` is PyPI's. Confirmed the hosted file is
byte-identical without downloading 664 MB: the S3 ETag on `whl/cu124`
and `whl/nightly/cu124` is `97a2573180f8d6968a8068086339ff75`, exactly
PyPI's published md5, and content-length matches at `664752741` on both
`download.pytorch.org` and `download-r2.pytorch.org`.
- Ran the full `upload_package_using_simple_index` path with a stubbed
S3: the link lands in the three channel-root indexes with the href
intact as `/whl/cu124/...`, is absent from `whl/nightly/cu126`, and no
href anywhere was mangled into a `pypi.nvidia.com/.../whl/` URL.
- Fed the result through pip's own `parse_links` collector, both as the
root index and as the same bytes copied into a `cu124/` subdirectory
(what `manage_v2.py` does): 151 links in each, both resolving to the
same absolute wheel URL against the serving host, sha256 recognised as
the link hash.
- The patched root index still satisfies
`manage_v2.index_has_external_links()` (the 150 mirrored
`pypi.nvidia.com` links), so the copy-to-subdirectory step is not
skipped.
- Confirmed via AST extraction that `nvidia_cudnn_cu12` is in
`PACKAGE_LINKS_ALLOW_LIST` and that `cu124` matches
`ACCEPTED_SUBDIR_PATTERNS`. This also matches observed state: every
`cu*` subdir index currently has an ETag byte-identical to its channel
root.
Not exercised: the S3/R2 `put` calls themselves (no boto3 or usable pip
on the machine this was written on), so a `dryrun: enabled` run is worth
doing first.
## Rollout
`mode=update-packages, package=torch` covers nightly + test; prod needs
a second run with `include-prod=enabled` (the `promote-env`-gated
`update-prod` job). The cu\* subdirectories pick the link up on the next
`manage_v2.py` run.1 parent a339e80 commit 90a18ed
1 file changed
Lines changed: 72 additions & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | 2 | | |
3 | 3 | | |
4 | | - | |
| 4 | + | |
5 | 5 | | |
6 | 6 | | |
7 | 7 | | |
| |||
919 | 919 | | |
920 | 920 | | |
921 | 921 | | |
| 922 | + | |
| 923 | + | |
| 924 | + | |
| 925 | + | |
| 926 | + | |
| 927 | + | |
| 928 | + | |
| 929 | + | |
| 930 | + | |
| 931 | + | |
| 932 | + | |
| 933 | + | |
| 934 | + | |
| 935 | + | |
| 936 | + | |
| 937 | + | |
| 938 | + | |
| 939 | + | |
| 940 | + | |
| 941 | + | |
| 942 | + | |
| 943 | + | |
| 944 | + | |
| 945 | + | |
| 946 | + | |
| 947 | + | |
| 948 | + | |
| 949 | + | |
| 950 | + | |
| 951 | + | |
| 952 | + | |
| 953 | + | |
| 954 | + | |
| 955 | + | |
922 | 956 | | |
923 | 957 | | |
924 | 958 | | |
| |||
1028 | 1062 | | |
1029 | 1063 | | |
1030 | 1064 | | |
| 1065 | + | |
| 1066 | + | |
| 1067 | + | |
| 1068 | + | |
1031 | 1069 | | |
1032 | 1070 | | |
1033 | 1071 | | |
| |||
1136 | 1174 | | |
1137 | 1175 | | |
1138 | 1176 | | |
| 1177 | + | |
| 1178 | + | |
| 1179 | + | |
| 1180 | + | |
| 1181 | + | |
| 1182 | + | |
| 1183 | + | |
| 1184 | + | |
| 1185 | + | |
| 1186 | + | |
| 1187 | + | |
| 1188 | + | |
| 1189 | + | |
| 1190 | + | |
| 1191 | + | |
| 1192 | + | |
| 1193 | + | |
| 1194 | + | |
| 1195 | + | |
| 1196 | + | |
| 1197 | + | |
| 1198 | + | |
| 1199 | + | |
| 1200 | + | |
| 1201 | + | |
| 1202 | + | |
| 1203 | + | |
| 1204 | + | |
| 1205 | + | |
| 1206 | + | |
| 1207 | + | |
| 1208 | + | |
| 1209 | + | |
1139 | 1210 | | |
1140 | 1211 | | |
1141 | 1212 | | |
| |||
0 commit comments