Commit 7dea3c6
authored
Fix ROCm index URLs dropping the parent-directory hop (#8446)
## Problem
Follow-up to #8436. The ROCm indices it generates point at URLs that do
not
exist. From
[whl/nightly/rocm7.14/rocm-sdk-device-gfx1010/](https://download.pytorch.org/whl/nightly/rocm7.14/rocm-sdk-device-gfx1010/):
```html
<a href="https://repo.amd.com/rocm/whl-multi-arch/rocm-sdk-device-gfx1010/rocm_sdk_device_gfx1010-7.13.0-py3-none-linux_x86_64.whl">
```
AMD serves a per-package *index* at `whl-multi-arch/<package>/`, but
stores the
*wheels* flat in `whl-multi-arch/`:
| URL | result |
|---|---|
|
`.../whl-multi-arch/rocm-sdk-device-gfx1010/rocm_sdk_device_gfx1010-7.13.0-...whl`
(published) | **403** |
| `.../whl-multi-arch/rocm_sdk_device_gfx1010-7.13.0-...whl` (actual
location) | 206 |
Every link in every ROCm package index is currently broken.
## Cause
AMD's index hrefs carry the parent hop:
```html
<a href="../rocm_sdk_device_gfx1010-7.13.0-py3-none-linux_x86_64.whl">
```
`replace_relative_links_with_absolute()` normalised them with:
```python
url = url.lstrip("./")
url = url.lstrip("/")
```
`str.lstrip("./")` strips leading `.` and `/` **characters**, not a
prefix, so
`../x.whl` collapses to `x.whl`, which is then concatenated onto the
package
URL — silently producing the 403 path.
## Fix
Resolve the href with `urllib.parse.urljoin`, which honours `../`.
**Scoped to ROCm** via the existing `is_amd_package()`: NVIDIA
(`pypi.nvidia.com/<pkg>/`, plain-filename hrefs) and PyPI (absolute
`files.pythonhosted.org` hrefs) do not use `../` and keep the existing
code
path, so their generated indices are unchanged.
## Test plan
Ran the real AMD index HTML for 6 ROCm packages through the patched
function and
HTTP range-requested every resulting URL:
```
rocm 2/2 reachable
rocm-sdk-core 4/4 reachable
rocm-sdk-libraries 4/4 reachable
rocm-sdk-device-gfx1010 4/4 reachable
rocm-sdk-device-gfx942 2/2 reachable
rocm-sdk-device-gfx90a 3/3 reachable
-------------------------------------------
TOTAL after fix 19/19 reachable
currently published 0/4 reachable
```
Non-ROCm paths asserted byte-identical to today's output:
| index | href in | href out |
|---|---|---|
| NVIDIA | `x.whl#sha256=a` |
`https://pypi.nvidia.com/nvidia-cublas-cu12/x.whl#sha256=a` |
| PyPI | `https://files.pythonhosted.org/...` | unchanged |
`black --check` clean, `flake8` clean, `py_compile` passes.
## After merge
The published indices are regenerated by the `update-s3-dependencies`
workflow,
so the broken links will be corrected on its next run — no manual S3/R2
cleanup
needed.
Co-authored-by: Andrey Talman <atalman@users.noreply.github.com>1 parent 39a32b2 commit 7dea3c6
1 file changed
Lines changed: 12 additions & 2 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
2 | 2 | | |
3 | 3 | | |
4 | 4 | | |
| 5 | + | |
5 | 6 | | |
6 | 7 | | |
7 | 8 | | |
| |||
836 | 837 | | |
837 | 838 | | |
838 | 839 | | |
839 | | - | |
| 840 | + | |
| 841 | + | |
| 842 | + | |
840 | 843 | | |
841 | 844 | | |
842 | 845 | | |
843 | 846 | | |
844 | 847 | | |
845 | 848 | | |
| 849 | + | |
846 | 850 | | |
847 | 851 | | |
848 | 852 | | |
| |||
864 | 868 | | |
865 | 869 | | |
866 | 870 | | |
| 871 | + | |
| 872 | + | |
| 873 | + | |
| 874 | + | |
867 | 875 | | |
868 | 876 | | |
869 | 877 | | |
| |||
902 | 910 | | |
903 | 911 | | |
904 | 912 | | |
905 | | - | |
| 913 | + | |
| 914 | + | |
| 915 | + | |
906 | 916 | | |
907 | 917 | | |
908 | 918 | | |
| |||
0 commit comments