Skip to content

Commit 7dea3c6

Browse files
authored
Fix ROCm index URLs dropping the parent-directory hop (#8446)
## Problem Follow-up to #8436. The ROCm indices it generates point at URLs that do not exist. From [whl/nightly/rocm7.14/rocm-sdk-device-gfx1010/](https://download.pytorch.org/whl/nightly/rocm7.14/rocm-sdk-device-gfx1010/): ```html <a href="https://repo.amd.com/rocm/whl-multi-arch/rocm-sdk-device-gfx1010/rocm_sdk_device_gfx1010-7.13.0-py3-none-linux_x86_64.whl"> ``` AMD serves a per-package *index* at `whl-multi-arch/<package>/`, but stores the *wheels* flat in `whl-multi-arch/`: | URL | result | |---|---| | `.../whl-multi-arch/rocm-sdk-device-gfx1010/rocm_sdk_device_gfx1010-7.13.0-...whl` (published) | **403** | | `.../whl-multi-arch/rocm_sdk_device_gfx1010-7.13.0-...whl` (actual location) | 206 | Every link in every ROCm package index is currently broken. ## Cause AMD's index hrefs carry the parent hop: ```html <a href="../rocm_sdk_device_gfx1010-7.13.0-py3-none-linux_x86_64.whl"> ``` `replace_relative_links_with_absolute()` normalised them with: ```python url = url.lstrip("./") url = url.lstrip("/") ``` `str.lstrip("./")` strips leading `.` and `/` **characters**, not a prefix, so `../x.whl` collapses to `x.whl`, which is then concatenated onto the package URL — silently producing the 403 path. ## Fix Resolve the href with `urllib.parse.urljoin`, which honours `../`. **Scoped to ROCm** via the existing `is_amd_package()`: NVIDIA (`pypi.nvidia.com/<pkg>/`, plain-filename hrefs) and PyPI (absolute `files.pythonhosted.org` hrefs) do not use `../` and keep the existing code path, so their generated indices are unchanged. ## Test plan Ran the real AMD index HTML for 6 ROCm packages through the patched function and HTTP range-requested every resulting URL: ``` rocm 2/2 reachable rocm-sdk-core 4/4 reachable rocm-sdk-libraries 4/4 reachable rocm-sdk-device-gfx1010 4/4 reachable rocm-sdk-device-gfx942 2/2 reachable rocm-sdk-device-gfx90a 3/3 reachable ------------------------------------------- TOTAL after fix 19/19 reachable currently published 0/4 reachable ``` Non-ROCm paths asserted byte-identical to today's output: | index | href in | href out | |---|---|---| | NVIDIA | `x.whl#sha256=a` | `https://pypi.nvidia.com/nvidia-cublas-cu12/x.whl#sha256=a` | | PyPI | `https://files.pythonhosted.org/...` | unchanged | `black --check` clean, `flake8` clean, `py_compile` passes. ## After merge The published indices are regenerated by the `update-s3-dependencies` workflow, so the broken links will be corrected on its next run — no manual S3/R2 cleanup needed. Co-authored-by: Andrey Talman <atalman@users.noreply.github.com>
1 parent 39a32b2 commit 7dea3c6

1 file changed

Lines changed: 12 additions & 2 deletions

File tree

s3_management/update_dependencies.py

Lines changed: 12 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,7 @@
22
import re
33
import time
44
from typing import Dict, List
5+
from urllib.parse import urljoin
56

67
import boto3 # type: ignore[import-untyped]
78

@@ -836,13 +837,16 @@ def download(url: str) -> bytes:
836837
return conn.read()
837838

838839

839-
def replace_relative_links_with_absolute(html: str, base_url: str) -> str:
840+
def replace_relative_links_with_absolute(
841+
html: str, base_url: str, resolve_relative_paths: bool = False
842+
) -> str:
840843
"""
841844
Replace all relative links in HTML with absolute links.
842845
843846
Args:
844847
html: HTML content as string
845848
base_url: Base URL to prepend to relative links
849+
resolve_relative_paths: Resolve "../" segments against base_url (ROCm only)
846850
847851
Returns:
848852
Modified HTML with absolute links
@@ -864,6 +868,10 @@ def replace_href(match):
864868
):
865869
return full_match
866870

871+
# ROCm wheels sit flat in whl-multi-arch/, so AMD hrefs are "../<wheel>.whl"
872+
if resolve_relative_paths:
873+
return f'href="{urljoin(base_url, url)}"'
874+
867875
# Remove leading ./ or /
868876
url = url.lstrip("./")
869877
url = url.lstrip("/")
@@ -902,7 +910,9 @@ def upload_index_html(
902910
) -> None:
903911
"""Upload modified index.html to S3 and R2 with absolute links"""
904912
# Replace relative links with absolute links
905-
modified_html = replace_relative_links_with_absolute(html, base_url)
913+
modified_html = replace_relative_links_with_absolute(
914+
html, base_url, resolve_relative_paths=is_amd_package(pkg_name)
915+
)
906916

907917
index_key = f"{prefix}/{pkg_name}/index.html"
908918

0 commit comments

Comments
 (0)