Fix ROCm index URLs dropping the parent-directory hop - #8446
Merged
Conversation
Follow-up to pytorch#8436. AMD serves a per-package index at whl-multi-arch/<package>/ but stores the wheels flat in whl-multi-arch/, so every href is "../<wheel>.whl". replace_relative_links_with_absolute() normalised those with str.lstrip("./"), which strips leading "." and "/" *characters* rather than a prefix, so "../x.whl" collapsed to "x.whl" and was joined onto the package URL. The published indices therefore point at whl-multi-arch/<package>/<wheel>.whl, which returns HTTP 403: https://download.pytorch.org/whl/nightly/rocm7.14/rocm-sdk-device-gfx1010/ Resolve the href with urllib.parse.urljoin instead, which honours the parent hop. Scoped to ROCm via is_amd_package(): NVIDIA and PyPI indices do not use "../" hrefs and keep the existing code path.
|
@atalman is attempting to deploy a commit to the Meta Open Source Team on Vercel. A member of the Team first needs to authorize it. |
meta-cla
Bot
added
the
CLA Signed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Follow-up to #8436. The ROCm indices it generates point at URLs that do not
exist. From
whl/nightly/rocm7.14/rocm-sdk-device-gfx1010/:
AMD serves a per-package index at
whl-multi-arch/<package>/, but stores thewheels flat in
whl-multi-arch/:.../whl-multi-arch/rocm-sdk-device-gfx1010/rocm_sdk_device_gfx1010-7.13.0-...whl(published).../whl-multi-arch/rocm_sdk_device_gfx1010-7.13.0-...whl(actual location)Every link in every ROCm package index is currently broken.
Cause
AMD's index hrefs carry the parent hop:
replace_relative_links_with_absolute()normalised them with:str.lstrip("./")strips leading.and/characters, not a prefix, so../x.whlcollapses tox.whl, which is then concatenated onto the packageURL — silently producing the 403 path.
Fix
Resolve the href with
urllib.parse.urljoin, which honours../.Scoped to ROCm via the existing
is_amd_package(): NVIDIA(
pypi.nvidia.com/<pkg>/, plain-filename hrefs) and PyPI (absolutefiles.pythonhosted.orghrefs) do not use../and keep the existing codepath, so their generated indices are unchanged.
Test plan
Ran the real AMD index HTML for 6 ROCm packages through the patched function and
HTTP range-requested every resulting URL:
Non-ROCm paths asserted byte-identical to today's output:
x.whl#sha256=ahttps://pypi.nvidia.com/nvidia-cublas-cu12/x.whl#sha256=ahttps://files.pythonhosted.org/...black --checkclean,flake8clean,py_compilepasses.After merge
The published indices are regenerated by the
update-s3-dependenciesworkflow,so the broken links will be corrected on its next run — no manual S3/R2 cleanup
needed.