Skip to content

Keep the PyPI-sourced CUDA deps in PACKAGE_LINKS_ALLOW_LIST only - #8514

Open
atalman wants to merge 2 commits into
pytorch:mainfrom
atalman:atalman/links-only-pypi-deps
Open

Keep the PyPI-sourced CUDA deps in PACKAGE_LINKS_ALLOW_LIST only#8514
atalman wants to merge 2 commits into
pytorch:mainfrom
atalman:atalman/links-only-pypi-deps

Conversation

@atalman

@atalman atalman commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

Five packages were listed in both allow lists:

cuda_bindings  cuda_pathfinder  cuda_python  cuda_toolkit  nvidia_ml_py

They are sourced from PyPI by update_dependencies.py, so their index is meant to be copied from the parent rather than regenerated from a wheel listing — which is exactly what #8483 added them to PACKAGE_LINKS_ALLOW_LIST for. That PR left them in PACKAGE_ALLOW_LIST, so manage_v2 still generates an index for them from whatever wheels it finds, competing with the copied one.

The two lists are meant to be disjoint. Every other entry in PACKAGE_LINKS_ALLOW_LISTtyping-extensions, the nvidia-* set, intel-*, onemkl-* — is absent from PACKAGE_ALLOW_LIST. These five were the only overlap.

This drops them from PACKAGE_ALLOW_LIST. PACKAGE_LINKS_ALLOW_LIST is untouched, so all five still get copied into every arch subdirectory.

BEFORE  PACKAGE_ALLOW=235  LINKS=97  in both=[cuda_bindings, cuda_pathfinder, cuda_python, cuda_toolkit, nvidia_ml_py]
AFTER   PACKAGE_ALLOW=230  LINKS=97  in both=[]

Verified with an AST parse of both lists before and after: the links list is byte-identical, and all five remain in it.

Symptom this is aimed at

whl/test/cu134 advertises both of these in its PEP 503 listing while the targets 403:

package cu126 cu130 cu132 cu134 parent
cuda-pathfinder 200 200 200 403 200
cuda-python 200 200 200 403 200
cuda-bindings 200 200 200 200 200
cuda-toolkit 200 200 200 200 403
<a href="cuda-pathfinder/">cuda-pathfinder</a>   -> 403
<a href="cuda-python/">cuda-python</a>           -> 403

Honest caveat on causation

I have not proven the double-listing is what breaks those two. cuda-bindings and cuda-toolkit are double-listed in exactly the same way and resolve fine, so overlap alone is clearly not sufficient. The change is right on its own terms — the lists are meant to be disjoint and #8483 plainly intended these to be links-only — but treat "this fixes the 403s" as a hypothesis, not a claim.

Two index runs were still in flight while I was looking (a 19:12 dispatch and a 19:21 schedule), and typing-extensions self-healed on the first run after #8513 landed without any list change. So it is worth re-probing those four URLs after the next run regardless of whether this merges.

ruff format and ruff check clean.

cuda_python, cuda_bindings, cuda_pathfinder, cuda_toolkit and nvidia_ml_py
appeared in both allow lists. They are sourced from PyPI by
update_dependencies.py, so their index is meant to be copied from the parent
rather than regenerated: that is what pytorch#8483 added them to
PACKAGE_LINKS_ALLOW_LIST for. It left them in PACKAGE_ALLOW_LIST, which keeps
manage_v2 generating an index for them from whatever wheels it finds,
competing with the copied one.

They were the only five entries present in both lists; the links list is
untouched, so all five still get copied into the arch subdirectories.

whl/test/cu134 currently advertises cuda-pathfinder and cuda-python in its
listing while both targets 403.
@vercel

vercel Bot commented Aug 12, 2026

Copy link
Copy Markdown

@atalman is attempting to deploy a commit to the Meta Open Source Team on Vercel.

A member of the Team first needs to authorize it.

@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Aug 12, 2026
@atalman

atalman commented Aug 12, 2026

Copy link
Copy Markdown
Contributor Author

Correcting the framing on this one: it does not fix the cuda-pathfinder / cuda-python 403s on cu134. I had it down as a likely cause with a caveat; having traced it properly, it is not.

Those indexes are written by update_dependencies.py, not by the PACKAGE_LINKS_ALLOW_LIST copy path in manage_v2. cuda-pathfinder was registered under the vllm project only, so no whl/*/cu*/cuda-pathfinder/ was ever produced for a torch arch. A live run confirms it — the identical 18 packages went to cu126/cu130/cu132/cu134 and neither of these was among them.

That is fixed in #8515.

This PR still stands on its own: the two allow lists are meant to be disjoint, these five were the only overlap, and #8483 plainly intended them to be links-only. Just judge it as list hygiene rather than as a fix for the 403s.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant