Keep the PyPI-sourced CUDA deps in PACKAGE_LINKS_ALLOW_LIST only - #8514
Keep the PyPI-sourced CUDA deps in PACKAGE_LINKS_ALLOW_LIST only#8514atalman wants to merge 2 commits into
Conversation
cuda_python, cuda_bindings, cuda_pathfinder, cuda_toolkit and nvidia_ml_py appeared in both allow lists. They are sourced from PyPI by update_dependencies.py, so their index is meant to be copied from the parent rather than regenerated: that is what pytorch#8483 added them to PACKAGE_LINKS_ALLOW_LIST for. It left them in PACKAGE_ALLOW_LIST, which keeps manage_v2 generating an index for them from whatever wheels it finds, competing with the copied one. They were the only five entries present in both lists; the links list is untouched, so all five still get copied into the arch subdirectories. whl/test/cu134 currently advertises cuda-pathfinder and cuda-python in its listing while both targets 403.
|
@atalman is attempting to deploy a commit to the Meta Open Source Team on Vercel. A member of the Team first needs to authorize it. |
|
Correcting the framing on this one: it does not fix the cuda-pathfinder / cuda-python 403s on cu134. I had it down as a likely cause with a caveat; having traced it properly, it is not. Those indexes are written by That is fixed in #8515. This PR still stands on its own: the two allow lists are meant to be disjoint, these five were the only overlap, and #8483 plainly intended them to be links-only. Just judge it as list hygiene rather than as a fix for the 403s. |
Five packages were listed in both allow lists:
They are sourced from PyPI by
update_dependencies.py, so their index is meant to be copied from the parent rather than regenerated from a wheel listing — which is exactly what #8483 added them toPACKAGE_LINKS_ALLOW_LISTfor. That PR left them inPACKAGE_ALLOW_LIST, somanage_v2still generates an index for them from whatever wheels it finds, competing with the copied one.The two lists are meant to be disjoint. Every other entry in
PACKAGE_LINKS_ALLOW_LIST—typing-extensions, thenvidia-*set,intel-*,onemkl-*— is absent fromPACKAGE_ALLOW_LIST. These five were the only overlap.This drops them from
PACKAGE_ALLOW_LIST.PACKAGE_LINKS_ALLOW_LISTis untouched, so all five still get copied into every arch subdirectory.Verified with an AST parse of both lists before and after: the links list is byte-identical, and all five remain in it.
Symptom this is aimed at
whl/test/cu134advertises both of these in its PEP 503 listing while the targets 403:Honest caveat on causation
I have not proven the double-listing is what breaks those two.
cuda-bindingsandcuda-toolkitare double-listed in exactly the same way and resolve fine, so overlap alone is clearly not sufficient. The change is right on its own terms — the lists are meant to be disjoint and #8483 plainly intended these to be links-only — but treat "this fixes the 403s" as a hypothesis, not a claim.Two index runs were still in flight while I was looking (a 19:12 dispatch and a 19:21 schedule), and
typing-extensionsself-healed on the first run after #8513 landed without any list change. So it is worth re-probing those four URLs after the next run regardless of whether this merges.ruff formatandruff checkclean.