Skip to content

Fix NCCL preloading#485

Merged
kmaehashi merged 4 commits intocupy:mainfrom
kmaehashi:fix-nccl-preload
Feb 16, 2026
Merged

Fix NCCL preloading#485
kmaehashi merged 4 commits intocupy:mainfrom
kmaehashi:fix-nccl-preload

Conversation

@kmaehashi
Copy link
Member

This PR changes the verifier to install NCCL with pip instead of install_library.py.

Now that CuPy uses cuda-pathfinder, NCCL installed via install_library.py will not be used if cuda-pathfinder can discover NCCL from other locations such as $CUDA_PATH. However, several nvidia/cuda docker images we use for verification (e.g. nvidia/cuda:12.1.1-runtime-ubuntu22.04) contains outdated NCCL (e.g. libnccl2=2.17.1-1+cuda12.1) which is no longer supported by CuPy. This will cause import cupy_backends.cuda.libs.nccl to fail with undefined symbol: ncclCommSplit error, causing the wheel verification process to fail.

@kmaehashi kmaehashi merged commit bacad54 into cupy:main Feb 16, 2026
1 check passed
@kmaehashi kmaehashi deleted the fix-nccl-preload branch February 16, 2026 16:18
@kmaehashi
Copy link
Member Author

Again self-merging due to urgency 🙇 cc/ @asi1024

@kmaehashi kmaehashi self-assigned this Feb 16, 2026
chainer-ci pushed a commit to chainer-ci/cupy-release-tools that referenced this pull request Feb 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant