Skip to content

Add nvidia-cuda-{nvrtc, nvcc} as a dependency for cuDF wheels #18686

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
May 12, 2025

Conversation

brandon-b-miller
Copy link
Contributor

With the merge of NVIDIA/numba-cuda#155 we need to depend on these two wheels if we want numba-cuda to be able to find the runtime libraries it needs in the final cuDF environment.

@brandon-b-miller brandon-b-miller requested a review from a team as a code owner May 6, 2025 19:47
@brandon-b-miller brandon-b-miller requested a review from bdice May 6, 2025 19:47
@jakirkham jakirkham added improvement Improvement / enhancement to an existing function non-breaking Non-breaking change labels May 6, 2025
@jakirkham
Copy link
Member

Seeing the following error on CI:

numba.cuda.cudadrv.driver.LinkerError: [222] Call to cuLinkAddData results in CUDA_ERROR_UNSUPPORTED_PTX_VERSION
ptxas application ptx input, line 9; fatal   : Unsupported .version 8.8; current version is '8.7'

Looks like pynvjitlink is enabled in this code

numba_config.CUDA_ENABLE_PYNVJITLINK = True

Not sure what else we are missing

@brandon-b-miller
Copy link
Contributor Author

brandon-b-miller commented May 6, 2025

These errors

ptxas application ptx input, line 9; fatal   : Unsupported .version 8.8; current version is '8.7'

Indicate the PTX is coming from the CUDA 12.9 NVVM library. The version of the nvvm wheel that ends up on the user system cant be newer than the driver it seems. We're probably getting the latest now so that's why it shows up.

@brandon-b-miller
Copy link
Contributor Author

reproduced using local CI, following up.

@vyasr
Copy link
Contributor

vyasr commented May 7, 2025

After NVIDIA/numba-cuda#155 are these hard dependencies of numba-cuda? If so, why aren't they part of the dependency list there? If not and that PR is only adding optional support (i.e. the wheels are used if available but other sources are also OK) then should we figure out what we want the final state to be for cudf? I'm fine with making these hard dependencies if we want to say that cudf will always get these libraries from wheels going forward, I just want to make sure that we are aware of the choices we are making if they are not forced upon us by numba-cuda.

@brandon-b-miller brandon-b-miller requested a review from a team as a code owner May 7, 2025 22:14
@github-actions github-actions bot added the Python Affects Python cuDF API. label May 7, 2025
@github-actions github-actions bot removed the Python Affects Python cuDF API. label May 8, 2025
@brandon-b-miller
Copy link
Contributor Author

are these hard dependencies of numba-cuda?

No, they are not. The current strategy is for the traditional numba-cuda not to depend on CUDA wheels and to be able to leverage system packages where necessary. However in the future there may be a one liner cuDF can use to install numba-cuda with cuda wheel support so that cuDF doesn't have to manage the dependencies explicitly.

@github-actions github-actions bot added the Python Affects Python cuDF API. label May 8, 2025
@github-actions github-actions bot removed the Python Affects Python cuDF API. label May 12, 2025
@GPUtester GPUtester moved this to In Progress in cuDF Python May 12, 2025
@brandon-b-miller
Copy link
Contributor Author

/merge

@rapids-bot rapids-bot bot merged commit 3248bcc into rapidsai:branch-25.06 May 12, 2025
127 checks passed
@github-project-automation github-project-automation bot moved this from In Progress to Done in cuDF Python May 12, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
improvement Improvement / enhancement to an existing function non-breaking Non-breaking change
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

4 participants