Segfaults with cudnn>=9.11 for pre-Turing devices (<=sm_70)

[h-vetinari]

When preparing the pytorch v2.8 release, @mgorny ran into a bunch of segfaults. After some painful debugging against the 2.7 branch (diff environments against last known passing run, try to recreate passing run, then relax constraints again one by one), the conclusion was that we need to pin to cudnn <9.11, but not really why that occurs.

Segfaults looked like

........................................................................ [ 23%]
Fatal Python error: Segmentation fault

Thread 0x00007f295ffff640 (most recent call first):
  <no Python frame>

Thread 0x00007f2b48cc6640 (most recent call first):

resp.

=================================== FAILURES ===================================
____________________________ test/test_autograd.py _____________________________
[gw0] linux -- Python 3.13.5 $PREFIX/bin/python3.13
worker 'gw0' crashed while running 'test/test_autograd.py::TestAutogradDeviceTypeCUDA::test_rnn_backward_to_input_but_not_parameters_cuda'
_____________________________ test/test_modules.py _____________________________
[gw1] linux -- Python 3.13.5 $PREFIX/bin/python3.13
worker 'gw1' crashed while running 'test/test_modules.py::TestModuleCUDA::test_cpu_gpu_parity_nn_BatchNorm1d_eval_mode_cuda_float32'
_______________________________ test/test_nn.py ________________________________
[gw2] linux -- Python 3.13.5 $PREFIX/bin/python3.13
worker 'gw2' crashed while running 'test/test_nn.py::TestNNDeviceTypeCUDA::test_CTCLoss_cudnn_cuda'
_____________________________ test/test_modules.py _____________________________
[gw3] linux -- Python 3.13.5 $PREFIX/bin/python3.13
worker 'gw3' crashed while running 'test/test_modules.py::TestModuleCUDA::test_cpu_gpu_parity_nn_BatchNorm1d_eval_mode_cuda_float64'
_______________________________ test/test_nn.py ________________________________
[gw4] linux -- Python 3.13.5 $PREFIX/bin/python3.13
worker 'gw4' crashed while running 'test/test_nn.py::TestNN::test_RNN_change_dropout'
================== xdist: maximum crashed workers reached: 4 ===================

but note the maximum crashed workers reached, so there's likely many more.

One way to test this for the @conda-forge/cudnn folks would be to install pytorch v2.7.1 (where we only have a cudnn >=9.10.1.4,<10.0a0 constraint), and then run the test suite. For python v2.8.0, you'd have to destructively alter the environment (e.g. copy newer cudnn into $PREFIX), because the metadata won't let the solver install newer cudnn.

Obviously we want to get rid of the upper bound ASAP, because other feedstocks pulling in cudnn v9.x will create >=9.x,<10 run-exports, and then be incompatible with pytorch v2.8.

Sidenote: @carterbox spoke of an ABI break between pytorch v2.7.0 and v2.7.1 related to v2.7.1 having been built against the pybind v3 ABI, but AFAIU that is only a question for packages building on top of pytorch, not pytorch itself; so I don't think this is the reason, but thought I'd mention it for completeness.

[mgorny]

A few more notes:

1. I wasn't able to reproduce the problem on Quansight's internal testing machine (with sm75), but it happened reliably on the CI (with sm70).
2. I'm not 100% sure if it's >=9.11 or >=9.12. I've pinned <9.11 because that was the previous from the last successful run. When we're past the release, I'll create a small PR to do another run with <9.12, and hopefully have a better reference point for testing.
3. To avoid it, I needed to pin cudnn in run dependencies. Building against older cudnn, then running tests with newer cudnn also caused the segfault.

[carterbox]

Hey @h-vetinari,@mgorny. The cuDNN developers think this is caused by the deprecation of support for archs before Turing (sm_75) which was announced with cuDNN 9.9 and then removed in cuDNN 9.11. Please let us know if this doesn't align with your observations.

[mgorny]

Oh yeah, that would explain precisely why it fails on CI but not on our testing machine.

So FWIU we either need to remove support for <7.5, or keep the pin, is that correct? Can you think of any other solution?

[h-vetinari]

The cuDNN developers think this is caused by the deprecation of support for archs before Turing (sm_75)

I don't mind this in principle, but this should IMO really raise an error at config time, rather than running into very painful debugging at runtime.

Beyond that, I think that it'll massively complicate (or restrict) the testing we're able to do for pytorch, because the opengpu server apparently only only has a sm_70 card and realistically that will probably be a multi-months effort to replace.
CC @aktech @dharhas @rgommers @jaimergp

[carterbox]

Wearing my conda-forge hat, looks like you'll have to do something like the following:

- libcudnn-dev  <9.11  # [(cuda_compiler_verison or '').startswith('12')]
- libcudnn-dev >=9.11  # [(cuda_compiler_verison or '').startswith('13')]

I don't think there's any config-time things the devs could do, maybe a compile time check might work?

[h-vetinari]

Wearing my conda-forge hat, looks like you'll have to do something like the following:

Irrespective of the CUDA version, we'd have to cap cudnn for the testing phase of pytorch, because cudnn removing an architecture (in the same 9.x major line no less) just means that nothing can run the GPU code on the V100 of the opengpu server anymore.

Would it be possible to build to build newer cudnn with more architecture support for the CUDA 12.x builds? I would neither like to have a permanent cap for cudnn (meaning we test with nothing like real life environments), nor add a cap to the pytorch metadata (then everyone gets conflicts), nor stop testing the GPU paths for pytorch - those are all bad choices.

because the opengpu server apparently only only has a sm_70 card and realistically that will probably be a multi-months effort to replace.

Hardware upgrades may even be completely off the table (still TBD), but that just shows how big of a blast radius this will have for us.

[rgommers]

Hardware upgrades may even be completely off the table (still TBD), but that just shows how big of a blast radius this will have for us.

Most likely I think is that we'd want to switch to (or add) something like https://cirrus-runners.app/. That's quite cost-effective for open source CI, and currently has RTX 4000 series GPUs (Ada, compute capability 8.9). But yes indeed - that's both extra effort to make work and extra $$ to pay for monthly/yearly.

[h-vetinari]

Would it be possible to build to build newer cudnn with more architecture support for the CUDA 12.x builds?

@conda-forge/cudnn, I'd propose to fix this in the build scripts here, but since we're doing binary repackaging, this needs to happen upstream. We should not be dropping architectures for CUDA 12.x; doing so is a bug that breaks our major-version cudnn pinning.

Ideally this should be

5.0;6.0;7.0;7.5;8.0;8.6;9.0;10.0;12.0+PTX

CC @conda-forge/cuda

[traversaro]

For reference, the removal of sm_70 from cudnn puzzled several users also in carpentries-incubator/reproducible-ml-workflows#57 (comment) . fyi @matthewfeickert

[traversaro]

Totally OT, but this is exactly the case for which it would be convenient to have some sort of virtual package-like system to declare compatibility for given compute architectures, a bit like what is being developed as part of WheelNext: https://github.com/wheelnext/nvidia-variant-provider/blob/a8fe68361bef52c8cdd1015e1aff23e2662eec46/README.md#how-it-works .