1.19.0 breaks on CUDA 12.9

[khluu]

Upgrading to nvidia-cudnn-frontend 1.19.0 seems to break for CUDA 12.9 environment. In particular, I run into this error RuntimeError: Unable to load any libcudart.so.* library a lot. If I downgrade to 1.18.0 then it works.

To reproduce:

import torch
from flashinfer.gemm.gemm_base import build_cudnn_gemm_with_per_tensor_q_graph
graph = build_cudnn_gemm_with_per_tensor_q_graph(
    a_shape=(1, 128, 4096), a_stride=(128*4096, 4096, 1),
    b_shape=(1, 4096, 3072), b_stride=(4096*3072, 3072, 1),
    a_type=torch.float8_e4m3fn, b_type=torch.float8_e4m3fn,
    o_type=torch.bfloat16, device=torch.device('cuda:0'))

I looked at the commit history and saw there is this change
df73764#diff-fa77515a2e2c2b179c7d1734c20ed2f56ec6aaf419ac6c4f3184c1c1c9fb5f23R88
but I didn't see any mention of CUDA version support change in the release notes. Is this expected?

My environment:
CUDA: Cuda compilation tools, release 12.9, V12.9.86 Build cuda_12.9.r12.9/compiler.36037853_0
Nvidia driver version: 580.65.06

[Anerudhan]

Hi @khluu ,

Thanks for reporting this.

cudnn-frontend should be cuda version independent. Let me try to reproduce this and update.

Thanks

[Anerudhan]

This seems to be the offending change. I have 1.19.1 prepared for another unrelated bug. Let me roll in this change as well.

Thanks for reporting again!

[khluu]

That's the exact line we noticed on our side as well. Thanks for taking care of this!

[Anerudhan]

Posted 1.19.1 @khluu can you please verify?

Thanks