h100 support

[Lucas-TY]

Hi, does this repo support H100 GPUs? I noticed that the kernel only supports up to the SM80 architecture.

[ZetangForward]

same question...

I meet the compile error:

arse-Attention-main/csrc/block_sparse_attn/src/flash_fwd_split_hdim96_fp16_sm80.cu -o /nas/tangzecheng.tzc/x-attention-main/Block-Sparse-Attention-main/build/temp.linux-x86_64-cpython-310/csrc/block_sparse_attn/src/flash_fwd_split_hdim96_fp16_sm80.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -O3 -std=c++17 -U__CUDA_NO_HALF_OPERATORS__ -U__CUDA_NO_HALF_CONVERSIONS__ -U__CUDA_NO_HALF2_OPERATORS__ -U__CUDA_NO_BFLOAT16_CONVERSIONS__ --expt-relaxed-constexpr --expt-extended-lambda --use_fast_math -lineinfo -gencode arch=compute_80,code=sm_80 -gencode arch=compute_90,code=sm_90 --threads 4 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=block_sparse_attn_cuda -D_GLIBCXX_USE_CXX11_ABI=0 -ccbin /home/tangzecheng.tzc/.conda/envs/xattn/bin/x86_64-conda-linux-gnu-cc
FAILED: /nas/tangzecheng.tzc/x-attention-main/Block-Sparse-Attention-main/build/temp.linux-x86_64-cpython-310/csrc/block_sparse_attn/src/flash_fwd_split_hdim96_fp16_sm80.o 
/home/tangzecheng.tzc/.conda/envs/xattn/bin/nvcc --generate-dependencies-with-compile --dependency-output /nas/tangzecheng.tzc/x-attention-main/Block-Sparse-Attention-main/build/temp.linux-x86_64-cpython-310/csrc/block_sparse_attn/src/flash_fwd_split_hdim96_fp16_sm80.o.d -I/nas/tangzecheng.tzc/x-attention-main/Block-Sparse-Attention-main/csrc/block_sparse_attn -I/nas/tangzecheng.tzc/x-attention-main/Block-Sparse-Attention-main/csrc/block_sparse_attn/src -I/nas/tangzecheng.tzc/x-attention-main/Block-Sparse-Attention-main/csrc/cutlass/include -I/home/tangzecheng.tzc/.conda/envs/xattn/lib/python3.10/site-packages/torch/include -I/home/tangzecheng.tzc/.conda/envs/xattn/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -I/home/tangzecheng.tzc/.conda/envs/xattn/lib/python3.10/site-packages/torch/include/TH -I/home/tangzecheng.tzc/.conda/envs/xattn/lib/python3.10/site-packages/torch/include/THC -I/home/tangzecheng.tzc/.conda/envs/xattn/include -I/home/tangzecheng.tzc/.conda/envs/xattn/include/python3.10 -c -c /nas/tangzecheng.tzc/x-attention-main/Block-Sparse-Attention-main/csrc/block_sparse_attn/src/flash_fwd_split_hdim96_fp16_sm80.cu -o /nas/tangzecheng.tzc/x-attention-main/Block-Sparse-Attention-main/build/temp.linux-x86_64-cpython-310/csrc/block_sparse_attn/src/flash_fwd_split_hdim96_fp16_sm80.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -O3 -std=c++17 -U__CUDA_NO_HALF_OPERATORS__ -U__CUDA_NO_HALF_CONVERSIONS__ -U__CUDA_NO_HALF2_OPERATORS__ -U__CUDA_NO_BFLOAT16_CONVERSIONS__ --expt-relaxed-constexpr --expt-extended-lambda --use_fast_math -lineinfo -gencode arch=compute_80,code=sm_80 -gencode arch=compute_90,code=sm_90 --threads 4 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=block_sparse_attn_cuda -D_GLIBCXX_USE_CXX11_ABI=0 -ccbin /home/tangzecheng.tzc/.conda/envs/xattn/bin/x86_64-conda-linux-gnu-cc
nvcc warning : incompatible redefinition for option 'compiler-bindir', the last value of this option was used
In file included from /nas/tangzecheng.tzc/x-attention-main/Block-Sparse-Attention-main/csrc/block_sparse_attn/src/flash_fwd_launch_template.h:14,
                 from /nas/tangzecheng.tzc/x-attention-main/Block-Sparse-Attention-main/csrc/block_sparse_attn/src/flash_fwd_split_hdim96_fp16_sm80.cu:5:
/nas/tangzecheng.tzc/x-attention-main/Block-Sparse-Attention-main/csrc/block_sparse_attn/src/flash_fwd_kernel.h:10:10: fatal error: cute/algorithm/copy.hpp: No such file or directory
   10 | #include <cute/algorithm/copy.hpp>
      |          ^~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
In file included from /nas/tangzecheng.tzc/x-attention-main/Block-Sparse-Attention-main/csrc/block_sparse_attn/src/flash_fwd_launch_template.h:14,
                 from /nas/tangzecheng.tzc/x-attention-main/Block-Sparse-Attention-main/csrc/block_sparse_attn/src/flash_fwd_split_hdim96_fp16_sm80.cu:5:
/nas/tangzecheng.tzc/x-attention-main/Block-Sparse-Attention-main/csrc/block_sparse_attn/src/flash_fwd_kernel.h:10:10: fatal error: cute/algorithm/copy.hpp: No such file or directory
   10 | #include <cute/algorithm/copy.hpp>
      |          ^~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
In file included from /nas/tangzecheng.tzc/x-attention-main/Block-Sparse-Attention-main/csrc/block_sparse_attn/src/flash_fwd_launch_template.h:14,
                 from /nas/tangzecheng.tzc/x-attention-main/Block-Sparse-Attention-main/csrc/block_sparse_attn/src/flash_fwd_split_hdim96_fp16_sm80.cu:5:
/nas/tangzecheng.tzc/x-attention-main/Block-Sparse-Attention-main/csrc/block_sparse_attn/src/flash_fwd_kernel.h:10:10: fatal error: cute/algorithm/copy.hpp: No such file or directory
   10 | #include <cute/algorithm/copy.hpp>
      |          ^~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
fatal   : Could not open input file /tmp/tmpxft_000f46d4_00000000-8_flash_fwd_split_hdim96_fp16_sm80.compute_80.cpp1.ii

@JerryGJX

[zeroCAY]

same question...

I meet the compile error:

arse-Attention-main/csrc/block_sparse_attn/src/flash_fwd_split_hdim96_fp16_sm80.cu -o /nas/tangzecheng.tzc/x-attention-main/Block-Sparse-Attention-main/build/temp.linux-x86_64-cpython-310/csrc/block_sparse_attn/src/flash_fwd_split_hdim96_fp16_sm80.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -O3 -std=c++17 -U__CUDA_NO_HALF_OPERATORS__ -U__CUDA_NO_HALF_CONVERSIONS__ -U__CUDA_NO_HALF2_OPERATORS__ -U__CUDA_NO_BFLOAT16_CONVERSIONS__ --expt-relaxed-constexpr --expt-extended-lambda --use_fast_math -lineinfo -gencode arch=compute_80,code=sm_80 -gencode arch=compute_90,code=sm_90 --threads 4 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=block_sparse_attn_cuda -D_GLIBCXX_USE_CXX11_ABI=0 -ccbin /home/tangzecheng.tzc/.conda/envs/xattn/bin/x86_64-conda-linux-gnu-cc
FAILED: /nas/tangzecheng.tzc/x-attention-main/Block-Sparse-Attention-main/build/temp.linux-x86_64-cpython-310/csrc/block_sparse_attn/src/flash_fwd_split_hdim96_fp16_sm80.o 
/home/tangzecheng.tzc/.conda/envs/xattn/bin/nvcc --generate-dependencies-with-compile --dependency-output /nas/tangzecheng.tzc/x-attention-main/Block-Sparse-Attention-main/build/temp.linux-x86_64-cpython-310/csrc/block_sparse_attn/src/flash_fwd_split_hdim96_fp16_sm80.o.d -I/nas/tangzecheng.tzc/x-attention-main/Block-Sparse-Attention-main/csrc/block_sparse_attn -I/nas/tangzecheng.tzc/x-attention-main/Block-Sparse-Attention-main/csrc/block_sparse_attn/src -I/nas/tangzecheng.tzc/x-attention-main/Block-Sparse-Attention-main/csrc/cutlass/include -I/home/tangzecheng.tzc/.conda/envs/xattn/lib/python3.10/site-packages/torch/include -I/home/tangzecheng.tzc/.conda/envs/xattn/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -I/home/tangzecheng.tzc/.conda/envs/xattn/lib/python3.10/site-packages/torch/include/TH -I/home/tangzecheng.tzc/.conda/envs/xattn/lib/python3.10/site-packages/torch/include/THC -I/home/tangzecheng.tzc/.conda/envs/xattn/include -I/home/tangzecheng.tzc/.conda/envs/xattn/include/python3.10 -c -c /nas/tangzecheng.tzc/x-attention-main/Block-Sparse-Attention-main/csrc/block_sparse_attn/src/flash_fwd_split_hdim96_fp16_sm80.cu -o /nas/tangzecheng.tzc/x-attention-main/Block-Sparse-Attention-main/build/temp.linux-x86_64-cpython-310/csrc/block_sparse_attn/src/flash_fwd_split_hdim96_fp16_sm80.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -O3 -std=c++17 -U__CUDA_NO_HALF_OPERATORS__ -U__CUDA_NO_HALF_CONVERSIONS__ -U__CUDA_NO_HALF2_OPERATORS__ -U__CUDA_NO_BFLOAT16_CONVERSIONS__ --expt-relaxed-constexpr --expt-extended-lambda --use_fast_math -lineinfo -gencode arch=compute_80,code=sm_80 -gencode arch=compute_90,code=sm_90 --threads 4 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=block_sparse_attn_cuda -D_GLIBCXX_USE_CXX11_ABI=0 -ccbin /home/tangzecheng.tzc/.conda/envs/xattn/bin/x86_64-conda-linux-gnu-cc
nvcc warning : incompatible redefinition for option 'compiler-bindir', the last value of this option was used
In file included from /nas/tangzecheng.tzc/x-attention-main/Block-Sparse-Attention-main/csrc/block_sparse_attn/src/flash_fwd_launch_template.h:14,
                 from /nas/tangzecheng.tzc/x-attention-main/Block-Sparse-Attention-main/csrc/block_sparse_attn/src/flash_fwd_split_hdim96_fp16_sm80.cu:5:
/nas/tangzecheng.tzc/x-attention-main/Block-Sparse-Attention-main/csrc/block_sparse_attn/src/flash_fwd_kernel.h:10:10: fatal error: cute/algorithm/copy.hpp: No such file or directory
   10 | #include <cute/algorithm/copy.hpp>
      |          ^~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
In file included from /nas/tangzecheng.tzc/x-attention-main/Block-Sparse-Attention-main/csrc/block_sparse_attn/src/flash_fwd_launch_template.h:14,
                 from /nas/tangzecheng.tzc/x-attention-main/Block-Sparse-Attention-main/csrc/block_sparse_attn/src/flash_fwd_split_hdim96_fp16_sm80.cu:5:
/nas/tangzecheng.tzc/x-attention-main/Block-Sparse-Attention-main/csrc/block_sparse_attn/src/flash_fwd_kernel.h:10:10: fatal error: cute/algorithm/copy.hpp: No such file or directory
   10 | #include <cute/algorithm/copy.hpp>
      |          ^~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
In file included from /nas/tangzecheng.tzc/x-attention-main/Block-Sparse-Attention-main/csrc/block_sparse_attn/src/flash_fwd_launch_template.h:14,
                 from /nas/tangzecheng.tzc/x-attention-main/Block-Sparse-Attention-main/csrc/block_sparse_attn/src/flash_fwd_split_hdim96_fp16_sm80.cu:5:
/nas/tangzecheng.tzc/x-attention-main/Block-Sparse-Attention-main/csrc/block_sparse_attn/src/flash_fwd_kernel.h:10:10: fatal error: cute/algorithm/copy.hpp: No such file or directory
   10 | #include <cute/algorithm/copy.hpp>
      |          ^~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
fatal   : Could not open input file /tmp/tmpxft_000f46d4_00000000-8_flash_fwd_split_hdim96_fp16_sm80.compute_80.cpp1.ii

@JerryGJX

hello,do u solve the problem? I use A100 and meet the same problem.

[lzx1413]

you should run 'git submodule update --init --recursive' to add the cutlass submodule