SageAttention3

[Rogala]

The sageattention3_blackwell folder has appeared in the official repository. As I understand it, version 3 can already be compiled, but I get 100+ errors. ChatGPT crashed and couldn't help me solve the problem. Are you already working on this version?

[FurkanGozukara]

yes published @woct0rdho

I hope we can make it work on Windows

https://github.com/thu-ml/SageAttention/tree/main/sageattention3_blackwell

[FurkanGozukara]

i tried any my compile failed - this can compile this repo

compile_failed.txt

[deepbeepmeep]

If you modify: sageattn/blackwell/kernl_traits.h line 153 with the following it will compile on windows:
using BlkScaledConfig = flash::BlockScaledConfig<16>;

[Myoko]

How should I clone the source code to my local machine?

[woct0rdho]

Now the main branch of my repo can be compiled on Windows, but I still need some time to figure out how to port to ABI3.

Also, torch 2.9.0 RC1 (the beta version) is published. I guess the ABI will no longer change so we can start building wheels with it. However, you may encounter pytorch/pytorch#148317

[woct0rdho]

Hi all, let's test the wheels (You can download the artifact at the bottom of the webpage):

• PyTorch 2.7: https://github.com/Comfy-Org/wheels/actions/runs/18097324899
• PyTorch 2.8: https://github.com/Comfy-Org/wheels/actions/runs/18097774723
• PyTorch 2.9: https://github.com/Comfy-Org/wheels/actions/runs/18097780962

After installing the wheel, run test_sageattn3.py to see if it works. (If I understand correctly, the error can be as large as 0.3 or 0.4. Or maybe I did something wrong.)

What's known for now:

• sageattn3 only works on Blackwell GPUs (such as RTX 50xx), CUDA >= 12.8 (therefore PyTorch >= 2.7).
• sageattn3 and sageattention are two individual packages. You can install both.
• In ComfyUI, --use-sage-attention does not enable sageattn3. You need some custom node to enable it, such as ComfyUI-WanVideoWrapper, or PatchSageAttentionKJ in ComfyUI-KJNodes if you use native workflows.
• sageattn3 cannot be simply applied to Wan, otherwise there will be too much quality degradation. You need to apply sageattn2 at the first and the last time steps. See https://github.com/thu-ml/SageAttention/blob/main/sageattention3_blackwell/README.md
• PatchSageAttentionKJ node does not apply sageattn3 in the above way. ComfyUI-WanVideoWrapper supports applying sageattn3 in the above way, but we need this fix: Fix sageattn3 in Patch Sage Attention Node  kijai/ComfyUI-KJNodes#477

I've successfully ported sageattn3 to ABI3 and tested it on Linux + 50xx. However, on Windows, the wheels commonly give the CUDA error: misaligned address.

I don't have 50xx at home, and it's hard to rent a server online with Windows + 50xx, so I don't have any idea how to debug this. Let's see if someone can help.

Update: There is some progress at thu-ml#323

[Rogala]

(venv) H:\3.13.7-2.8.0-12.9-compile>pip install sageattn3-1.0.0+cu128torch2.8.0-cp39-abi3-win_amd64.whl
Processing h:\3.13.7-2.8.0-12.9-compile\sageattn3-1.0.0+cu128torch2.8.0-cp39-abi3-win_amd64.whl
Installing collected packages: sageattn3
Successfully installed sageattn3-1.0.0+cu128torch2.8.0

(venv) H:\3.13.7-2.8.0-12.9-compile>python test_sageattn3.py
q (4, 32, 64, 128) cuda:0 torch.float16
Traceback (most recent call last):
  File "H:\3.13.7-2.8.0-12.9-compile\test_sageattn3.py", line 48, in <module>
    main()
    ~~~~^^
  File "H:\3.13.7-2.8.0-12.9-compile\test_sageattn3.py", line 43, in main
    print("sage vs math:", get_rtol_atol(out_sage, out_math))
                           ~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
  File "H:\3.13.7-2.8.0-12.9-compile\test_sageattn3.py", line 12, in get_rtol_atol
    diff = (actual - expect).abs()
torch.AcceleratorError: CUDA error: misaligned address
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1
Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.

There may be an error because I have torch 2.8.0+cu129?

[jovan2009]

(venv) H:\3.13.7-2.8.0-12.9-compile>pip install sageattn3-1.0.0+cu128torch2.8.0-cp39-abi3-win_amd64.whl Processing h:\3.13.7-2.8.0-12.9-compile\sageattn3-1.0.0+cu128torch2.8.0-cp39-abi3-win_amd64.whl Installing collected packages: sageattn3 Successfully installed sageattn3-1.0.0+cu128torch2.8.0

(venv) H:\3.13.7-2.8.0-12.9-compile>python test_sageattn3.py q (4, 32, 64, 128) cuda:0 torch.float16 Traceback (most recent call last): File "H:\3.13.7-2.8.0-12.9-compile\test_sageattn3.py", line 48, in main() ~~~~^^ File "H:\3.13.7-2.8.0-12.9-compile\test_sageattn3.py", line 43, in main print("sage vs math:", get_rtol_atol(out_sage, out_math)) ~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "H:\3.13.7-2.8.0-12.9-compile\test_sageattn3.py", line 12, in get_rtol_atol diff = (actual - expect).abs() torch.AcceleratorError: CUDA error: misaligned address CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect. For debugging consider passing CUDA_LAUNCH_BLOCKING=1 Compile with TORCH_USE_CUDA_DSA to enable device-side assertions.

There may be an error because I have torch 2.8.0+cu129?

I encounter the same error while testing. And probably I have same torch version + cuda combination.
Edit: This is the error I got on wheel that I compiled myself. It compiled successfully but I got this error testing it.
Edit2: for the sake of making a proper report:

c:\B\SageAttention\sageattention3_blackwell\tests>python test_sageattn3.py
q (4, 32, 64, 128) cuda:0 torch.float16
Traceback (most recent call last):
  File "c:\B\SageAttention\sageattention3_blackwell\tests\test_sageattn3.py", line 48, in <module>
    main()
    ~~~~^^
  File "c:\B\SageAttention\sageattention3_blackwell\tests\test_sageattn3.py", line 43, in main
    print("sage vs math:", get_rtol_atol(out_sage, out_math))
                           ~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
  File "c:\B\SageAttention\sageattention3_blackwell\tests\test_sageattn3.py", line 12, in get_rtol_atol
    diff = (actual - expect).abs()
torch.AcceleratorError: CUDA error: misaligned address
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1
Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.


c:\B\SageAttention\sageattention3_blackwell\tests>python -c "import torch; print(torch.__version__)"
2.8.0+cu129

c:\B\SageAttention\sageattention3_blackwell\tests>nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2025 NVIDIA Corporation
Built on Tue_May_27_02:24:01_Pacific_Daylight_Time_2025
Cuda compilation tools, release 12.9, V12.9.86
Build cuda_12.9.r12.9/compiler.36037853_0


c:\B\SageAttention\sageattention3_blackwell\tests>

[wowgiansd-cloud]

I got itm how can i use it on wan?

[woct0rdho]

@wowgiansd-cloud You can use kijai's ComfyUI-WanVideoWrapper after this change: kijai/ComfyUI-WanVideoWrapper#1321

[msola-ht]

Hi all, let's test the wheels (You can download the artifact at the bottom of the webpage):大家好，让我们测试一下 wheel（您可以在网页底部下载工件）：

• PyTorch 2.7: https://github.com/Comfy-Org/wheels/actions/runs/18097324899
• PyTorch 2.8: https://github.com/Comfy-Org/wheels/actions/runs/18097774723
• PyTorch 2.9: https://github.com/Comfy-Org/wheels/actions/runs/18097780962

After installing the wheel, run test_sageattn3.py to see if it works. (If I understand correctly, the error can be as large as 0.3 or 0.4. Or maybe I did something wrong.)安装完轮子后，运行 test_sageattn3.py 查看是否工作。（如果理解正确，错误可能高达 0.3 或 0.4。或者可能是我做错了什么。）

What's known for now:目前已知信息：

• sageattn3 only works on RTX 50xx, CUDA >= 12.8 (therefore PyTorch >= 2.7).sageattn3 只在 RTX 50xx 上工作，CUDA >= 12.8（因此 PyTorch >= 2.7）。
• sageattn3 和 sageattention are two individual packages. You can install both. sageattn3 和 sageattention 是两个独立的包。您可以安装它们两个。
• In ComfyUI, --use-sage-attention does not enable sageattn3. You need some custom node to enable it, such as https://github.com/wallen0322/ComfyUI-SageAttention3在 ComfyUI 中， --use-sage-attention 不会启用 sageattn3。您需要一些自定义节点来启用它，例如 https://github.com/wallen0322/ComfyUI-SageAttention3
• sageattn3 cannot be simply applied to Wan, otherwise there will be too much quality degradation. You need to apply sageattn2 at the first and the last time steps. See https://github.com/thu-ml/SageAttention/blob/main/sageattention3_blackwell/README.mdsageattn3 不能简单地应用于 Wan，否则会有过多的质量退化。您需要在第一个和最后一个时间步应用 sageattn2。请参阅 https://github.com/thu-ml/SageAttention/blob/main/sageattention3_blackwell/README.md
• kijai's ComfyUI-WanVideoWrapper supports sageattn3, but we need this change: sageattn3 import rename kijai/ComfyUI-WanVideoWrapper#1321kijai 的 ComfyUI-WanVideoWrapper 支持 sageattn3，但我们需要这个更改： sageattn3 import rename kijai/ComfyUI-WanVideoWrapper#1321

271+128+311

(F:\AI_SD\ComfyUI_Portable\venv) F:\AI_SD\ComfyUI_Portable\ComfyUI>python "E:\Downloads\test_sageattn3.py"
q (4, 32, 64, 128) cuda:0 torch.float16
Traceback (most recent call last):
  File "E:\Downloads\test_sageattn3.py", line 48, in <module>
    main()
  File "E:\Downloads\test_sageattn3.py", line 43, in main
    print("sage vs math:", get_rtol_atol(out_sage, out_math))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "E:\Downloads\test_sageattn3.py", line 12, in get_rtol_atol
    diff = (actual - expect).abs()
           ^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: CUDA error: misaligned address
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1
Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.


(F:\AI_SD\ComfyUI_Portable\venv) F:\AI_SD\ComfyUI_Portable\ComfyUI>

[gonztok]

I got a RuntimeError

(venv) C:\AI\ComfyUI>pip install "sageattn3-1.0.0+cu128torch2.8.0-cp39-abi3-win_amd64.whl"
Processing c:\ai\comfyui\sageattn3-1.0.0+cu128torch2.8.0-cp39-abi3-win_amd64.whl
Installing collected packages: sageattn3
Successfully installed sageattn3-1.0.0+cu128torch2.8.0

[notice] A new release of pip is available: 24.2 -> 25.2
[notice] To update, run: python.exe -m pip install --upgrade pip

(venv) C:\AI\ComfyUI>python test_sageattn3.py
q (4, 32, 64, 128) cuda:0 torch.float16
Traceback (most recent call last):
  File "C:\AI\ComfyUI\test_sageattn3.py", line 48, in <module>
    main()
  File "C:\AI\ComfyUI\test_sageattn3.py", line 43, in main
    print("sage vs math:", get_rtol_atol(out_sage, out_math))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\AI\ComfyUI\test_sageattn3.py", line 12, in get_rtol_atol
    diff = (actual - expect).abs()
           ^^^^^^^^^^^^^^^^^^^^^^^
torch.AcceleratorError: CUDA error: misaligned address
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1
Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.

This is the error with CUDA_LAUNCH_BLOCKING=1

(venv) C:\AI\ComfyUI>set CUDA_LAUNCH_BLOCKING=1

(venv) C:\AI\ComfyUI>python test_sageattn3.py
q (4, 32, 64, 128) cuda:0 torch.float16
Traceback (most recent call last):
  File "C:\AI\ComfyUI\test_sageattn3.py", line 48, in <module>
    main()
  File "C:\AI\ComfyUI\test_sageattn3.py", line 42, in main
    out_sage = sageattn3_blackwell(q, k, v)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\AI\ComfyUI\venv\Lib\site-packages\sageattn3\api.py", line 146, in sageattn3_blackwell
    o_fp4 = blockscaled_fp4_attn(
            ^^^^^^^^^^^^^^^^^^^^^
  File "C:\AI\ComfyUI\venv\Lib\site-packages\sageattn3\api.py", line 132, in blockscaled_fp4_attn
    return fp4attn_cuda.fwd(qlist[0], klist[0], vlist[0], qlist[1], klist[1], vlist[1], delta_s, KL, softmax_scale, is_causal, per_block_mean, is_bf16)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\AI\ComfyUI\venv\Lib\site-packages\torch\_ops.py", line 1243, in __call__
    return self._op(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
torch.AcceleratorError: CUDA error: misaligned address
Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.

This is my setup:

pytorch version: 2.8.0+cu128
Set vram state to: NORMAL_VRAM
Device: cuda:0 NVIDIA GeForce RTX 5070 Ti : cudaMallocAsync
Python version: 3.12.7 (tags/v3.12.7:0b05ead, Oct  1 2024, 03:06:41) [MSC v.1941 64 bit (AMD64)]