Any Way To Get XFormers with AMD GPU

[KhunDoug]

I've seen several people posting different ways to get XFormers working with an AMD GPU. I've tried so many of these and none render an image. Instead I get a list of error text below the picture. And some just error out at startup. I have been able to get Python 3.11 functioning with torch and Stable Diffusion functions with the DirectML setting. This is a Windows 11 24H2 install with a Ryzen 5950X and an XFX 6800 XT GPU. Using DirectML I can see the GPU is getting used. But I read where some users claim XFormers gives a nice speed bump.

So has anyone found the secret sauce to get everything clicking? Or is this scheduled for a future release?

[ride5k]

xformers? no. but zluda is working, and working well--much better than DML.

[Soulreaver90]

I've tried but haven't gotten it to work. I know xformers is pushing an update that will fix issues with amd and bring support upto rocm 6.2/6.3. I'm not sure if anything needs to be done on the A1111 backend to support it however.

[rrunner77]

Zluda is working fine for A1111 - I had no issue with it. You can not do training.

XFormers

I was able to get it running on WSL2 after 3 days :)

python -m xformers.info
xFormers 0.0.30+6e8b9de.d20250206
memory_efficient_attention.ckF:                    available
memory_efficient_attention.ckB:                    available
memory_efficient_attention.ck_decoderF:            available
memory_efficient_attention.ck_splitKF:             available
memory_efficient_attention.cutlassF-pt:            unavailable
memory_efficient_attention.cutlassB-pt:            unavailable
[email protected]:             unavailable
[email protected]:             unavailable
[email protected]:             unavailable
[email protected]:             unavailable
memory_efficient_attention.triton_splitKF:         available
indexing.scaled_index_addF:                        available
indexing.scaled_index_addB:                        available
indexing.index_select:                             available
sp24.sparse24_sparsify_both_ways:                  available
sp24.sparse24_apply:                               available
sp24.sparse24_apply_dense_output:                  available
sp24._sparse24_gemm:                               available
[email protected]:                 available
[email protected]:                        available
swiglu.dual_gemm_silu:                             available
swiglu.gemm_fused_operand_sum:                     available
swiglu.fused.p.cpp:                                available
is_triton_available:                               True
pytorch.version:                                   2.7.0.dev20250206+rocm6.3
pytorch.cuda:                                      available
gpu.compute_capability:                            11.0
gpu.name:                                          AMD Radeon RX 7900 XTX
dcgm_profiler:                                     unavailable
build.info:                                        available
build.cuda_version:                                None
build.hip_version:                                 None
build.python_version:                              3.10.16
build.torch_version:                               2.7.0.dev20250206+rocm6.3
build.env.TORCH_CUDA_ARCH_LIST:                    None
build.env.PYTORCH_ROCM_ARCH:                       None
build.env.XFORMERS_BUILD_TYPE:                     None
build.env.XFORMERS_ENABLE_DEBUG_ASSERTIONS:        None
build.env.NVCC_FLAGS:                              None
build.env.XFORMERS_PACKAGE_FROM:                   None
source.privacy:                                    open source

but unfortunately when the generation of image is running on the last step it show a error:

NotImplementedError: No operator found for `memory_efficient_attention_forward` with inputs:
         query       : shape=(1, 4096, 1, 512) (torch.float32)
         key         : shape=(1, 4096, 1, 512) (torch.float32)
         value       : shape=(1, 4096, 1, 512) (torch.float32)
         attn_bias   : <class 'NoneType'>
         p           : 0.0
    `ckF` is not supported because:
        max(query.shape[-1], value.shape[-1]) > 256
        dtype=torch.float32 (supported: {torch.bfloat16, torch.float16})

Even xformers show that ckF is ok... For now no luck.
I am using ROCm 6.3 and torch 2.7 with 7900xtx.

it seems that there needs to be done something also on the A1111 to work with the xformers on AMD GPU.

What I found on the xformers git is that on cuda there are also cutlast and those do not exist on AMD GPUs. AMD supports only ckF and they can not have float32. even when I go for float16 it throw other error:

NotImplementedError: No operator found for `memory_efficient_attention_forward` with inputs:
      query       : shape=(1, 4096, 1, 512) (torch.float16)
      key         : shape=(1, 4096, 1, 512) (torch.float16)
      value       : shape=(1, 4096, 1, 512) (torch.float16)
      attn_bias   : <class 'NoneType'>
      p           : 0.0
 `ckF` is not supported because:
     max(query.shape[-1], value.shape[-1]) > 256

until now no luck to use xformers.