Suggestion for explicit compatibility matrix

[xiaoran007]

🚀 Feature

Add an explicit compatibility matrix for supported PyTorch and CUDA versions in the documentation or release notes.

Motivation

xFormers releases are strongly coupled to specific PyTorch and CUDA versions.
Currently, users need to manually infer compatibility by checking wheel tags or matching installation errors (e.g., ABI mismatch).
While the project provides prebuilt wheels for recent combinations such as PyTorch 2.8.0 + CUDA 12.8, there is no centralized compatibility matrix indicating which versions are officially supported per release.

This lack of visibility makes it difficult for users and downstream projects to determine the correct installation target.

Pitch

Include a concise compatibility table — for example, in the README or in each release note — listing the supported PyTorch and CUDA versions per xFormers release.
It could look something like:

xFormers version	Supported PyTorch	Supported CUDA	Notes
0.0.30	2.x–2.8	11.8, 12.6, 12.8	Latest tested combinations
0.0.xx	2.x–2.x	11.8, 12.1	...

Alternatives

None

Additional context

None

[jovan2009]

I agree. It's difficult or impossible to find a compatible xformers version, especially if you want to use it with a newer or nightly torch build and you don't have what is needed in terms of knowledge and software installed to compile it yourself.
Currently these are the latest beta /rc wheels available if want to install with pip install --pre and there is no indication with what torch+cuda are compiled (therefore compatible). I assume they are some cuda 12.9 builds but I don't know, I resorted a while ago to just compile myself from source for my own needs, currently using cuda 13 and torch nightly.
https://pypi.org/project/xformers/#files

[n0valis]

Just came here to check for why xformers 0.0.34.dev1096 requires torch==2.9.0,
because but you have torch 2.9.1+cu130 which is incompatible.

[jovan2009]

@n0valis

Just came here to check for why xformers 0.0.34.dev1096 requires torch==2.9.0, because but you have torch 2.9.1+cu130 which is incompatible.

Try to see if it works something like "pip install -U --pre --no-deps xformers". If it finds a version built with cuda 13 it might, even it is built with a slightly different version of torch. At least that worked for me before I started to build my own xformers builds.

Edit: it stopped working for me when all the xformers builds it could find where at max built with cuda 12.8 or 12.9, when I was already on cuda 13 torch builds.

[n0valis]

Try to see if it works something like "pip install -U --pre --no-deps xformers". If it finds a version built with cuda 13 it might, even it is built with a slightly different version of torch. At least that worked for me before I started to build my own xformers builds.

Edit: it stopped working for me when all the xformers builds it could find where at max built with cuda 12.8 or 12.9, when I was already on cuda 13 torch builds.

I cloned and used
TORCH_CUDA_ARCH_LIST="8.9;12.0" pip wheel .
--wheel-dir ~/xformers/wheels/ --no-cache-dir --no-build-isolation --no-deps
got
xformers-0.0.34+3f91ad6f.d20251115-cp39-abi3-linux_x86_64.whl

It's a delicate combination with flash-attn 2.8.3 and torch 2.9.1+cu130. Not tested yet though. But no dependency errors anymore.

[jovan2009]

Try to see if it works something like "pip install -U --pre --no-deps xformers". If it finds a version built with cuda 13 it might, even it is built with a slightly different version of torch. At least that worked for me before I started to build my own xformers builds.
Edit: it stopped working for me when all the xformers builds it could find where at max built with cuda 12.8 or 12.9, when I was already on cuda 13 torch builds.

I cloned and used TORCH_CUDA_ARCH_LIST="8.9;12.0" pip wheel . --wheel-dir ~/xformers/wheels/ --no-cache-dir --no-build-isolation --no-deps got xformers-0.0.34+3f91ad6f.d20251115-cp39-abi3-linux_x86_64.whl

It's a delicate combination with flash-attn 2.8.3 and torch 2.9.1+cu130. Not tested yet though. But no dependency errors anymore.

I am too using a separate flash attention 2.8.3 wheel (If that's what you meant). Built from source directly from https://github.com/Dao-AILab/flash-attention.git Building xformers with the flash attention provided whithin its submodules always fails for me. When I found out that building it separately from its original repository works together with xformers during use I stopped trying building it from whitin xformers.

[n0valis]

I am too using a separate flash attention 2.8.3 wheel (If that's what you meant). Built from source directly from https://github.com/Dao-AILab/flash-attention.git Building xformers with the flash attention provided whithin its submodules always fails for me. When I found out that building it separately from its original repository works together with xformers during use I stopped trying building it from whitin xformers.

Yes I also built it from Dao-AILab source. I think that's still fresh and warm.
Do you know a test routine that proofes xformers is actually using flash-attn?
When using in ComfyUI it starts with
xformers version: 0.0.34+3f91ad6f.d20251115
Using xformers attention
So not sure whether that is not its own attention.
I would like to test it with a little script, but don't know how...

[jovan2009]

Do you know a test routine that proofes xformers is actually using flash-attn?

Not really, beside the fact that it shows it enabled when I run python -m xformers.info. That, and the fact that whenever I start ComfyUI I see some sort of message coming from torch that a "I don't remember what exactly" module or function was replaced or overridden by "I don't remember what exactly" flash attention module or function. Kinda wonky message too, it ends with an open parenthesis, something like bla-bla bla bla(, like it should be something after that but it is missing.
All I know for sure is that it doesn't crash. If it's used it happens probably mostly during VAE encoding and decoding, based on the fact that I see in comfyUI's console xformers and VAE together in the same sentence. The rest of the time responsible for optimizations is probably Sageattention 2.2. I base my assumption on the fact that if you try to specify to ComfyUI in starting options to use flash attention you get a message that the option can't be used together with --use-sageattention

[n0valis]

I base my assumption on the fact that if you try to specify to ComfyUI in starting options to use flash attention you get a message that the option can't be used together with --use-sageattention

Yeah well together is probably a contradiction for xformers.
Because the line changes from
No start option: Using xformers attention
and
--use-flash-attention: Using xformers attention
to
--use-sage-attention: Using sage attention

The problem for me is that if flash-attn is not installed and you won't use a starting option (otherwise you'll get flash-attn package must be installed first.) the line would still be Using xformers attention.
So we don't really know what happens under the hood. But I guess xformers is clever enough to find and use it.

[jovan2009]

I base my assumption on the fact that if you try to specify to ComfyUI in starting options to use flash attention you get a message that the option can't be used together with --use-sageattention

Yeah well together is probably a contradiction for xformers. Because the line changes from No start option: Using xformers attention and --use-flash-attention: Using xformers attention to --use-sage-attention: Using sage attention

The problem for me is that if flash-attn is not installed and you won't use a starting option (otherwise you'll get flash-attn package must be installed first.) the line would still be Using xformers attention. So we don't really know what happens under the hood. But I guess xformers is clever enough to find and use it.

Well, there are other features in xformers beside flash attention, I guess they will be used even without flash attention installed. I'm just a graphic designer, not a developer, how to check more thoroughly or where to look for that information inside code bases (ComfyUI, xformers or anywhere else), is completely over my "pay grade", so to say.