Feature Request: ROCm support (AMD GPU)

[gururise]

Could you please add official AMD ROCm support to this library? An unofficial working port already exists:

https://github.com/broncotc/bitsandbytes-rocm

Thank You

[TimDettmers]

Amazing! Thank you for bringing this to my attention. I will try to get in touch with the author of the ROCm library and support AMD GPUs by default.

[YellowRoseCx]

Amazing! Thank you for bringing this to my attention. I will try to get in touch with the author of the ROCm library and support AMD GPUs by default.

that would be AMAZING! especially with you recently adding 8 bit support. I tried to make my own merge of the forks but I don't really know what I'm doing and don't think I did it correctly

[anonymous721]

If the ROCm fork does get merged in, would the Int8 Matmul compatibility improvements also work for AMD GPUs?

[deftdawg]

@TimDettmers, curious if AMD support any nearer to being merged? @agrocylo made a PR (#296) based somewhat on @broncotc's fork...

[gururise]

EDIT: A slightly newer version branched from v0.37 available here:
https://github.com/Titaniumtown/bitsandbytes-rocm/tree/patch-2

[elukey]

The Wikimedia foundation is really interested in the ROCm support too, since Nvidia is not viable for us due to open-source constraints. @TimDettmers we offer any help (testing/review/etc..) to help merge this feature, it would be really great for the ML open source ecosystem. Thanks in advance!

[Aria-K-Alethia]

EDIT: A slightly newer version branched from v0.37 available here: https://github.com/Titaniumtown/bitsandbytes-rocm/tree/patch-2

Hi,
I'm also seeking an AMD-GPU-compatible version.
I tried your patch-2 version but the code still cannot work.
The error info looks like:

  File "/home/.local/lib/python3.8/site-packages/bitsandbytes/autograd/__init__.py", line 1, in <module>
    from ._functions import undo_layout, get_inverse_transform_indices
  File "/home/.local/lib/python3.8/site-packages/bitsandbytes/autograd/_functions.py", line 9, in <module>
    import bitsandbytes.functional as F
  File "/home/.local/lib/python3.8/site-packages/bitsandbytes/functional.py", line 17, in <module>
    from .cextension import COMPILED_WITH_CUDA, lib
  File "/home/.local/lib/python3.8/site-packages/bitsandbytes/cextension.py", line 74, in <module>
    raise RuntimeError('''
RuntimeError: 
        CUDA Setup failed despite GPU being available. Inspect the CUDA SETUP outputs above to fix your environment!
        If you cannot find any issues and suspect a bug, please open an issue with detals about your environment:
        https://github.com/TimDettmers/bitsandbytes/issues

I use AMD MI200 card.
Do you have any idea on this?
Many thanks.

[PatchouliPatch]

Hello, I was wondering how far-off the ROCm support is. I'm trying to see if my 7900XTX will be useful in a project of mine. The Llama2 quick start guide makes use of bitsandbytes, and as far as I know there isn't any other alternatives.

[jiagaoxiang]

Found this rocm version of bitsandbytes: https://github.com/Lzy17/bitsandbytes-rocm/tree/main

[mauricioscotton]

The only rocm version that worked for me on GFX900 was this one: https://github.com/agrocylo/bitsandbytes-rocm
All the others failed to compile/install
(Rocm 5.2)