Would this work with apple silicon?

[openSourcerer9000]

Qlora on apple is pretty janky, given that this works on AMD, could apple silicon be supported?

[woct0rdho]

Most of the kernels here are written in Triton. Triton on Apple is still an ongoing work. There are a few attempts in triton-lang/triton#4824 , and I guess currently the most usable one is https://github.com/imperatormk/triton-ext , but I haven't tried.

You also need to port the HIP kernels in https://github.com/woct0rdho/torch-ggml-ops to MPS. It should not be too hard, given that llama.cpp already works on Apple.