Multi-backend support: Apple Silicon / Mac (call for contributors + help fleshing out the details)

[Titus-von-Koeller]

Apple Silicon support is planned between Q4/2024 - Q2/2025: We're actively looking for contributors to implement this and help us flesh out a concrete plan as well as detailed list of requirements to fulfill by anyone interested in implementing a part.

We don't have enough resources to do this work ourselves, so we're dependent on community contributions.

Please introduce yourselves, which skills you bring to the table, which things you foresee need doing and feel free to constructively comment on each other. Let's see where things evolve from there!

---

If you're interested to work on this full-time leading the implementation (with mentoring) and enabling the community on this topic: Most likely, we'll soon have limited funding for this through the Mozilla foundation), please contact us by tagging @Titus-von-Koeller and @matthewdouglas in a Github discussion on our repo. This also applies if funding is no incentive for you, we're a community project!

Let's see from there, better to contact us even if you're unsure and we see if there's a way that works for everyone.

[markuskreitzer]

I'm happy to help!

[polarNerd204]

willing to help test if needed. Running M2 ultra w 128gram 60cores.
not a developer but REALLY WANT this roadblock fixed.....along with all the insightface problems.

[neal3000]

I might be able to help, depending on what kind of skills you need.

[WenheLI]

Happy to help! Want to see some detailed plans!

[bghira]

see the torchao repo for examples of 4bit/8bit MPS kernels

[rahulmisal27]

I am happy to help!

[mrs83]

I am happy to help

[jiyzhang]

I'm happy to help

[kelvinkrisna]

Hey @Titus-von-Koeller and @matthewdouglas

I own a MacBook M1 and Post Graduate Artificial Intelligence student.

Also I have an extensive background on Software Engineering in different languages such as Delphi, C, Java, JavaScript and Python.
I hope I can contribute with this project.

[gabripo]

Hallo @Titus-von-Koeller and hello @matthewdouglas 😄

I own a MacBook M1 and play around with Huggingface .
I can play for your purpose, as well. 👍🏻
I am fluent in Python, have developed personal projects in the ML field, work as Software Developer and send PRs only if the code is robust and mantainable.

Since I experienced some issues while loading a quantized model with bitsandbytes, I am eager to help!
You can find my contact information on my profile.

[ajm896]

I would like to help! I am a software dev turned IT that's getting a bit rusty but picking for AI research. Have an M1 and M2 for testing, also AMD card in a desktop for some crossover testing.

[adhithyasash1]

happy to help, needs mentoring, thanks!

[lynkos]

I'd also like to help! I have a MacBook Pro M3 Max, am very comfortable + familiar with Python, and have some experience in ML/AI (here's one of my projects, as an example), though I'm far from an expert.

[i183x]

I would love to help! I explicitly do not have skills for this i think but i think it would be fun n i would love to learn! I have M1 so that would help, thanks for all the work so far..

[csimoes1]

I can work for free if there is some mentoring available. I have 10+ years of programming experience, much in Java but I'm capable but not an expert in Python. I also really enjoyed assembly back in college. I have a Mac M3 hardware as well. Let me know if you still need help @Titus-von-Koeller @matthewdouglas

[csimoes1]

Is anyone actually working on this? I was going to start to take a look at it, but would love to know if anyone has actually made progress?

[lazasha111211]

I can work for free if there is some mentoring available. I have 10+ years of programming experience, I have a Mac M4 Pro hardware as well. Can I offer any help? @Titus-von-Koeller @matthewdouglas

[dougdaly]

Happy to help w/ testing and troubleshooting.
I code primarily in Python w/ 10+ years experience.
I have a Mac M4 Max hardware.
@dougdaly