NPU Linux support

[GreyXor]

Hello AMD,

Kudos on advancing AI capabilities with gaia! I recently got my hands on an AMD Ryzen AI 9 HX 370 with XDNA2 and successfully compiled and loaded the amdxdna driver.

Could you add support for linux and amdxdna driver ?

Thanks!

Context:

• [Feature request] Any plans for AMD XDNA AI Engine support on Ryzen 7x40 processors? ggml-org/llama.cpp#1499
• https://www.amd.com/en/technologies/xdna.html
• https://www.amd.com/en/products/processors/laptop/ryzen/ai-300-series/amd-ryzen-ai-9-hx-370.html
• https://github.com/amd/xdna-driver
• https://www.amd.com/en/products/processors/consumer/ryzen-ai.html

[Mizokuiam]

Hey @GreyXor

The core issue here boils down to Gaia's current dependence on certain Windows-specific APIs and libraries. Porting this over to Linux, specifically to accommodate the amdxdna driver's interface, will involve a decent chunk of refactoring. We'll need to abstract away those Windows dependencies and introduce a cross-platform layer to handle the driver interactions. This is similar to some work I did on the ROCm platform a while back, so I have a good idea of the challenges involved.

Right now, we don't have a concrete timeline for full Linux support, but we're actively exploring different architectural approaches. One idea being tossed around is leveraging something like Vulkan for hardware abstraction, but that has its own set of trade-offs.

In the meantime, if you're comfortable working with cross-compilation, you might be able to create a temporary workaround by wrapping the amdxdna driver calls in a compatibility layer. However, that would be a non-trivial undertaking.

Ping me if you have other questions down the line.

[kovtcharov]

Thank you for the kind words, @GreyXor. To add to @Mizokuiam comment, we don't have much to share at this point in regards to Linux support, however given the obvious demand we are seeing from the community, this has spurred some discussions internally. Once we have something conrete to share, I will post here and let you know.

[wangnan789]

I would also like to see support for Linux and ***BSD,
Thank you for your work
this translation is from Bing Translatese

[tcsenpai]

Just another +1 on this, I am in the same situation of OP (HX 370, xdna drivers compiled and working) and this would be a game changer

[tangmeng09]

I have buy ASUS Pro16 laptop on last year which has AI 9 365 CPU/GPU/NPU. Linux is my favourite platform. Expecting the lovely GPU/NPU to join the AI calculation, and expecting ROCm support of gfx1150 on next release

[CleverLittleMaker]

+1 from another Linux AI researcher. I just got my brand spankin' new AMD Ryzen AI Max+ 395 128GB system, and um, Linux support is highly desirable but sadly lacking currently for those who do AI stuff.

To be honest, I'm kinda surprised that GAIA was developed in Windows first, since everyone I know doing AI work uses Linux.... But I'm not complaining! Please keep up with the hard work and progress. :)

Cheers,

Eric

[kovtcharov]

Quick update on Linux support, the LLM tool GAIA uses under the hood built by our team called Lemonade is now supporting Linux via llama.cpp/Vulkan backend. @CleverLittleMaker, this means you'll be able to run any GGUF model on your AMD Ryzen AI Max+ 395 128GB system using the Ryzen AI CPU or iGPU (NPU support will come later).

GAIA support for Linux will also follow shortly in about 1-2 weeks. Thank you all for the feedback, we take it seriously and try to build the tools that matter most to everyone here.

More details on the latest Lemonade Server release here. Try it out here.

[njmo]

Hello @kovtcharov, thank you for your efforts. I was wondering if there are any updates, as it’s been three weeks since your last message

[infinitevalence]

+1 for linux support, the fact that we dont have NPU support in Ollama or LM Studio, and also dont have linux support in GAIA shows a real lack of support or belief in local AI.