GPU passthrough availability?

[vivekpatani]

Would I be able to passthrough GPU devices to the container either atomically or in slices? Thanks.

[egernst]

Hey @vivekpatani - we do not currently support this. If you have feedback on what you'd like to see specifically, let us know. Converting this to a discussion.

[marty-sullivan]

Honestly, this would be the only reason I would ever use this. If you aren't offering virtualization / passthrough of the GPU and other Apple hardware, why would anyone want this over using Docker Desktop?

[gbraad]

@AndreasKunar
All sources are here:

git clone https://copr-dist-git.fedorainfracloud.org/git/slp/mesa-krunkit/mesa

The reason for the COPR and RPM, is to allow this easily to be consumed from eg. Podman Desktop. It would be possible to also build this on OBS. It is not restricted... just convenience for our usecase.
Note: @slp can probably give a better answer around this.

[slp]

My issue with containers/krunkit is, that Vulkan remoting from the container to the host to my knowledge requires a certain patched mesa-vulkan-driver,, only available in Fedora and similar. Limiting containers to be based on Fedora/Redhat/Centos to me limits usability.

The need for a patched mesa comes from the fact that virtio-gpu doesn't have the ability to negotiate blob alignment with the host. We extended the virtio spec to address this missing feature, and we'll be sending a kernel patch implementing that extension to the spec. Once that's merged, we can propose another patch to upstream mesa to fix the issue the right way.

In the meantime, there's no option but to use a patched mesa (we can't upstream it because that's basically a hack). That said, there's nothing in those downstream patches that ties them up to a Fedora/Red Hat/CentOS distro. Other distros can pick them up and apply them as needed.

[AndreasKunar]

gbraad+slp - thanks a lot for the clarification and for implementing it in the first place.

[DemiMarie]

Worth noting that Vulkan remoting is not a secure solution for untrusted workloads. One needs kernel-userspace API forwarding for that. On Linux, that would forward ioctls; on macOS, I presume it would forward IOKit operations.

[yongjer]

such as ollama or pytorch can use mps in the container

[PengYicong]

True, it's such a waste for Macs especially those with large memory and abundant GPU cores size to only run on CPU mode to do AI workloads.

[mavenugo]

am curious to learn this use-case. Users of ollama or pytorch can still use MPS while using the native macOS. Why is supporting it as a linux container important?

[okaris]

@mavenugo supporting it as a linux container is important. if it's linux it's a bonus because many systems, especially production systems are running on linux hosts or containers.

[PylotLight]

am curious to learn this use-case. Users of ollama or pytorch can still use MPS while using the native macOS. Why is supporting it as a linux container important?

Do you not understand the point and benefit of containerised workloads?
Being able to run a cluster of apps and services all orcastrated in one central location rather than manually running some command to stand up a temporary client/server. Having everything in a single image over manually cloning a repo and setting up deps to then always have a terminal session open somewhere to have one or multiple servers running, is immensely easier beyond compare.. does this cover it even a little?

[okaris]

@egernst not sure if my issue belongs here or not but I think it's relevant.

apple/containerization#46

to quote my use case, which is pretty much the same with @yongjer

I’m building inference.sh, a platform that orchestrates AI workloads across cloud and local machines. On Linux and Windows (WSL2) we use Docker for containerization. On macOS, we currently manage runtimes directly on the host for performance, but the goal is to containerize cleanly across all platforms. Getting GPU access inside containers on Apple Silicon is the missing piece.

[qdrddr]

We need GPU pass through (preferable) or some workaround like Docker/Podman did for macOS with model runner to use AI on macOS.
And we need FlashAttention, it’s simply a must if you want to compete with Windows/Linux with Nvidia for GenAI.

[qdrddr]

Metal flash attention for python
https://github.com/bghira/universal-metal-flash-attention

[frob]

This was the first thing I was hoping for when I heard Apple was making a native container solution. Honestly, I have no reason to switch from podman or docker if nothing new is added. The existing solutions work well enough now. Please do something well that the existing solutions do poorly.

[spamshaker]

I totally agree, we have many alternatives that can't to this as well as this brand new solution :)
Only KVM without possibility to use the expensive (M Series) hardware without hacking, so far...

[mavenugo]

Curious to understand the use-case. Will you be able to expand your use-case to have GPU device made available to Linux container via Containerization while one could use the same GPU natively by running applications outside the container :-) ... I understand the irony of this question in this project. But this is something I'm trying to understand.

[okaris]

@mavenugo i think people would be just as happy if there was a macos container that had gpu passthrough, the goal is to have a controlled environment, and not touch the host as much as we can

[alshdavid]

I'd also like to see sharing of the onboard GPU with Vulkan support on the guest.

Support for passing through a thunderbolt/usb4 eGPU to the Linux container would be fantastic too.

[mavenugo]

Can you please share more on this use-case and how will it make your life better ?

[alshdavid]

Vulkan support enables the use of hardware agnostic AI workloads via a Vulkan backend. It also enables the potential for hardware accelerated graphical applications to be virtualized (like games, assuming we can implement something like looking glass on top of the containerization implementation)

eGPU passthrough enables the same things however with the use of more powerful accelerators that feature native CUDA/ROCm/Vulkan on the guest. Again, great for ML and potentially gaming.

To get around this currently, I SSH into an external PC and my MacBook Pro acts exclusively as a thin client. When travelling, I have had to buy a seperate Mini PC which I take with me. Despite my MacBook having the hardware capability to handle my workloads, without access to common APIs (like Vulkan) or without the ability to attach an external GPU (to gain access to those APIs within a VM), it can only ever be a thin client.

[andrewcrook]

[okaris]

There is https://developer.apple.com/documentation/paravirtualizedgraphics so we are actually not far from it. If your project needs/wants containers then probably venvs are not the solution you want. I doubt anyone asking for this is unaware of venv options. I use Docker + UV for what I'm trying to achieve they are not mutually exclusive

[DemiMarie]

Apple deliberately used a VM-per-container approach. I expect this is for security, which makes me doubt they will be okay with insecure GPU acceleration approaches like passing Vulkan commands through to the host. Secure GPU acceleration without hardware support (for SR-IOV or PCI passthrough) requires exposing the host kernel driver to the guest and running the shader compiler there. This means either custom container images with a (presumably proprietary) Apple-specific runtime, or providing an interface that Mesa can use and porting Mesa to M3 and M4.

[DemiMarie]

Yup. That ensures that a malicious or compromised container can't just use a Linux kernel vulnerability to compromise another container. It also needs a VM escape.

[LandonTClipp]

PCI passthrough doesn't require hardware support, only SR-IOV does. Maybe I am misunderstanding what you are saying.

[DemiMarie]

Apple GPUs are not behind an IOMMU and cannot be passed through to a guest.

[LandonTClipp]

That's really bizarre. Thank you for your insight!

[DemiMarie]

It’s for performance reasons. The GPU has its own MMU controlled by the kernel driver.

[DemiMarie]

I want to see support for external GPU passthrough, without having to trust the external GPU to not emulate a keyboard or mouse.

[DemiMarie]

Thunderbolt devices are PCI devices and can be passed through as such.

[DemiMarie]

I have no idea what IOMMU group Thunderbolt PCI tunnels use. If they use the same group as a host device (USB controller?), then Thunderbolt should simply not be used.

[c0nleyinnnn]

Linux is very necessary for scientific computing in bioinformatics most of the time. Apple's native container support is very exciting, but the lack of support for native Metal will undoubtedly limit Apple silicon unified-memory design. I am currently using my Mac for bioinformatics analysis. Although pytorch (mps) runs well most of the time, Mac is still a second-class citizen on the hardware in my research field, even though I personally tested the overall analysis process and the performance of Mac is very satisfactory. I hope the development team can pay enough attention to the great value of Mac in academic research and seriously consider adding native Metal support to the container.

[mavenugo]

Can you please share more details on your use-case and why is Linux essential for this particular use-case ? Especially you mentioned that pytorch (mps) works well on native macOS. I understand the irony of this question in this project. But knowing a solid use-case will go a long way to justify investments in this area.

[jtkiley]

I'm not in bioinformatics, but a lot of data science and academic workflows are like this.

I don't install Python on my Macs (I use three regularly) at all anymore. I simply create containers (via devcontainers in VSCode) with all of the configuration I need, which is then in the repo. When pulling it anywhere (Mac or PC), VS Code builds the container, and I can get to work. It also helps with issues on the host side like needing libraries (via Linux package manager like apt) for a particular project, but having to install it at the host system level and potentially dealing with conflicts. So, going container-only is quite appealing in general for data science.

GPU compute is often orders of magnitude faster than CPU compute. Originally, this was mostly helpful for ML (PyTorch and tensorflow). But, now, it's useful for local LLMs (see Docker's feature to work around the GPU passthrough issue on Macs), which are useful both for development and sometimes how something may be deployed. Beyond that, many other parts of the data science ecosystem are working on GPU compute (often via Nvidia RAPIDS libraries), like the work on polars dataframes.

Maybe the easiest way to see the value is to try a GPU-enabled workflow on Windows with an nvidia GPU versus a Mac. On Windows and Linux, nvidia GPU passthrough simply works with no work or configuration. Many libraries or model runners look for CUDA devices automatically. On the Mac, it simply isn't possible in a container, which is materially limiting now and is likely to get worse. That's why Docker made their workaround, but it only works for LLMs (and is a a bit of a configuration hassle).

In sum, perhaps the arguments are something like this:

• Containerized workflows are important for academics (for collaboration, portability, host stability, and reproducibility) and industry data science workflows more generally. It generates a ton of value for little work.
• Linux is particularly attractive for containers, in part because that's where all the work has been done. Deciding to use containers practically means you're going to use Linux. It also helps that distros have a robust package management ecosystem. They also run everywhere (e.g. GitHub codespaces).
• GPU passthrough greatly accelerates existing workflows, particularly in hot areas like ML and LLMs, and increasingly across data science more generally.
• On nvidia hardware, GPU passthrough simply works, with essentially no configuration, indirection, or hassle. It's been that way for years.
• Adding GPU passthrough moves Macs from a material competitive disadvantage (i.e. many GPU things can't be done for only software reasons; the small subset that is possible still requires cumbersome workarounds) compared to PCs with Nvidia GPUs into a superior competitive position (i.e. memory gets very expensive on Nvidia cards compared to unified memory on Macs; Mac CPU cores are already great; Mac GPUs are a lot slower than Nvidia cards, but the memory makes it compelling and good enough). In other words, not needing to spend $4-5k on a PC will pay for a lot of high-margin Mac model, memory, and SoC upgrades, from people who already use/prefer Macs.
• Engagement has been high on Mac GPU support issues for years across the data science ecosystem. I think it's fair to say that some variant of GPU support (originally native and now passthrough), has been the single top ask for people doing data science and similar academic research for several years. In my view, by a mile, it's the single biggest caveat to the overall data science experience on (Apple Silicon) Macs being otherwise quite good.
• Having GPU passthrough would greatly improve the value scaling from entry-level to higher-end Macs. Now, most of the value comes from scaling up RAM, and that's less important (for non-GPU) than it used to be. Unleashing the GPU puts the key differentiator to work in those Max and Ultra SoCs.

That's more than I planned to write (on an iPhone), and I'm happy to expand on anything if that helps clarify the value.

[scottrbaxter]

@mavenugo You seem to be asking several individuals about their use-case for containerization and why this abstraction would improve quality of life. I'm genuinely confused as to why there is such focus on this... essentially you're asking why containerization itself has become so popular over the years. it certainly makes rapid development significantly easier, cleaner and much more portable. There are no shortage of reasons users would be interested in GPU accessibility in Linux containers on a macOS host, and the hardware is insanely powerful compared to the energy required to run it... Is this really so difficult to comprehend why the community would be able to utilize this feature? Ideally, i would like to keep individual components abstracted away from other services and the host, especially in dev environments where several services are able to run alongside each other. of course users can develop on the dedicated host, but build/test/deploy workflow is not nearly as clean compared to templating a base image with pinned requirements in a containerized deployment. being the maintainer of this project, i'm certain you're well aware of all of this... so the real question is, what is the hesitation with implementing GPU access within this native containerization solution? no doubt it is complex, but this seems like such an obvious interest for a feature request. the only reason i could understand hesitation is if there is a known limitation with current macOS and existing hardware... and reading through this thread seems like this topic is trying to be avoided, or at least delayed for whatever reason. i sincerely hope this is not the case. any chance that GPU access is already being considered internally?

[DemiMarie]

@andrewcrook I'm proposing forwarding IOKit (?) calls from the guest to the host, as that is where the security boundary is. This requires a conformant Vulkan runtime in the guest that supports Apple GPUs, and the only such Vulkan runtime is Mesa on Asahi. That's why I think Apple should contribute this support to the Asahi project! MoltenVK would require that the containers contain an Apple-specific (and presumably proprietary) plugin, which I don't think would go over very well.

[mertalev]

Just to clarify, would such an approach allow for (near) native-level GPU performance, or would it require significant overhead?

[AndreasKunar]

Just to clarify. DMR/Docker Model Runner implements an openAI API compatible LLM server based on llama.cpp in the host and exposes the APi to the containers. It has no performance downside, but its just that, a specific LLM server. The Vulkan API redirects of Podman,… introduce a >= 30% penalty vs. GPU-native according to my measurements (see my medium.com articles for details). NVIDIAs GPU access from containers, based on CUDA has no measurable overhead to me (OK, it needs some alignment of driver/CUDA versions between host and containers). For me, the desired implementation for Apple to procude advantages would be MLX or Metal APIs available in the containers/hypervisor. An implementation not only for Linix containers, but also for macOS VMs. Vulkan-remoting is already implemented by other container solutions, no advantage for using Apple‘s containers (except VM isolation, which the others could probably also easily implement)

[DemiMarie]

VM isolation is not simple to implement. The reason is that API forwarding approaches are insecure: the user-mode driver is not¹ a security boundary, and it is not safe to pass untrusted input to it. Only the kernel-mode driver is secure against malicious input. If you are interested in more details, I presented on GPU acceleration at Xen Project Summit 2024.

Since the user-mode driver is not a security boundary, it must not run on the host. Instead, it must run in the guest and make RPCs to the host instead of system calls. The Linux version of this is virtio-GPU native contexts, which was presented at X Developer Conference 2022.

Footnotes

1. Exceptions include WebGL and WebGPU, where shaders from the web are untrusted, and images and memory buffers received from other processes, which are not trusted. ↩

[DemiMarie]

@mertalev System call forwarding via an async API gives very close to native performance for graphics workloads: 90% or more is expected, and 98% or more is typical. For compute workloads that do a lot of memory mapping and unmapping larger performance hits are expected. In particular, retryable page faults are not supported so all guest memory that the GPU may access must be explicitly pinned.

[auzi68]

I'm actually quite amazed this wasn't one of the first things added when designing this project. Looking online, there's massive amounts of articles where people are frustrated by not being able to do this without work arounds or performance loss. This should 100% be apart of this project. For me, without this, I really don't see a reason to use it instead of Docker or Podman.

[frob]

Thats because the feature is currently blocked by Apple's hypervisor not supporting GPU passthrough

I think the surprise in the lack of this feature is compounded by the fact that it was built by Apple. If Apple built this you'd think they could add it to their own hypervisor.

[ivanfioravanti]

Any plans to add this for v1 release? This would be a real differentiator against any other competitors out there.

[alshdavid]

Would love to be able to attach an NVIDIA or AMD eGPU by thunderbolt dock and have access to Vulkan & CUDA/ROCm within the VM container 🙏

[diogoviannaaraujo]

At least give us a mac guest on mac host container with gpu sharing, if linux its not an option, would help a lot with running automated isolated environments

[qdrddr]

PoC for metal flash attention with Python bindings, tested for images.
https://github.com/bghira/universal-metal-flash-attention

[andrewcrook]

I dont see how this is relevant to GPU pass through?

[qdrddr]

It’s not. But both are quite anticipated on Apple Silicon and would serve the same purpose: development and local use of LLMs on Mac

[kpouget]

POC of llama.cpp running at near-native speed inside a Linux container (easy to test with Podman Desktop and RamaLama and libkrun VMs)
https://developers.redhat.com/articles/2025/09/18/reach-native-speed-macos-llamacpp-container-inference

That's not a general purpose solution though, it can only works for applications relying on the GGML library (llama.cpp, whisper.cpp not tested but should work).

It follows the same idea of the Mesa/Venus/Vulkan API forwarding already available in libkrun VMs.

[kpouget]

I'll have to review what "a 'trusted-environment' llama.cpp server-implementation" means exactly, and what remains of the container when running this way

This does NOT resolve the security-boundaries needed in containers, e.g. if you want to run not fully trusted software in containers, which require GPU-access - unlike as already implemented for NVIDIA-CUDA,....

yes, this level of isolation requires driver and/or hardware level assistance ...

[AndreasKunar]

Apologies for me not being clear in my wording. With my misleading "trusted environment" comment, I meant that the Docker Model Runner's (DMR) llama-cpp-based server got implemented and verified by the Docker team and is part of it's code-base. If you trust Docker as supplier, you also trust the DMR feature. No special trust-isolation, but it's not a separate open-source component, which would need a supplier/component verification (including its supply-chain). For me, this is why Docker+DMR for macOS was/is interesting. And it also packages models as OCI-artifacts. But it's very specific.

However, I think Apple still should implement some kind of Apple-Silicon GPU API virtualization for GPU-access from containers. Vulkan or other (MLX, Metal,...).

[DemiMarie]

API virtualization isn't secure. One needs to virtualize the API exposed by the kernel driver.

[DemiMarie]

That can be entirely in userspace, but it requires allowing Mesa to access the macOS kernel driver and providing the needed APIs for that.

[qdrddr]

The problem with libkrun is that workaround is applicable only for some popular LLMs. But not all missable AI models. Absolute majority of the models you still have to run on the host, and can't be containerized. libkrun is a half-baked solution.

This so much Mikita AI development on macOS, simply because GPU pass through is prohibited. Apple needs to either allow the GPU passthrough or find a better solution than libkrun.

[riley-1995]

@mavenugo

Apple Silicon is limiting robotics and STEM students. We need GPU/OpenGL support in Linux VMs for essential tools like ROS 2 and Gazebo.

I’m a CS graduate student studying ML and robotics while using an Apple M4 Max for coursework and research. I run ROS 2 Humble and Gazebo Fortress inside an Ubuntu VM (via UTM) because these tools aren’t fully supported natively on macOS. ROS 2 provides official prebuilt binaries only for Ubuntu. Packages like Cartographer and Navigation2 rely on Ubuntu’s apt-based dependency chain, and Gazebo’s ROS 2 plugins are built and distributed through the same Ubuntu package ecosystem.

The main limitation is that Virtualization.framework doesn’t expose GPU or OpenGL 3.3+ to Linux guests. Gazebo’s GUI immediately fails with “OpenGL 3.3 not supported,” leaving only headless mode. That blocks visualization and debugging, which are central to robotics education and simulation.

The hardware is more than capable, but without GPU/OpenGL passthrough, students and developers can’t use Apple Silicon for modern robotics or simulation workflows.

Extending ParavirtualizedGraphics or adding GPU/3D acceleration passthrough for Linux guests would make Apple Silicon a first-class platform for robotics, AI, and research. It would let us stay within the macOS ecosystem instead of relying on external, non-Apple devices or resorting to a compromised VM workflow with impactful limitations.

[DemiMarie]

It should be possible to use a (slow) software renderer via llvmpipe.

[riley-1995]

I don't believe this will work for my case. Reading through the llvmpipe documentation, I do not see support for arm processors.
https://docs.mesa3d.org/drivers/llvmpipe.html

[DemiMarie]

It definitely works on Arm.

[brickheadbs]

I'm just adding my comment to show I also agree with everything said above.

I tend to speak direct, so not trying to be a jerk here :) My apologies if I don't sound kind.

I understand there are security implications any time you share a resource. But why would you ever NOT want to enable access to a large part of your system resources? If containers are CPU only, you are eliminating 60% of what I got a MacStudio for. I've been doing some homelab project this year, and it is SO much more difficult than it should be. Why handicap the products in any way? It just works would win pros over as well as 'normal' users. I'm not a dev, just a long time tech enthusiast.

Final rant: To me this is right up there with a young guy being able to develop Whisky for FREE to be able to play Windows games on mac. Why wouldn't apple produce that natively? they are really good at virtualisation. I love my mac, but the biggest thing that tempts me away is compatibility.

OK, that's all I have to say. :D If anyone with pull reads this, realize when you make our lives easy we throw money at you

[DemiMarie]

Final rant: To me this is right up there with a young guy being able to develop Whisky for FREE to be able to play Windows games on mac. Why wouldn't apple produce that natively? they are really good at virtualisation. I love my mac, but the biggest thing that tempts me away is compatibility.

This is a problem that looks simple, but is actually extremely hard. The reason (which should be in some sort of FAQ) is:

• Containers run Linux, not macOS.
• It is very difficult to include external libraries into containers. Having the libraries built in to the containers is much more reliable.
• There is an implementation of OpenGL/OpenCL/Vulkan that supports Apple GPUs and even GPU virtualization. However:
  • It is M1/M2 only.
  • It expects to talk to the Linux driver, not the macOS one.

The last two points are the key. GPU acceleration for Linux containers on Apple silicon relies on the Asahi project. To provide GPU virtualization for Linux containers in a secure and upstreamable way, Apple would need to add M3/M4/M5 support to Mesa, and then provide a host virtio-GPU native context implementation that talked to the macOS kernel.

There is an easier route, which is to rely on KosmicKrisp in the container and expose a paravirtualized version of Metal. However, I highly doubt Apple will go this route. From the very beginning, this framework has run one container per VM. This implies a very strong focus on isolating containers from each other and the host. Running the Metal runtime on the host would not meet this requirement. Furthermore, I don’t know if upstream Mesa maintainers would accept it.

For M1 and M2 Macs, your best option for GPU virtualization is to run Asahi Linux on the host and use its native context implementation, which does work quite well.

[slp]

There is an easier route, which is to rely on KosmicKrisp in the container and expose a paravirtualized version of Metal. However, I highly doubt Apple will go this route. From the very beginning, this framework has run one container per VM. This implies a very strong focus on isolating containers from each other and the host. Running the Metal runtime on the host would not meet this requirement. Furthermore, I don’t know if upstream Mesa maintainers would accept it.

Until Apple comes up with their own solution, krunkit does support GPU acceleration for Linux guests/containers on Apple Silicon Macs using libkrun, Venus and MoltenVK.

Podman Desktop supports krunkit/libkrun since quite a while (it's now the default virtualization engine for Podman Desktop), and Lima just gained support for it too. You can also use it standalone with you favorite Linux distro, just make sure the context (VM or container in a VM) where you run the AI workload has a patched Mesa package installed (we're working on removing this requirement, but since it requires changes in the Linux kernel and Mesa is going to take a while to get upstream and trickle down to every distro).

[DemiMarie]

There is an easier route, which is to rely on KosmicKrisp in the container and expose a paravirtualized version of Metal. However, I highly doubt Apple will go this route. From the very beginning, this framework has run one container per VM. This implies a very strong focus on isolating containers from each other and the host. Running the Metal runtime on the host would not meet this requirement. Furthermore, I don’t know if upstream Mesa maintainers would accept it.

Until Apple comes up with their own solution, krunkit does support GPU acceleration for Linux guests/containers on Apple Silicon Macs using libkrun, Venus and MoltenVK.

Podman Desktop supports krunkit/libkrun since quite a while (it's now the default virtualization engine for Podman Desktop), and Lima just gained support for it too. You can also use it standalone with you favorite Linux distro, just make sure the context (VM or container in a VM) where you run the AI workload has a patched Mesa package installed (we're working on removing this requirement, but since it requires changes in the Linux kernel and Mesa is going to take a while to get upstream and trickle down to every distro).

Would it be possible to run the MoltenVK (or, in the future, KosmicKrisp) in a very tightly sandboxed process? Otherwise, this defeats the isolation the VM provides.