Currently GPU accelerated Lima VMs are only possible on macOS using the Krunkit driver. We should consider adding native GPU passthrough(VFIO) to Linux hosts using the Qemu driver to provide near native CUDA/ROCm performance for AI workloads and possibly some gaming(when we support Windows guests in the future)🕹️.
However, there’s a caveat: PCI passthrough only works with multiple graphics cards. A workaround does exists for passing a single graphics card. The downside is that you will loose display on the host side(if running a desktop env) and have to use SSH to control it from the guest unless you have a pair of dGPUs or a single dGPU with iGPU.
A rough workflow
- Step 1: Enabling
IOMMU on the host for device isolation (Requires sudo)
- Step 2: Identifying the target GPU's PCI address. The whole group has to be passed like audio controller and any other devices present (No sudo)
- Step 3: Unbinding from the host and binding it to
VFIO(/dev/vfio/<group_number>), also changing the ownership (Requires sudo)
- Step 4: Launching the Lima VM by adding the PCI address of their GPU to a new field
pciAddresses under QEMUOpts (No sudo)
- Step 5: Teardown and restoration of the GPU from
vfio-pci (Requires sudo)
Workaround for the privileged needs
The steps 1, 3 & 5 are the main hurdles here. Maybe we can
- Create a separate standalone executable like
lima-vfio-helper like how we currently do for socket_vmnet?
$ limactl sudoers | sudo tee /etc/sudoers.d/lima
$ limactl start --name=default template://vmnet
- Use Polkit?
- Provide a detailed doc and make the user configure the sudo parts?
Originally posted by @unsuman in #4849
Currently GPU accelerated Lima VMs are only possible on macOS using the Krunkit driver. We should consider adding native GPU passthrough(VFIO) to Linux hosts using the Qemu driver to provide near native CUDA/ROCm performance for AI workloads and possibly some gaming(when we support Windows guests in the future)🕹️.
However, there’s a caveat: PCI passthrough only works with multiple graphics cards. A workaround does exists for passing a single graphics card. The downside is that you will loose display on the host side(if running a desktop env) and have to use SSH to control it from the guest unless you have a pair of dGPUs or a single dGPU with iGPU.
A rough workflow
IOMMUon the host for device isolation (Requires sudo)VFIO(/dev/vfio/<group_number>), also changing the ownership (Requires sudo)pciAddressesunderQEMUOpts(No sudo)vfio-pci(Requires sudo)Workaround for the privileged needs
The steps 1, 3 & 5 are the main hurdles here. Maybe we can
lima-vfio-helperlike how we currently do forsocket_vmnet?Originally posted by @unsuman in #4849