Add NVidia Datacenter (V100/A100/H100) GPU assignment to CloudStack guests

[MejdiB]

ISSUE TYPE

• Enhancement Request

COMPONENT NAME

Apache Cloudstack UI, Agent, KVM, API, etc.

CLOUDSTACK VERSION

Apache Cloudstack 4.15.1

CONFIGURATION

Small setup with 1 Zone, 1 Pod, 2 Clusters: One for CPU virtualization purposes and the other for GPU virtualization purposes.
The GPU cluster consists of 2 nodes with 4 Nvidia A100 cards per node.

OS / ENVIRONMENT

Red Hat Enterpise Linux 8.4 with KVM/QEMU/libvirt (Cloudstack Agent) installed on several GPU hosts with several NVidia A100 AI cards and Cloudstack setup and hosts running on RHEL 8.4 as well.

SUMMARY

Currently, support for Nvidia A100 GPU is not available in Cloudstack. Under Service Offerings -> New Service Offering, I can only define the "old" GRID K1 and K2 cards as GPU ressources. GUI and underlying functionality support for A100 and/or V100 cards are missing. Furthermore, no GPU ressources are displayed zone-wide e.g. on the dashboard when adding a host with A100 cards. As far as I know, KVM virtualization with RHEL 8.4 is supported.

STEPS TO REPRODUCE

Adding a host/node with A100 cards to a cluster in Cloudstack. No Zone-wide GPU ressources are displayed on the dashboard as well as no possibilites to define Compute Offerings with A100 capabilities on the Cloudstack GUI.

EXPECTED RESULTS

Add the A100 card as an option in Service Offering > GPU. When chosen, users can define how man vGPUs to pass to the virtual machine. On the dashboard of Cloudstack, the available amount of GPU ressources are displayed.

[PPisz]

Very good idea.

• The installation of NVIDIA software on the KVM side should be on the Administrator side.
• GPU Offering should allow you to define NVIDIA vGPU profile, (this information can be easily retrieved from the appropriate path) e.g .:

nvidia-355 - CUDA - num_heads=4, frl_config=60, framebuffer=1024M, max_resolution=5120x2880, max_instance=32
nvidia-356 - CUDA - num_heads=4, frl_config=60, framebuffer=2048M, max_resolution=7680x4320, max_instance=16
nvidia-357 - CUDA - num_heads=4, frl_config=60, framebuffer=4096M, max_resolution=7680x4320, max_instance=8
nvidia-358 - CUDA - num_heads=4, frl_config=60, framebuffer=8192M, max_resolution=7680x4320, max_instance=4
nvidia-359 - CUDA - num_heads=4, frl_config=60, framebuffer=16384M, max_resolution=7680x4320, max_instance=2
nvidia-360 - CUDA - num_heads=4, frl_config=60, framebuffer=32768M, max_resolution=7680x4320, max_instance=1

• The list of generated UUIDs for the vGPU can be passed to the CS or the CS can generate the vGPU
• The whole process should allow you to connect a vGPU device as offering (compute type), not as extraconfig.
• The dasboard should show the amount of used / max vGPU

The manual vGPU connection process is described here:
https://lab.piszki.pl/cloudstack-kvm-and-running-vm-with-vgpu/

Automation should allow you to use GPU Offering instead of extraconfig, that's enough.

[rg9975]

Has this been discussed for priority in a release? These are common GPU chips and having this implemented directly as a supported service offering would be ideal.

[btzq]

Im sure there are going to be lots of GPU models being released, not only from Nvidia, but AMD as well?

Any plans to support AMD GPUs? Or at least the implementation should be vendor neutral with the option of easily supporting future GPU versions?

[DaanHoogland]

@btzq , I am pretty sure it is vendor neutral, just not model neutral yet. sounds like a necessary improvement for the exiting times to come though. (cc @weizhouapache any idea?)

[btzq]

@DaanHoogland well i sure hope its vendor neutral! Although NVIDIA is popular right now, their H100 and B200 is costing around USD 30K-40K per unit! While AMD MI300X (which is better performing than the H100) is 1/4 the price.