Making GPU invisble to Hyprland

[Libadoxon]

I'm currently trying to follow https://github.com/blu3bird/vfio-nvidia-hot-swap/wiki to make my vm GPU "hot swappable". For this I need to tell everything else to not use it. I'm stuck on telling wayland to completely hide it. The guide uses a udev rule for this:

# assign nvidia card to seat1 but remove master-of-seat tag to force CanGraphical=false
# this causes wayland login managers and compositors to completely ignore it
TAG=="seat", ENV{ID_FOR_SEAT}=="drm-pci-0000_01_00_0", ENV{ID_SEAT}="seat1", TAG-="master-of-seat"

I tried this as well but I can't start the vm, it complains about Hyprland using the GPU. I know about AQ_DRM_DEVICES and have set it to /dev/dri/card2 which is my iGPU. But this seems to not be enough.
Instead of a NVDIA GPU I have an AMD one, so two AMD GPUs in total. This doesn't seem like the problem here but who knows.

Hyprland is also able to still see the GPU, which I think shouldn't be the case, eg running hyprctl systeminfo still lists both cards

[Terrails]

Not sure if this issue is exactly related to this, but I am having a similar issue where Hyprland uses an Nvidia eGPU I have plugged in through a USB4 dock, even though AQ_DRM_DEVICES is explicitly set to only the AMD iGPU of the laptop.

It doesn't directly use the GPU so that it is visible through nvidia-smi, but the issue is that the moment that the eGPU is plugged in, I cannot unload the nvidia_drm module as it is in use by Hyprland, which makes it impossible to hotswap. Exiting Hyprland via SUPER + M and then unloading the nvidia modules works flawlessly, but I wouldnt consider that hotswap anymore.

I have found the root of this issue via lsof. It seems like Hyprland loads a shared library called /usr/lib/libnvidia-allocator.so.570.153.02 on my system and this makes it impossible to hotswap (specifically unplug) the GPU when Hyprland is running without things starting to act weird if unplugged. This should usually work without any issues.

Basically to reproduce

1. Start Hyprland with the Thunderbolt/USB4 eGPU not plugged in.
2. Confirm via lsof | grep nvidia and lsmod | grep nvidia that nothing related to nvidia is loaded
3. Plug in the eGPU
4. Immediately, the nvidia modules are loaded and there are a bunch of file handles to /usr/lib/libnvidia-allocator.so.570.153.02 returned by lsof. All of these handles are held by Hyprland.