NVIDIA_VISIBLE_DEVICES=void is in the container spec file twice

[lx1036]

nvidia-container-toolkit: 1.18.0
containerd: 1.7.22

When I create a GPU Pod with one GPU card like nvidia.com/l40s: '1', and check the container spec file in the node:
cat /data/containerd-state/io.containerd.runtime.v2.task/k8s.io/da111ad6430ef318fd9941d1db03b613a5e559cd471f289432dc2125280befd7/config.json

the NVIDIA_VISIBLE_DEVICES env is like this, and env NVIDIA_VISIBLE_DEVICES=void has twice:

I check the code, https://github.com/NVIDIA/nvidia-container-toolkit/blob/v1.18.0/internal/modifier/cdi.go#L171-L187 , when devices=[runtime.nvidia.com/gpu=GPU-XXX]，the uniqueModes will be {"auto", "auto"}, so the code will add env NVIDIA_VISIBLE_DEVICES=void twice.

So, my question is:
(1) env NVIDIA_VISIBLE_DEVICES=void should be in the container spec file? Is it a bug?
(2) env NVIDIA_VISIBLE_DEVICES=void has twice, Is it a bug?

[elezar]

The presence of NVIDIA_VISIBLE_DEVICES=void in the container is correct. This indicates that the nvidia-container-runtime correctly applied the modifications to the container. Note that envvars are appended and later envvars take precedence. (We set the envvar to void to ensure that no further modification takes place if the nvidia runtime is invoked after something like containerd made modifications to the spec when native CDI support is used.

With that said, the fact that NVIDIA_VISIBLE_DEVICES=void appears twice in the list of envvars is a bug, but should not affect behaviour. Thanks for bringing this to our attention.