[Bug]: gpu-feature-discovery panics in GetVendorSpecificCapability() on malformed PCI vendor capability length

[wy0824]

1. Quick Debug Information

• OS/Version(e.g. RHEL8.6, Ubuntu22.04): Ubuntu 22.04.5 LTS
• Kernel Version: 6.8.0-117-generic
• Container Runtime Type/Version(e.g. Containerd, CRI-O, Docker): containerd 2.1.6
• K8s Flavor/Version(e.g. K8s, OCP, Rancher, GKE, EKS): Kubernetes v1.33.6

2. Issue or feature description

When gpu-feature-discovery starts on one H100 node, it crashes immediately with:

panic: runtime error: slice bounds out of range [224:0]

The panic comes from:

github.com/NVIDIA/k8s-device-plugin/internal/vgpu.(*PCIDevice).GetVendorSpecificCapability()
    /build/internal/vgpu/pciutil.go:140

From tracing the current source, the panic happens because GetVendorSpecificCapability() slices d.Config[pos : pos+length] using a capability length read from PCI config space without validating that pos+length <= len(d.Config).

Relevant code path:

if id == PciCapabilityVendorSpecificID {
    capability := d.Config[pos+PciCapabilityListID : pos+PciCapabilityListID+length]
    return capability, nil
}

This seems to be a missing bounds check in the vGPU detection path used by gpu-feature-discovery.

Some additional context:

• GPU Operator version: v26.3.2
• gpu-feature-discovery image: nvcr.io/nvidia/k8s-device-plugin:v0.19.2
• Affected node type: NVIDIA-H100-80GB-HBM3
• Driver version on the affected node: 580.105.08
• Other nodes in the same cluster are healthy, including another NVIDIA-H100-80GB-HBM3 node with the same driver version. Only one node consistently reproduces the panic.

This makes it look like the node has at least one device exposing an unexpected or malformed vendor-specific capability record, and the current code path panics instead of treating it as an invalid capability and continuing.

Expected behavior:

• gpu-feature-discovery should not panic on malformed or unexpected PCI capability data.
• It should either skip the invalid device/capability or return a normal error that the existing non-fatal vGPU label path can handle.

Actual behavior:

• gpu-feature-discovery enters CrashLoopBackOff and the node loses GFD refreshes because a single malformed capability record causes a process-wide panic.

3. Information to attach (optional if deemed irrelevant)

Common error checking:

• The output of nvidia-smi -a on your host
• Your docker configuration file (e.g: /etc/docker/daemon.json)
• The k8s-device-plugin container logs
• The kubelet logs on the node (e.g: sudo journalctl -r -u kubelet)

Additional information that might help better understand your environment and reproduce the bug:

• Docker version from docker version
• Docker command, image and tag used
• Kernel version from uname -a
• Any relevant kernel output lines from dmesg
• NVIDIA packages version from dpkg -l '*nvidia*' or rpm -qa '*nvidia*'
• NVIDIA Container Toolkit version from nvidia-ctk --version

Current pod log:

I0701 02:22:16.311991    3105 main.go:163] Starting OS watcher.
I0701 02:22:16.312371    3105 main.go:168] Loading configuration.
I0701 02:22:16.313443    3105 main.go:180]
Running with config:
{
  "version": "v1",
  "flags": {
    "migStrategy": "single",
    "failOnInitError": true,
    "gdrcopyEnabled": null,
    "gdsEnabled": null,
    "mofedEnabled": null,
    "useNodeFeatureAPI": true,
    "deviceDiscoveryStrategy": "auto",
    "plugin": {
      "passDeviceSpecs": null,
      "deviceListStrategy": null,
      "deviceIDStrategy": null,
      "cdiAnnotationPrefix": null,
      "nvidiaCTKPath": null,
      "containerDriverRoot": "/driver-root"
    },
    "gfd": {
      "oneshot": false,
      "noTimestamp": false,
      "sleepInterval": "1m0s",
      "outputFile": "/etc/kubernetes/node-feature-discovery/features.d/gfd",
      "machineTypeFile": "/sys/class/dmi/id/product_name"
    }
  },
  "resources": {
    "gpus": null
  },
  "sharing": {
    "timeSlicing": {}
  },
  "imex": {}
}
I0701 02:22:16.427009    3105 factory.go:58] Using NVML manager
I0701 02:22:16.446257    3105 main.go:214] Start running
2026/07/01 02:22:16 WARNING: unable to detect IOMMU FD for [0000:18:00.0 open /sys/bus/pci/devices/0000:18:00.0/vfio-dev: no such file or directory]: %!v(MISSING)
2026/07/01 02:22:16 WARNING: unable to detect IOMMU FD for [0000:2a:00.0 open /sys/bus/pci/devices/0000:2a:00.0/vfio-dev: no such file or directory]: %!v(MISSING)
2026/07/01 02:22:16 WARNING: unable to detect IOMMU FD for [0000:3a:00.0 open /sys/bus/pci/devices/0000:3a:00.0/vfio-dev: no such file or directory]: %!v(MISSING)
2026/07/01 02:22:16 WARNING: unable to detect IOMMU FD for [0000:5d:00.0 open /sys/bus/pci/devices/0000:5d:00.0/vfio-dev: no such file or directory]: %!v(MISSING)
2026/07/01 02:22:16 WARNING: unable to detect IOMMU FD for [0000:ab:00.0 open /sys/bus/pci/devices/0000:ab:00.0/vfio-dev: no such file or directory]: %!v(MISSING)
2026/07/01 02:22:16 WARNING: unable to detect IOMMU FD for [0000:ba:00.0 open /sys/bus/pci/devices/0000:ba:00.0/vfio-dev: no such file or directory]: %!v(MISSING)
2026/07/01 02:22:16 WARNING: unable to detect IOMMU FD for [0000:db:00.0 open /sys/bus/pci/devices/0000:db:00.0/vfio-dev: no such file or directory]: %!v(MISSING)
I0701 02:22:16.950143    3105 main.go:160] Exiting
panic: runtime error: slice bounds out of range [224:0]

goroutine 1 [running]:
github.com/NVIDIA/k8s-device-plugin/internal/vgpu.(*PCIDevice).GetVendorSpecificCapability(0x1bb13c0?)
    /build/internal/vgpu/pciutil.go:140 +0x1eb
github.com/NVIDIA/k8s-device-plugin/internal/vgpu.(*Lib).Devices(0x2baff29f7980?)
    /build/internal/vgpu/vgpu.go:74 +0x126
github.com/NVIDIA/k8s-device-plugin/internal/lm.vgpuLabeler.Labels({{0x1f292a0?, 0x2baff2551c60?}})
    /build/internal/lm/vgpu.go:41 +0x33
github.com/NVIDIA/k8s-device-plugin/internal/lm.list.Labels({0x2baff23b4b40, 0x2, 0x2baff1dad720?})
    /build/internal/lm/list.go:36 +0x82
github.com/NVIDIA/k8s-device-plugin/internal/lm.list.Labels({0x2baff1dad840, 0x2, 0x1f292a0?})
    /build/internal/lm/list.go:36 +0x82
main.(*gfd).run(0x2baff1dad998, 0x2baff2524a80)
    /build/cmd/gpu-feature-discovery/main.go:272 +0xf4
main.start(0x2baff252a680, 0x2baff24fad80)
    /build/cmd/gpu-feature-discovery/main.go:221 +0x5b5
main.main.func1(0x2baff252a680?)
    /build/cmd/gpu-feature-discovery/main.go:48 +0x17
github.com/urfave/cli/v2.(*Command).Run(0x2baff252f340, 0x2baff252a680, {0x2baff2a84010, 0x1, 0x1})
    /build/vendor/github.com/urfave/cli/v2/command.go:276 +0x7c2
github.com/urfave/cli/v2.(*App).RunContext(0x2baff2294a00, {0x1f3ddd0, 0x32e43e0}, {0x2baff2a84010, 0x1, 0x1})
    /build/vendor/github.com/urfave/cli/v2/app.go:333 +0x5a5
github.com/urfave/cli/v2.(*App).Run(...)
    /build/vendor/github.com/urfave/cli/v2/app.go:307
main.main()
    /build/cmd/gpu-feature-discovery/main.go:126 +0x12a5

[amarkdotdev]

Opened #1928 for this. Bounds check before slicing the vendor capability so GFD skips bad PCI entries instead of panicking.