You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Without MIG (time slicing or MPS). When multiple logical GPUs (replicas) correspond to one physical GPU UUID, an Xid error on that UUID should report all of them as unhealthy to the kubelet. Currently only one logical GPU is reported. Since the physical GPU itself is broken, every logical GPU tied to it should become unhealthy.
With MIG. Even when multiple MIG instances inside one physical GPU are broken, at most one is reported as unhealthy, and in some cases none at all. If the value of parentToDeviceMap[uuid] happens to be a pointer to a device that is not broken, the GI and CI check rejects the event and nothing is reported.
1. Quick Debug Information
2. Issue or feature description
parentToDeviceMap[uuid]happens to be a pointer to a device that is not broken, the GI and CI check rejects the event and nothing is reported.