feat(scheduler): add node nouse gpuuuid function

[ZhengW22]

What type of PR is this?

What this PR does / why we need it:
This PR adds the capability to disable GPUs at the node level by applying annotations to nodes. GPUs matching the specified UUIDs will no longer be allocated to any pods.

The implementation works by setting the used count of the corresponding node GPUs to their maximum capacity when calculating nodeUsage, effectively occupying those resources. This approach maintains compatibility with scheduling logic for different types of GPU cards.

Which issue(s) this PR fixes:
No.

Special notes for your reviewer:
No.

Does this PR introduce a user-facing change?:
No.

[archlitchi]

please rebase the code to pass the CI

[hami-robot]

New changes are detected. LGTM label has been removed.

[ZhengW22]

@Shouren @archlitchi Hello, I have rebased the code and fix some problems.

[ZhengW22]

@Shouren Hello, can you help me to review the new code?

[wawa0210]

/gemini review

[gemini-code-assist]

Code Review

This pull request introduces a useful feature to disable specific GPUs on a node via annotations. The overall approach of filtering out disabled devices from the scheduler's node cache is sound. However, the description of the implementation in the PR body is inaccurate. It states that disabled GPUs are marked as fully utilized, but the code actually removes them from the node's device list. Please update the description to reflect the actual implementation.

I've found a couple of critical bugs. One is that UUIDs from annotations are not correctly trimmed, which would cause the feature to fail with whitespace in the annotation value. Another is a reference to a constant that doesn't seem to be defined, which will cause a compilation failure. I've also provided some suggestions for refactoring to improve code clarity and for adding a test case to improve coverage. Please review the comments.

[gemini-code-assist]

The constant metax.MetaxNoUseUUID is used here, but it does not appear to be defined in the metax package based on the provided files. This will likely cause a compilation error. Please ensure this constant is defined, for example in pkg/device/metax/device.go or a similar file, with a value like "metax-tech.com/nouse-gpuuuid".

[gemini-code-assist]

There's a bug in this section. The disableGPUUUID variable, which may contain leading/trailing whitespace, is used as a map key. The trimmed version id should be used instead to ensure correct matching. Additionally, the map initialization logic can be simplified to be more idiomatic.

                            if vendorWithDisableGPUUUIDMap[vendor] == nil {
                                vendorWithDisableGPUUUIDMap[vendor] = make(map[string]bool)
                            }
                            vendorWithDisableGPUUUIDMap[vendor][id] = true

[gemini-code-assist]

This loop for filtering devices can be simplified for better readability. Using a continue statement would make the logic clearer and avoid the removeFlag variable. This also resolves the shadowing of the ok variable, which is a good practice to avoid.

    for _, d := range nodeInfo.Devices {
        if disableGPUUUIDMap, ok := vendorWithDisableGPUUUIDMap[d.DeviceVendor]; ok {
            if _, disabled := disableGPUUUIDMap[d.ID]; disabled {
                klog.V(5).Infof("Disabling gpu with uuid: %s", d.ID)
                continue
            }
        }
        tmp = append(tmp, d)
    }

[gemini-code-assist]

The test suite for rmDeviceByNodeAnnotation is good, but it would be beneficial to add a test case that verifies that UUIDs with leading/trailing whitespace in the annotation value are handled correctly. This would ensure the strings.TrimSpace logic is effective and prevent regressions.

For example:

{
    name: "Test remove device with whitespace in annotation",
    args: args{
        nodeInfo: &device.NodeInfo{
            Node:    &corev1.Node{ObjectMeta: metav1.ObjectMeta{Annotations: map[string]string{nvidia.GPUNoUseUUID: " " + id1 + " "}}},
            Devices: []device.DeviceInfo{{DeviceVendor: nvidia.NvidiaGPUDevice, ID: id1}},
        },
    },
    want: []device.DeviceInfo{},
},

[wawa0210]

@ZhengW22 Could you please take a look at the critical comments in gemini-code-assist?