Skip to content

bugfix(noderesourcetopology): reject non-single-numa-node policies in NodeResourceTopology plugin #886

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

googs1025
Copy link
Member

@googs1025 googs1025 commented Apr 16, 2025

What type of PR is this?

/kind bug

What this PR does / why we need it:

  • reject non-single-numa-node policies in NodeResourceTopology plugin

Which issue(s) this PR fixes:

Fixes #885

Special notes for your reviewer:

Does this PR introduce a user-facing change?

reject non-single-numa-node policies in NodeResourceTopology plugin 

@k8s-ci-robot k8s-ci-robot added kind/bug Categorizes issue or PR as related to a bug. do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels Apr 16, 2025
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: googs1025
Once this PR has been reviewed and has the lgtm label, please assign huang-wei for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot requested a review from seanmalloy April 16, 2025 01:23
@k8s-ci-robot k8s-ci-robot added the size/S Denotes a PR that changes 10-29 lines, ignoring generated files. label Apr 16, 2025
Copy link

netlify bot commented Apr 16, 2025

Deploy Preview for kubernetes-sigs-scheduler-plugins canceled.

Name Link
🔨 Latest commit eace5f5
🔍 Latest deploy log https://app.netlify.com/sites/kubernetes-sigs-scheduler-plugins/deploys/67ff4c454a7f710008577eb9

@k8s-ci-robot k8s-ci-robot added release-note Denotes a PR that will be considered when it comes time to generate release notes. and removed do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. labels Apr 16, 2025
@@ -180,6 +180,11 @@ func singleNUMAPodLevelHandler(lh logr.Logger, pod *v1.Pod, zones topologyv1alph
return nil
}

func rejectNonSingleNUMANodeHandler(lh logr.Logger, _ *v1.Pod, _ topologyv1alpha2.ZoneList, nodeInfo *framework.NodeInfo) *framework.Status {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we add another handler to handle this case

… NodeResourceTopology plugin

Signed-off-by: googs1025 <[email protected]>
@googs1025
Copy link
Member Author

I'm not sure if my understanding is wrong. We have the following policy.

           // RestrictedTopologyManagerPolicy is a mode in which kubelet only allows
	// pods with optimal NUMA node alignment for requested resources
	RestrictedTopologyManagerPolicy = "restricted"
	// BestEffortTopologyManagerPolicy is a mode in which kubelet will favour
	// pods with NUMA alignment of CPU and device resources.
	BestEffortTopologyManagerPolicy = "best-effort"
	// NoneTopologyManagerPolicy is a mode in which kubelet has no knowledge
	// of NUMA alignment of a pod's CPU and device resources.
	NoneTopologyManagerPolicy = "none"
	// SingleNumaNodeTopologyManagerPolicy is a mode in which kubelet only allows
	// pods with a single NUMA alignment of CPU and device resources.
	SingleNumaNodeTopologyManagerPolicy = "single-numa-node"

Do we only have the none case to handle (not sure)? Perhaps a better approach is for us to provide a prefilter prescore plugin that directly skip status whenpolicy == none.

@k8s-ci-robot
Copy link
Contributor

@googs1025: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
pull-scheduler-plugins-integration-test-master eace5f5 link true /test pull-scheduler-plugins-integration-test-master

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@googs1025
Copy link
Member Author

@ffromani friendly ping 😄

If you have some time, could you give me some advice? I don't know much about this plugin, but I'd love to learn more, and it would be great if I could help.

@ffromani
Copy link
Contributor

Thanks for the fix. Codewise looks a good direction. The problem is designwise. We need to review the use cases and decide how do we want to treat these scenarios. There were reasons why the code currently auto-passes node without alignment requirements. We need to recover the past context, resume the conversation, re-evaluate and update the docs.

@PiotrProkop
Copy link
Contributor

@googs1025 what if we have cluster with mixed configuration and some nodes have single-numa-node policy enabled and some don't? Then we would always filter out nodes without single-numa-node?

@ffromani
Copy link
Contributor

@googs1025 what if we have cluster with mixed configuration and some nodes have single-numa-node policy enabled and some don't? Then we would always filter out nodes without single-numa-node?

this ^ mixed nodes was the contention point also back in time. Both approaches have pros and cons.

Our (RH) experience is that users tend to have homogeneous cluster, but in the general case we support mixed clusters.

@PiotrProkop
Copy link
Contributor

@googs1025 what if we have cluster with mixed configuration and some nodes have single-numa-node policy enabled and some don't? Then we would always filter out nodes without single-numa-node?

this ^ mixed nodes was the contention point also back in time. Both approaches have pros and cons.

Our (RH) experience is that users tend to have homogeneous cluster, but in the general case we support mixed clusters.

From my experience, we sometimes have mixed configuration due to node configuration changes etc. but we still want max cluster utilization.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/bug Categorizes issue or PR as related to a bug. release-note Denotes a PR that will be considered when it comes time to generate release notes. size/S Denotes a PR that changes 10-29 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[NodeResourceTopology] Nodes with != single-numa-node topology-policy are automatically filtered-in
4 participants