feat: the scheduler support SpreadByLabel constraints #3203

chaunceyjiang · 2023-02-23T09:28:54Z

What type of PR is this?
/kind feature

What this PR does / why we need it:

Which issue(s) this PR fixes:
Fixes #3187

Special notes for your reviewer:
the scheduler support SpreadByLabel constraints
Does this PR introduce a user-facing change?:

`karmada-scheduler`: the scheduler support SpreadByLabel constraints.

karmada-bot · 2023-02-23T09:29:04Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
To complete the pull request process, please assign kevin-wangzefeng after the PR has been reviewed.
You can assign the PR to them by writing /assign @kevin-wangzefeng in a comment when ready.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Signed-off-by: chaunceyjiang <[email protected]>

chaunceyjiang · 2023-02-28T06:18:01Z

apiVersion: policy.karmada.io/v1alpha1
kind: PropagationPolicy
metadata:
  name: nginx
spec:
  resourceSelectors:
    - apiVersion: apps/v1
      kind: Deployment
      name: nginx
  placement:
    clusterAffinity:
      labelSelector:
        matchExpressions:
        - key: stage
          operator: In
          values:
          - dev
          - prod
    replicaScheduling:
      replicaSchedulingType: Divided
      replicaDivisionPreference: Weighted
      weightPreference:
        dynamicWeight: AvailableReplicas
    spreadConstraints:
    - maxGroups: 2
      minGroups: 2
      spreadByLabel: stage

chaunceyjiang · 2023-03-03T06:14:32Z

/cc @RainbowMango @Garrybest @jwcesign

Please take a look.

XiShanYongYe-Chang · 2023-03-03T08:36:38Z

/assign

RainbowMango · 2023-03-13T12:35:53Z

Ping @XiShanYongYe-Chang

RainbowMango · 2023-03-14T03:34:41Z

pkg/scheduler/core/spreadconstraint/select_clusters.go

 	if _, exist := spreadConstraintMap[policyv1alpha1.SpreadByFieldRegion]; exist {
 		return selectBestClustersByRegion(spreadConstraintMap, groupClustersInfo)
 	} else if _, exist := spreadConstraintMap[policyv1alpha1.SpreadByFieldCluster]; exist {
 		return selectBestClustersByCluster(spreadConstraintMap[policyv1alpha1.SpreadByFieldCluster], groupClustersInfo, needReplicas)


I have a question about the previous logic.

Here bypass the logic according to spreadByField, what if we have two spread constraints? E.g:

spreadConstraints: - spreadByField: region maxGroups: 2 minGroups: 2 - spreadByField: cluster maxGroups: 2 minGroups: 2

This example has two constraints, one is spreadByField: region the other one is spreadByField: cluster. For this case, we are going to select the cluster only by region?

Oh, I get it.

karmada/pkg/scheduler/core/spreadconstraint/select_clusters_by_region.go

Line 21 in 50cb376

regions := selectRegions(groupClustersInfo.Regions, spreadConstraintMap[policyv1alpha1.SpreadByFieldRegion], spreadConstraintMap[policyv1alpha1.SpreadByFieldCluster])

Here checked the cluster group.

Sorry, I didn't understand the purpose of your question. 'spreadByField' and 'spreadByLabel' cannot co-exist.

XiShanYongYe-Chang · 2023-03-14T04:01:42Z

pkg/scheduler/core/spreadconstraint/select_clusters_by_labels.go

+			if selector.Matches(labels.Set(clusterInfos[j].Cluster.GetLabels())) {
+				selectedClusters = append(selectedClusters, clusterInfos[j])
+			}


I have a question if two clusters have the same label(same key and value), should they be considered one group or two groups?

But two clusters that have the same label(same key but different value) should be considered two groups, right?

XiShanYongYe-Chang · 2023-03-14T04:06:05Z

pkg/scheduler/core/spreadconstraint/select_clusters.go

+	} else if hasSpreadByLabel {
+		return selectBestClustersByClusterLabels(spreadConstraints, groupClustersInfo, needReplicas)


Grouping by label and field cannot take effect at the same time?

Such as the below config:

... spreadConstraints: - maxGroups: 2 minGroups: 2 spreadByLabel: stage - maxGroups: 2 minGroups: 2 spreadByField: cluster ...

karmada/pkg/util/validation/validation.go

Lines 100 to 103 in 35f3e3f

// SpreadByField and SpreadByLabel should not co-exist

if len(constraint.SpreadByField) > 0 && len(constraint.SpreadByLabel) > 0 {

allErrs = append(allErrs, field.Invalid(fldPath.Index(index), constraint, "spreadByLabel should not co-exist with spreadByField"))

}

The only thing that's forbidden is the same constraint, they're in two constraints in the example.

Thank you for the reminder. Currently, spreadByLabel is being ignored in this situation. Do you have any good suggestions?

I'm sorry, I don't have a good idea yet. I feel that there are many factors to consider when implementing, and it may be complicated to implement.

RainbowMango · 2023-08-25T07:45:32Z

I'm trying to figure out which PR/Issue should be included in the coming v1.7 release which is planned at the end of this month.
I guess we don't have enough time for this feature, so I'm moving this to v1.8.

I'll try to get back on the scheduler in v1.8, let's see if we can move this forward then.

chenzhiguo · 2024-07-26T05:05:17Z

I was about to open the same issue and found that it already exists. Is there any progress on this? Are any additional PRs needed?

karmada-bot added the kind/feature Categorizes issue or PR as related to a new feature. label Feb 23, 2023

karmada-bot requested review from Garrybest and XiShanYongYe-Chang February 23, 2023 09:29

karmada-bot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Feb 23, 2023

chaunceyjiang force-pushed the spreadByLabel branch from 52f2d5f to c722d40 Compare February 23, 2023 10:33

RainbowMango added this to the v1.6 milestone Feb 23, 2023

feat: Support for SpreadByLabel constraints

a23ffdf

Signed-off-by: chaunceyjiang <[email protected]>

chaunceyjiang force-pushed the spreadByLabel branch from c722d40 to a23ffdf Compare February 27, 2023 06:15

karmada-bot requested review from RainbowMango and jwcesign March 3, 2023 06:14

karmada-bot assigned XiShanYongYe-Chang Mar 3, 2023

chaunceyjiang mentioned this pull request Mar 13, 2023

spreadByLabel报错 #3187

Open

RainbowMango reviewed Mar 14, 2023

View reviewed changes

XiShanYongYe-Chang reviewed Mar 14, 2023

View reviewed changes

RainbowMango modified the milestones: v1.6, v1.7 May 29, 2023

RainbowMango modified the milestones: v1.7, v1.8 Aug 25, 2023

RainbowMango modified the milestones: v1.8, v1.9 Jan 9, 2024

RainbowMango modified the milestones: v1.9, v1.10 Feb 29, 2024

RainbowMango modified the milestones: v1.10, v1.11 May 29, 2024

RainbowMango modified the milestones: v1.11, v1.12 Aug 28, 2024

RainbowMango modified the milestones: v1.12, v1.13 Nov 28, 2024

RainbowMango modified the milestones: v1.13, v1.14 Apr 28, 2025

RainbowMango modified the milestones: v1.14, v1.15 Jul 8, 2025

RainbowMango added the priority/awaiting-more-evidence Lowest priority. Possibly useful, but not yet enough support to actually get it done. label Jul 8, 2025

RainbowMango modified the milestones: v1.15, v1.17 Oct 15, 2025

		} else if hasSpreadByLabel {
		return selectBestClustersByClusterLabels(spreadConstraints, groupClustersInfo, needReplicas)

	// SpreadByField and SpreadByLabel should not co-exist
	if len(constraint.SpreadByField) > 0 && len(constraint.SpreadByLabel) > 0 {
	allErrs = append(allErrs, field.Invalid(fldPath.Index(index), constraint, "spreadByLabel should not co-exist with spreadByField"))
	}

feat: the scheduler support SpreadByLabel constraints #3203

Are you sure you want to change the base?

feat: the scheduler support SpreadByLabel constraints #3203

Uh oh!

Conversation

chaunceyjiang commented Feb 23, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

karmada-bot commented Feb 23, 2023

Uh oh!

chaunceyjiang commented Feb 28, 2023

Uh oh!

chaunceyjiang commented Mar 3, 2023

Uh oh!

XiShanYongYe-Chang commented Mar 3, 2023

Uh oh!

RainbowMango commented Mar 13, 2023

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

chaunceyjiang Mar 14, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

RainbowMango commented Aug 25, 2023

Uh oh!

chenzhiguo commented Jul 26, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

chaunceyjiang commented Feb 23, 2023 •

edited

Loading

chaunceyjiang Mar 14, 2023 •

edited

Loading