Design Suggestions for Scaler Based on Multiple Metrics from the Same Endpoint

[SpiritZhou]

We are designing a new in-house scaler that may be upstreamed to the community later, but we are facing a design choice. The pod target, which will handle different types of work items, will scale based on different metric values that come from the same endpoint. We are undecided on whether to combine these metrics into one trigger, which would require an uncommon scaler implementation, or to separate them into different triggers by type, which may lead to redundant configuration for customers.

For example, there are three work item types named workitemA, workitemB, and workitemC. A pod or worker will concurrently handle varying numbers of each work item type. An endpoint "https://workitemstatus.com" will return metric values in the format: "{WorkItemAPendingCount: i, WorkItemBPendingCount: j, WorkItemCPendingCount: k}". Now we have two scaler approach:

Option 1: scaler only handle one work item type and we adding multiple triggers. but customer need to write redundant config:

triggers:
   - type: inhouse-scaler
      metadata:
           endpoint: "https://workitemstatus.com"
           workItemType: workitemA
           concurrentWorkItemCount: 2
   - type: inhouse-scaler
      metadata:
           endpoint: "https://workitemstatus.com"
           workItemType: workitemB
           concurrentWorkItemCount: 4
   - type: inhouse-scaler
      metadata:
           endpoint: "https://workitemstatus.com"
           workItemType: workitemC
           concurrentWorkItemCount: 6

Option 2: scaler handle all three types together. The config is much simple but the implementation is uncommon.

triggers:
- type: inhouse-scaler
    metadata:
       endpoint:  "https://workitemstatus.com"
       concurrentWorkItemACount: 2
       concurrentWorkItemBCount: 4
       concurrentWorkItemCCount: 6

In normal scaler, one trigger just focus on setting one type of metric average target and returning metric value to HPA. In order to handle multiple metrics, we should always set the metric average target to 1 in GetMetricSpecForScaling and calculate the number of required workers (i.e. pods) in GetMetricsAndActivity. The logic to determine the number of workers (i.e. scaling logic) in GetMetricsAndActivity:

func (s *inhouse-scaler) GetMetricSpecForScaling(context.Context) []v2.MetricSpec {
	externalMetric := &v2.ExternalMetricSource{
		Metric: v2.MetricIdentifier{
			Name: name,
		},
		Target: 1,
	}
	metricSpec := v2.MetricSpec{External: externalMetric, Type: kafkaMetricType}
	return []v2.MetricSpec{metricSpec}
}

func (s *inhouse-scaler) GetMetricsAndActivity(ctx context.Context, metricName string) ([]external_metrics.ExternalMetricValue, bool, error) {
        pendingCountA, pendingCountB, pendingCountC := s.getMetricFromEndPoint(ctx)
	
        replicaA := pendingCountA / s.concurrentWorkItemACount
        replicaB := pendingCountB / s.concurrentWorkItemBCount
        replicaC := pendingCountC / s.concurrentWorkItemCCount

       finalPodReplica := max(replicaA, max(replicaB, replicaC))

	metric := GenerateMetricInMili(metricName, float64(finalPodReplica ))

	return []external_metrics.ExternalMetricValue{metric}, true, nil
}

Which one is the better practice?

[rickbrouwer]

Option 1, I think this is the most logical because the other scalers work like this.

You could perhaps make it so that you can specify multiple workItemTypes (comma separated) if that is useful if the concurrentWorkItemCount for the workItemTypes is the same.

[zroubalik]

Apologies for the delay. I tend to agree with Rick here. I understand the concerns with the configuration but having a separate triggers will allow you use Scaling Modifiers and similar techniques.
But on the other hand, if you wish to keep this scaler in-house (which is completely fine) then you could also use the other option, if you think it would suit your need better.

Not sure if this helps 🤷‍♂️ 😄

[JorTurFer]

I think that this strongly depends on what you're going to do with this, if you plan to donate it to KEDA, I think that the 1st option is the better one, as the approach of a metric per scaler is the option used by KEDA.
Said this, as k8s supports multiple metrics at once, the second option is perfectly fine if you plan to do something internal.
In the middle, you can write a custom add-on based on external scaler, external scaler supports multiple metric definitions at once, as well as multiple metric returns at once, so you can use KEDA with the second option and you can make the new scaler public via add-on if finally you decide to do it