Design Suggestions for Scaler Based on Multiple Metrics from the Same Endpoint

[SpiritZhou]

We are designing a new in-house scaler that may be upstreamed to the community later, but we are facing a design choice. The pod target, which will handle different types of work items, will scale based on different metric values that come from the same endpoint. We are undecided on whether to combine these metrics into one trigger, which would require an uncommon scaler implementation, or to separate them into different triggers by type, which may lead to redundant configuration for customers.

For example, there are three work item types named workitemA, workitemB, and workitemC. A pod or worker will concurrently handle varying numbers of each work item type. An endpoint "https://workitemstatus.com" will return metric values in the format: "{WorkItemAPendingCount: i, WorkItemBPendingCount: j, WorkItemCPendingCount: k}". Now we have two scaler approach:

Option 1: scaler only handle one work item type and we adding multiple triggers. but customer need to write redundant config:

triggers:
   - type: inhouse-scaler
      metadata:
           endpoint: "https://workitemstatus.com"
           workItemType: workitemA
           concurrentWorkItemCount: 2
   - type: inhouse-scaler
      metadata:
           endpoint: "https://workitemstatus.com"
           workItemType: workitemB
           concurrentWorkItemCount: 4
   - type: inhouse-scaler
      metadata:
           endpoint: "https://workitemstatus.com"
           workItemType: workitemC
           concurrentWorkItemCount: 6

Option 2: scaler handle all three types together. The config is much simple but the implementation is uncommon.

triggers:
- type: inhouse-scaler
    metadata:
       endpoint:  "https://workitemstatus.com"
       concurrentWorkItemACount: 2
       concurrentWorkItemBCount: 4
       concurrentWorkItemCCount: 6

In normal scaler, one trigger just focus on setting one type of metric average target and returning metric value to HPA. In order to handle multiple metrics, we should always set the metric average target to 1 in GetMetricSpecForScaling and calculate the number of required workers (i.e. pods) in GetMetricsAndActivity. The logic to determine the number of workers (i.e. scaling logic) in GetMetricsAndActivity:

func (s *inhouse-scaler) GetMetricSpecForScaling(context.Context) []v2.MetricSpec {
	externalMetric := &v2.ExternalMetricSource{
		Metric: v2.MetricIdentifier{
			Name: name,
		},
		Target: 1,
	}
	metricSpec := v2.MetricSpec{External: externalMetric, Type: kafkaMetricType}
	return []v2.MetricSpec{metricSpec}
}

func (s *inhouse-scaler) GetMetricsAndActivity(ctx context.Context, metricName string) ([]external_metrics.ExternalMetricValue, bool, error) {
        pendingCountA, pendingCountB, pendingCountC := s.getMetricFromEndPoint(ctx)
	
        replicaA := pendingCountA / s.concurrentWorkItemACount
        replicaB := pendingCountB / s.concurrentWorkItemBCount
        replicaC := pendingCountC / s.concurrentWorkItemCCount

       finalPodReplica := max(replicaA, max(replicaB, replicaC))

	metric := GenerateMetricInMili(metricName, float64(finalPodReplica ))

	return []external_metrics.ExternalMetricValue{metric}, true, nil
}

Which one is the better practice?

[rickbrouwer]

Option 1, I think this is the most logical because the other scalers work like this.

You could perhaps make it so that you can specify multiple workItemTypes (comma separated) if that is useful if the concurrentWorkItemCount for the workItemTypes is the same.