Admission Webhook blocks ScaledObject with fallback (using AverageValue) 2.17

[pinkfloydx33]

Report

We rolled out 2.17.0 in our dev stack yesterday. We are now unable to deploy ScaledObject resources with a fallback. We receive the following error:

admission webhook "vscaledobject.kb.io" denied the request: at least one trigger (that is not cpu or memory) has to have the AverageValue type for the fallback to be enabled

These particular scalers are using the type=redis. The HPA's generated by the scalers (at least, pre 2.17) contain an AverageValue metric so the error does not make sense to me

Expected Behavior

Rollout should succeed when fallback is specified.

Actual Behavior

Unable to roll out ScaledObject with the following error:

admission webhook "vscaledobject.kb.io" denied the request: at least one trigger (that is not cpu or memory) has to have the AverageValue type for the fallback to be enabled

Steps to Reproduce the Problem

1. Keda 2.17.0
2. Apply the following manifests

apiVersion: v1
kind: Secret
metadata:
  name: my-redis-secret
  namespace: default
type: Opaque
data:
  redis_username: YWRtaW4=
  redis_password: YWRtaW4=
---
apiVersion: keda.sh/v1alpha1
kind: TriggerAuthentication
metadata:
  name: my-redis-auth
  namespace: default
spec:
  secretTargetRef:
  - parameter: username
    name: my-redis-secret
    key: redis_username
  - parameter: password
    name: my-redis-secret
    key: redis_password
---
apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: my-queue-trigger
  namespace: default
spec:
  cooldownPeriod: 300
  fallback:
    behavior: static
    failureThreshold: 3
    replicas: 1
  maxReplicaCount: 1
  minReplicaCount: 0
  pollingInterval: 30
  scaleTargetRef:
    name: myproject
  triggers:
    - authenticationRef:
        name: my-redis-auth
      metadata:
        databaseIndex: '0'
        enableTLS: 'true'
        listLength: '5'
        listName: q:my:in
        port: '6380'
      type: redis
    - authenticationRef:
        name: my-redis-auth
      metadata:
        databaseIndex: '0'
        enableTLS: 'true'
        listLength: '5'
        listName: q:my:wait
        port: '6380'
      type: redis

3. Fail

Logs from KEDA operator

Webhook logs (there were not logs in the operator)

2025-04-08T12:44:45Z    INFO    triggerauthentication-validation-webhook        validating triggerauthentication creation for {
  "kind": "TriggerAuthentication",
  "apiVersion": "keda.sh/v1alpha1",
  "metadata": {
    "name": "my-redis-auth",
    "namespace": "default",
    "uid": "1aa13de3-45a1-47c5-8df3-f707bc66961d",
    "generation": 1,
    "creationTimestamp": "2025-04-08T12:44:45Z",
    "managedFields": [
      {
        "manager": "kubectl",
        "operation": "Apply",
        "apiVersion": "keda.sh/v1alpha1",
        "time": "2025-04-08T12:44:45Z",
        "fieldsType": "FieldsV1",
        "fieldsV1": {
          "f:spec": {
            "f:secretTargetRef": {}
          }
        }
      }
    ]
  },
  "spec": {
    "secretTargetRef": [
      {
        "parameter": "username",
        "name": "my-redis-secret",
        "key": "redis_username"
      },
      {
        "parameter": "password",
        "name": "my-redis-secret",
        "key": "redis_password"
      }
    ]
  },
  "status": {}
}
2025-04-08T12:44:45Z    ERROR   scaledobject-validation-webhook validation error        {"name": "my-queue-trigger", "error": "at least one trigger (that is not cpu or memory) has to have the `AverageValue` type for the fallback to be enabled"}
github.com/kedacore/keda/v2/apis/keda/v1alpha1.verifyFallback
        /workspace/apis/keda/v1alpha1/scaledobject_webhook.go:196
github.com/kedacore/keda/v2/apis/keda/v1alpha1.validateWorkload
        /workspace/apis/keda/v1alpha1/scaledobject_webhook.go:161
github.com/kedacore/keda/v2/apis/keda/v1alpha1.(*ScaledObject).ValidateCreate
        /workspace/apis/keda/v1alpha1/scaledobject_webhook.go:118
github.com/kedacore/keda/v2/apis/keda/v1alpha1.ScaledObjectCustomValidator.ValidateCreate
        /workspace/apis/keda/v1alpha1/scaledobject_webhook.go:90
sigs.k8s.io/controller-runtime/pkg/webhook/admission.(*validatorForType).Handle
        /workspace/vendor/sigs.k8s.io/controller-runtime/pkg/webhook/admission/validator_custom.go:91
sigs.k8s.io/controller-runtime/pkg/webhook/admission.(*Webhook).Handle
        /workspace/vendor/sigs.k8s.io/controller-runtime/pkg/webhook/admission/webhook.go:181
sigs.k8s.io/controller-runtime/pkg/webhook/admission.(*Webhook).ServeHTTP
        /workspace/vendor/sigs.k8s.io/controller-runtime/pkg/webhook/admission/http.go:119
sigs.k8s.io/controller-runtime/pkg/webhook/internal/metrics.InstrumentedHook.InstrumentHandlerInFlight.func1
        /workspace/vendor/github.com/prometheus/client_golang/prometheus/promhttp/instrument_server.go:60
net/http.HandlerFunc.ServeHTTP
        /usr/local/go/src/net/http/server.go:2220
github.com/prometheus/client_golang/prometheus/promhttp.InstrumentHandlerCounter.func1
        /workspace/vendor/github.com/prometheus/client_golang/prometheus/promhttp/instrument_server.go:147
net/http.HandlerFunc.ServeHTTP
        /usr/local/go/src/net/http/server.go:2220
github.com/prometheus/client_golang/prometheus/promhttp.InstrumentHandlerDuration.func2
        /workspace/vendor/github.com/prometheus/client_golang/prometheus/promhttp/instrument_server.go:109
net/http.HandlerFunc.ServeHTTP
        /usr/local/go/src/net/http/server.go:2220
net/http.(*ServeMux).ServeHTTP
        /usr/local/go/src/net/http/server.go:2747
net/http.serverHandler.ServeHTTP
        /usr/local/go/src/net/http/server.go:3210
net/http.(*conn).serve
        /usr/local/go/src/net/http/server.go:2092

KEDA Version

2.17.0

Kubernetes Version

1.30

Platform

Microsoft Azure

Scaler Details

Redis

Anything else?

This is an example of the HPA generated on the previous version. Note that the metric has a target of AverageValue

kind: HorizontalPodAutoscaler
apiVersion: autoscaling/v2
metadata:
  name: keda-hpa-my-queue-trigger
  labels:
    scaledobject.keda.sh/name: my-queue-trigger
  ownerReferences:
    - apiVersion: keda.sh/v1alpha1
      kind: ScaledObject
      name: my-queue-trigger
      uid: 41f2d8b7-d614-49a9-a303-e58e4f58996c
      controller: true
      blockOwnerDeletion: true
spec:
  scaleTargetRef:
    kind: Deployment
    name: myproject
    apiVersion: apps/v1
  minReplicas: 1
  maxReplicas: 1
  metrics:
    - type: External
      external:
        metric:
          name: s0-redis-q-my-in
          selector:
            matchLabels:
              scaledobject.keda.sh/name: my-queue-trigger
        target:
          type: AverageValue
          averageValue: '5'
    - type: External
      external:
        metric:
          name: s1-redis-q-my-wait
          selector:
            matchLabels:
              scaledobject.keda.sh/name: my-queue-trigger
        target:
          type: AverageValue
          averageValue: '10'

[wozniakjan]

there was a good amount of changes in the admission webhook logic related to fallbacks, a change that slipped through the tests must have been merged accidentally, this will be fixed in a 2.17.1.

Meanwhile, you can either disable webhook validation or roll back to 2.16.1, apologies for the inconvenience.

[rickbrouwer]

Other workaround for the time being is to add metricType: AverageValue.

So:

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: my-queue-trigger
  namespace: default
spec:
  cooldownPeriod: 300
  fallback:
    behavior: static
    failureThreshold: 3
    replicas: 1
  maxReplicaCount: 1
  minReplicaCount: 0
  pollingInterval: 30
  scaleTargetRef:
    name: myproject
  triggers:
    - authenticationRef:
        name: my-redis-auth
      metadata:
        databaseIndex: '0'
        enableTLS: 'true'
        listLength: '5'
        listName: q:my:in
        port: '6380'
      type: redis
      metricType: AverageValue
    - authenticationRef:
        name: my-redis-auth
      metadata:
        databaseIndex: '0'
        enableTLS: 'true'
        listLength: '5'
        listName: q:my:wait
        port: '6380'
      type: redis
      metricType: AverageValue