Fallback never triggers when GetMetricSpec fails and unreachable external scaler bypasses the fallback failure counter entirely

[cimaqsous-everai]

Report

When an external (gRPC) scaler becomes unreachable at runtime, fallback configured on the ScaledObject never activates, no matter how long the outage lasts. The failure counter (Health in the ScaledObject status) stays at 0 for the whole outage.

The reason is structural: for every poll cycle, KEDA first calls GetMetricSpecForScalingForScaler and only then fetches metrics for each returned spec (GetScaledObjectMetrics in pkg/scaling/scale_handler.go). The fallback machinery (fallback.GetMetricsWithFallback → IncrementFailure) is only invoked on the metrics-fetch path. When the scaler is unreachable, GetMetricSpec fails first, the error is only logged and emitted as a KEDAScalerFailed event, metricSpecs comes back empty, the for _, spec := range metricSpecs loop never runs which means the fallback counter is never incremented and fallback can never fire.

In other words: fallback protects against a reachable scaler failing to produce metric values, but not against the scaler being unreachable which is arguably the primary scenario fallback exists for.

Expected Behavior

After failureThreshold consecutive poll cycles in which the scaler cannot be reached (including GetMetricSpec failures), fallback activates and the workload is scaled to fallback.replicas.

Actual Behavior

Scaling freezes at the last computed replica count indefinitely. ScaledObject shows Ready: False with KEDAScalerFailed: got empty metric spec, the fallback failure counter stays at 0, and Fallback status stays False for the entire outage.
Real-world impact (how we found this): our external scaler's Service endpoints went empty for ~21 hours. Every ScaledObject pointing at it had fallback: {failureThreshold: 5, replicas: } configured,
yet none activated.
Autoscaling was frozen at the pre-outage replica count while queue backlog built up (p90 latency ~8s → ~2min). Across the incident window we captured hundreds of error getting metric spec for the scaler log lines and zero fallback activity or GetMetrics calls.

Steps to Reproduce the Problem

1. Deploy an external gRPC scaler and a ScaledObject with a single external trigger and fallback, e.g.:

  fallback:
    failureThreshold: 3                                                                                                                                                                                              
    replicas: 5                                                                                                                                                                                                    
  triggers:
    - type: external
      metadata:
        scalerAddress: my-external-scaler.default.svc:9090      

2. Let it run until healthy (HPA created, metrics flowing).
3. Make the scaler unreachable without touching KEDA, e.g. scale the external scaler deployment to 0 (or break its Service endpoints).
4. Wait well past failureThreshold × pollingInterval.
5. Observe: KEDAScalerFailed: got empty metric spec events, but kubectl get scaledobject shows FALLBACK: False forever, and the target workload is never scaled to fallback.replicas.

Logs from KEDA operator

ERROR scale_handler error getting metric spec for the scaler {"scaler": "externalScaler", "error": "rpc error: code = Unavailable desc = connection error: desc = \"transport: Error while dialing: dial tcp <scaler-vip>:9090: connect: connection refused\""} (repeated every poll cycle for the entire outage; no fallback log lines ever appear)

KEDA Version

2.20.1

Kubernetes Version

1.34

Platform

Google Cloud

Scaler Details

Custom external (gRPC)

Would you be open to contributing a fix?

Maybe

Anything else?

Count GetMetricSpec failures toward the same per-metric fallback health counter (needs a metric name when no spec is available and could use the last known spec from the HPA, which still exists in this scenario).