Interface to easily install and manage external scalers

[zroubalik]

Proposal

When users want to use external scalers in KEDA, they usually need to provide connection information in the trigger metadata:

- type: external
  metadata:
    scalerAddress: external-scaler-service:8080
    caCert : /path/to/tls/ca.pem
    tlsCertFile: /path/to/tls/cert.pem # Deprecated. https://github.com/kedacore/keda/issues/4549
    tlsClientCert: /path/to/tls/cert.pem
    tlsClientKey: /path/to/tls/key.pem
    unsafeSsl: false
    // ACTUAL TRIGGER CONFIG STARTS HERE
    activationThreshold: 10
    threshold: 20
    fooConfig: true
    barConfig: false
    ...

Let's provide an interface (CRD?), where administrators can install external scalers to KEDA on a cluster level including authenticationRef. This external scalers will define a name (that is unique and don't collide with built-in scalers).
Users can then just easily reference this external scaler as a built in one and provide only needed config (and don't need to specify external scaler connection settings).

Example

For example, let's see an example of external trigger my-awesome-scaler:

1. Let's define this CRD once (created in keda namespace)

apiVersion: scalers.keda.sh/v1alpha1
kind: ExternalScalerRegistration
metadata:
  name: my-awesome-scaler
spec:
 metadata:
    scalerAddress: external-scaler-service:8080
    caCert: /path/to/tls/ca.pem
    tlsCertFile: /path/to/tls/cert.pem  # Note: This field is deprecated. 
    tlsClientCert: /path/to/tls/cert.pem
    tlsClientKey: /path/to/tls/key.pem
    unsafeSsl: false
  authenticationRef:
    my-ta

2. users then can reference this trigger like this in their SO/SJ:

- type: my-awesome-scaler
  metadata:
    activationThreshold: 10
    threshold: 20
    fooConfig: true
    barConfig: false
    ...

Things to consider

• Should we allow arbitrary names for scalers (but check that they don't collide with built-in)? - it would be nice to not pust any restrictions. But what would happen if a user creates and registers external scaler foo in KEDA v 2.X but KEDA v 2.X+1 introduces a new built in scaler called foo?
• AuthenticationRef should be included in the scaler registration

Use-Case

Simplified management of external scalers

Is this a feature you are interested in implementing yourself?

Maybe

[tomkerkhove]

I think we can brush up the proposed CRD structure but I like this idea.

We should also consider allowing it to be scoped to the whole cluster or just a single namespace in case multiple teams want to run different scalers/configurations.

[JatinSanghvi]

If two scale targets need to use the same external scaler but with different auth information, it should still require having two external scaler registrations. This seems to be useful only when the trigger metadata will be common between two ScaledObjects. Let me know if I am mistaken. Also, shouldn't it be possible to have this supported for built-in scalers as well? Any reason to restrict the creation of CRD to keda namespace only?

[tomkerkhove]

If two scale targets need to use the same external scaler but with different auth information, it should still require having two external scaler registrations.

Correct, I believe this is also what Zbynek proposes in specs above

Also, shouldn't it be possible to have this supported for built-in scalers as well? Any reason to restrict the creation of CRD to keda namespace only?

I'm not sure if I'm following on this, where did you see that limitation? It should not be the case.

If you are referring to "We should also consider allowing it to be scoped to the whole cluster or just a single namespace in case multiple teams want to run different scalers/configurations." then my point was more that people should be able to create their scalers in their namespace and use it there, not only in keda namespace

[JatinSanghvi]

I'm not sure if I'm following on this, where did you see that limitation? It should not be the case.

Hi, I had most of my misconceptions cleared out by conversing with my teammate. From the text Let's define this CRD once (created in keda namespace), it seems as if we will ignore CRD out of keda namespace, which should be okay I guess.

[SpiritZhou]

I'm a little confused about this. Since other built-in scalers also require connection information in the trigger, it appears that the concept of built-in scalers, which need connection information, is different from that of registered scalers, which don't need it. Would it be more accurate to describe it as a TriggerAuthentication interface or common setting interface rather than a scaler interface?

[zroubalik]

I'm not sure if I'm following on this, where did you see that limitation? It should not be the case.

Hi, I had most of my misconceptions cleared out by conversing with my teammate. From the text Let's define this CRD once (created in keda namespace), it seems as if we will ignore CRD out of keda namespace, which should be okay I guess.

It is just an proposal. We might want to think about a CR defined on the cluster level (same as ClusterTriggerAuth) and maybe a CR defined on the Namespace level.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.