Enhance Security and Self-Service by Allowing Service Account Specification in Target Namespace for Workload Identity

[abelhoula]

Proposal

Allow the TriggerAuthentication resource in KEDA to specify a service account from the target namespace for workload identity, enhancing security and enabling a self-service model for managing scaling resources.

Use-Case

In multi-tenant Kubernetes environments, teams often manage their own namespaces and the resources within them, including service accounts. The current approach, where KEDA uses a service account from the keda-operator namespace for scaling operations, presents a challenge for these teams. It limits their ability to apply namespace-specific security policies or manage the lifecycle of these accounts independently. By allowing the specification of a service account in the target namespace, teams would gain the ability to manage their scaling operations more securely and autonomously.

Is this a feature you are interested in implementing yourself?

Yes

Anything else?

This proposal aims to strike a balance between security, flexibility, and operational efficiency in managing scaling operations with KEDA. I believe that implementing this feature will benefit many users operating in environments with strict security policies and those who advocate for a more self-service oriented approach to resource management.

[zroubalik]

I like this a lot, actually it is the area we want to improve for a long time. I am looking forward to see your take on this.

[JorTurFer]

Honestly, I like the idea but AFAIK, the only way to address it at this moment (due to how kubernetes works) it granting permissions to KEDA for requesting service account tokens (because the federations work based on the service account token), which is something I'm strongly against as it allows KEDA to bypass any RBAC in the cluster.

I'm happy with this feature if we find a way to get that token without requiring the permissions for requesting service account tokens by code.

In case of not being able to handle the token generation externally, I'd bet for pushing the multi tenat support rather than requesting tokens by code

[Richard87]

Hi, when implementing Keda on our platform we assumed the operator would create tokens as needed when using Workload Identity.

We would still love this feature, could the security concerns be mitigated by using RBAC for controlling who's service account Keda could access?

I think this would not change Kedas security posture more or less than just permitting Kedas own serviceaccount access to all the teams resourses (except for maybe miss-configuration).

But it would be safer for teams where only their app are allowed to use the identity.

Ref. https://kubernetes.io/docs/concepts/security/rbac-good-practices/#token-request

[JorTurFer]

we assumed the operator would create tokens as needed when using Workload Identity.

Yeah, the operator handles token generation for SDKs, but not for k8s. K8s tokens must be handled or provided by users atm. I don't have enough knowledge about this specific topic, so maybe you can help me with this question: Is there any way in k8s to limit the service account token which can be requested? I mean, I know that using RBAC I can create a role like this:

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  namespace: my-namespace
  name: keda-role-token-requester
rules:
- apiGroups: [""]
  resources: ["serviceaccounts/token"]
  verbs: ["create"]

But this allows to request a token in behalf on any service account within my-namespace. This can be safe enough for users with a service account by namespace, but other use cases deploy multiple service accounts into the same namespaces, and this role can potentially allows to KEDA to assume any of them, which can be risky.

I like the idea behind this change, but I want to be 100% sure about the security implications and caveats of it and we have to document them pretty well if we continue and add this feature. (I'd not deploy this role as default either to make users aware of it)

[Richard87]

We could leverage the rules resourceNames in the cluster role:

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: keda-role-token-requester
rules:
- apiGroups: [""]
  resources: ["serviceaccounts/token"]
  verbs: ["create"]
  resourceNames: ["keda-app"]

https://kubernetes.io/docs/reference/access-authn-authz/rbac/#referring-to-resources

I Just tested this:

apiVersion: v1
kind: ServiceAccount
metadata:
  name: keda-test
  namespace: default
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: keda-app
  namespace: default
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: keda-role-token-requester
rules:
  - apiGroups: [""]
    resources: ["serviceaccounts/token"]
    verbs: ["create"]
    resourceNames: ["keda-app"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: keda-role-token-requester
subjects:
  - kind: ServiceAccount
    name: keda-test
    namespace: default
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: keda-role-token-requester

Then to test it, I created a token for the keda-test service account k create token keda-test, and used that to create tokens for keda-app

Works great for keda-app:

POST https://API-SERVER:443/api/v1/namespaces/default/serviceaccounts/keda-app/token
Authorization: Bearer <CREATED TOKEN>
Accept: application/json
Content-Type: application/json

{
  "kind": "TokenRequest",
  "apiVersion": "authentication.k8s.io/v1",
  "spec": {
    "audiences": ["api://AzureADTokenExchange"],
    "expirationSeconds": 3600
  }
}

Response

{
  "kind": "TokenRequest",
  "apiVersion": "authentication.k8s.io/v1",
  "metadata": {
    "name": "keda-app",
    "namespace": "default",
    "creationTimestamp": "2024-05-30T07:56:45Z",
    "managedFields": [
      {
        "manager": "IntelliJ HTTP Client",
        "operation": "Update",
        "apiVersion": "authentication.k8s.io/v1",
        "time": "2024-05-30T07:56:45Z",
        "fieldsType": "FieldsV1",
        "fieldsV1": {
          "f:spec": {
            "f:audiences": {},
            "f:expirationSeconds": {}
          }
        },
        "subresource": "token"
      }
    ]
  },
  "spec": {
    "audiences": [
      "api://AzureADTokenExchange"
    ],
    "expirationSeconds": 36001,
    "boundObjectRef": null
  },
  "status": {
    "token": "aaaa.bbbb.cccc",
    "expirationTimestamp": "2024-05-30T17:56:46Z"
  }
}

Forbidden for default sa:

POST https://API-SERVER:443/api/v1/namespaces/default/serviceaccounts/default/token
Authorization: Bearer <CREATED TOKEN>
Accept: application/json
Content-Type: application/json

{
  "kind": "TokenRequest",
  "apiVersion": "authentication.k8s.io/v1",
  "spec": {
    "audiences": ["api://AzureADTokenExchange"],
    "expirationSeconds": 3600
  }
}

Response

{
  "kind": "Status",
  "apiVersion": "v1",
  "metadata": {},
  "status": "Failure",
  "message": "serviceaccounts \"default\" is forbidden: User \"system:serviceaccount:default:keda-test\" cannot create resource \"serviceaccounts/token\" in API group \"\" in the namespace \"default\"",
  "reason": "Forbidden",
  "details": {
    "name": "default",
    "kind": "serviceaccounts"
  },
  "code": 403
}

[JorTurFer]

I ❤️ the idea! I didn't know that RBAC supports resourceNames and it's exactly what I meant.
We can keep this disabled as default and clearly document the way to enable it (the extra RBAC required) and the possible risks and just bring the power to cluster admins to configure if they will allow this and in which terms.

@tomkerkhove @zroubalik WDYT?

BTW: Sorry for the slow response :(

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.