[SPARK-59519] Add opt-in Kueue RBAC rules to Helm chart

### What changes were proposed in this pull request?

This PR adds an opt-in Helm value `operatorRbac.kueue.enabled` (default `false`). When enabled, the operator is granted the RBAC rules that [Kueue](https://kueue.sigs.k8s.io/docs/tasks/dev/integrate_a_custom_job/) requires from an external framework integration:

| Group / Resource | Verbs | Granted via |
|---|---|---|
| `kueue.x-k8s.io` / `workloads` | get, list, watch, create, update, patch, delete | ClusterRole and Role |
| `kueue.x-k8s.io` / `workloads/status` | get, update, patch | ClusterRole and Role |
| `kueue.x-k8s.io` / `workloads/finalizers` | update | ClusterRole and Role |
| `kueue.x-k8s.io` / `resourceflavors`, `workloadpriorityclasses` | get, list, watch | ClusterRole only |
| `scheduling.k8s.io` / `priorityclasses` | get, list, watch | ClusterRole only |

The namespaced `workloads` rules live in the shared `operatorRbacRules` block under `{{- if }}`, like the existing `leases` rule. The cluster-scoped resources go into a new `operatorClusterRbacRules` wrapper used only by the ClusterRole, since a namespaced Role cannot grant them. `events.k8s.io/events` from the Kueue doc is intentionally left out: the operator only emits core `events`, which are already granted.

Also included: `helm test` assertions for the new grants, a `helm-tests` CI group `kueue` that installs Kueue v0.19.4 and runs them, a check that `workloads` create is denied with the default values, and a `docs/operations.md` entry.

### Why are the changes needed?

This is the deployment-side preparation for the Kueue integration (SPARK-59486, SPARK-59490, SPARK-59503). Keeping the grant opt-in avoids widening the operator ClusterRole for users who do not run Kueue. The operator runtime changes and the Kueue-side `integrations.externalFrameworks` configuration are out of scope.

### Does this PR introduce _any_ user-facing change?

Yes, a new Helm value `operatorRbac.kueue.enabled` (default `false`). With the default, the rendered manifests are identical to the current chart.

### How was this patch tested?

- `helm lint --strict` passes.
- `helm template` output with the default values is identical to `main`; with `operatorRbac.kueue.enabled=true` the ClusterRole gets all five rules and, when `operatorRbac.role.create=true`, each workload-namespace Role gets only the three `workloads` rules.
- `--set operatorRbac.kueue=null` is now rejected by the values schema like every other `operatorRbac` block.
- New `helm-tests / kueue` CI job: installs Kueue, runs `helm test` with the value enabled, then upgrades to the default values and asserts the operator service account is denied `create` on `workloads.kueue.x-k8s.io`.

### Was this patch authored or co-authored using generative AI tooling?

Generated-by: Claude Fable 5.1

Closes #827 from dongjoon-hyun/SPARK-59519.

Authored-by: Dongjoon Hyun <dongjoon@apache.org>
Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>

Author: dongjoon-hyun

.github/workflows/build_and_test.yml

@@ -238,6 +238,7 @@ jobs:
           - "1.37.0"
         test-group:
           - configmap-metadata
+          - kueue
           - network-policy
           - operator-command
     steps:
@@ -267,6 +268,13 @@ jobs:
         run: |
           kubectl get pods -A
           kubectl describe node
+      - name: Install Kueue
+        if: matrix.test-group == 'kueue'
+        run: |
+          kubectl apply --server-side -f https://github.com/kubernetes-sigs/kueue/releases/download/v0.19.4/manifests.yaml
+          # Kueue installs webhooks for pods, deployments and statefulsets with failurePolicy=Fail,
+          # so wait for the controller (whose readyz covers the webhook server) before helm install.
+          kubectl wait deploy/kueue-controller-manager -n kueue-system --for=condition=Available --timeout=5m
       - name: Run Spark K8s Operator Helm Tests
         run: |
           eval $(minikube docker-env)
@@ -277,6 +285,15 @@ jobs:
           build-tools/helm/spark-kubernetes-operator/
           minikube docker-env --unset
           helm test spark
+      - name: Verify Kueue RBAC is denied by default
+        if: matrix.test-group == 'kueue'
+        run: |
+          # The same impersonated check must answer "yes" first, otherwise a wrong subject
+          # would make the denial below vacuous.
+          kubectl auth can-i create workloads.kueue.x-k8s.io --as=system:serviceaccount:default:spark-operator
+          helm upgrade spark -f build-tools/helm/spark-kubernetes-operator/values.yaml \
+          build-tools/helm/spark-kubernetes-operator/
+          if kubectl auth can-i create workloads.kueue.x-k8s.io --as=system:serviceaccount:default:spark-operator; then exit 1; fi
   lint:
     name: "Linter and documentation"
     runs-on: ubuntu-26.04

build-tools/helm/spark-kubernetes-operator/templates/operator-rbac.yaml

@@ -122,6 +122,61 @@ rules:
       - patch
       - delete
 {{- end }}
+{{- if .Values.operatorRbac.kueue.enabled }}
+  - apiGroups:
+      - "kueue.x-k8s.io"
+    resources:
+      - workloads
+    verbs:
+      - get
+      - list
+      - watch
+      - create
+      - update
+      - patch
+      - delete
+  - apiGroups:
+      - "kueue.x-k8s.io"
+    resources:
+      - workloads/status
+    verbs:
+      - get
+      - update
+      - patch
+  - apiGroups:
+      - "kueue.x-k8s.io"
+    resources:
+      - workloads/finalizers
+    verbs:
+      - update
+{{- end }}
+{{- end }}
+
+{{/*
+RBAC rules used to create the operator clusterrole: the shared rules above plus
+the rules for cluster-scoped resources, which a namespaced role cannot grant
+*/}}
+{{- define "spark-operator.operatorClusterRbacRules" }}
+{{- include "spark-operator.operatorRbacRules" . }}
+{{- if .Values.operatorRbac.kueue.enabled }}
+  - apiGroups:
+      - "kueue.x-k8s.io"
+    resources:
+      - resourceflavors
+      - workloadpriorityclasses
+    verbs:
+      - get
+      - list
+      - watch
+  - apiGroups:
+      - "scheduling.k8s.io"
+    resources:
+      - priorityclasses
+    verbs:
+      - get
+      - list
+      - watch
+{{- end }}
 {{- end }}
 
 {{/*
@@ -170,7 +225,7 @@ kind: ClusterRole
 metadata:
   name: {{ .Values.operatorRbac.clusterRole.name }}
 {{- template "spark-operator.rbacLabelsAnnotations" $ }}
-{{- template "spark-operator.operatorRbacRules" $ }}
+{{- template "spark-operator.operatorClusterRbacRules" $ }}
 ---
 {{- end }}
 {{- if and (eq (include "spark-operator.dynamicConfig.enabled" .) "true") (eq .Values.operatorConfiguration.dynamicConfig.source "configMap") }}

build-tools/helm/spark-kubernetes-operator/templates/tests/test-rbac.yaml

@@ -77,6 +77,22 @@ spec:
             kubectl auth can-i watch  grpcroutes.gateway.networking.k8s.io --all-namespaces
             kubectl auth can-i create grpcroutes.gateway.networking.k8s.io --all-namespaces
           fi
+          {{- if .Values.operatorRbac.kueue.enabled }}
+
+          # The Kueue grant is opt-in and, like Gateway API, can only be asserted
+          # where the Kueue CRDs are installed.
+          if kubectl api-resources --api-group=kueue.x-k8s.io --no-headers -o name | grep -q workloads; then
+            kubectl auth can-i watch  workloads.kueue.x-k8s.io --all-namespaces
+            kubectl auth can-i create workloads.kueue.x-k8s.io --all-namespaces
+            kubectl auth can-i patch  workloads.kueue.x-k8s.io --subresource=status --all-namespaces
+            kubectl auth can-i update workloads.kueue.x-k8s.io --subresource=finalizers --all-namespaces
+            kubectl auth can-i watch  resourceflavors.kueue.x-k8s.io
+            kubectl auth can-i watch  workloadpriorityclasses.kueue.x-k8s.io
+          fi
+
+          # PriorityClass is built in, so this one needs no CRD-presence guard.
+          kubectl auth can-i watch  priorityclasses.scheduling.k8s.io
+          {{- end }}
 
           # The operator must not be cluster-admin. These also prove the suite is
           # able to observe a denial at all. `set -e` ignores a failing `!`

build-tools/helm/spark-kubernetes-operator/values.schema.json

@@ -428,7 +428,8 @@
         "clusterRoleBinding",
         "role",
         "roleBinding",
-        "configManagement"
+        "configManagement",
+        "kueue"
       ],
       "properties": {
         "serviceAccount": {
@@ -567,6 +568,19 @@
             }
           }
         },
+        "kueue": {
+          "type": "object",
+          "description": "Kueue integration RBAC",
+          "required": [
+            "enabled"
+          ],
+          "properties": {
+            "enabled": {
+              "type": "boolean",
+              "description": "Whether to grant the operator access to Kueue resources"
+            }
+          }
+        },
         "labels": {
           "type": "object",
           "description": "Labels for RBAC resources",

build-tools/helm/spark-kubernetes-operator/values.yaml

@@ -137,6 +137,12 @@ operatorRbac:
     create: true
     roleName: "spark-operator-config-monitor"
     roleBindingName: "spark-operator-config-monitor-role-binding"
+  kueue:
+    # If enabled, the operator (cluster)role would be granted access to Kueue workloads.
+    # The cluster-scoped resourceflavors, workloadpriorityclasses and priorityclasses are
+    # granted through the ClusterRole only, so {operatorRbac.clusterRole.create} is required
+    # for the operator to read them. Kueue itself must be installed separately.
+    enabled: false
   labels:
     "app.kubernetes.io/component": "operator-rbac"
 

docs/operations.md

@@ -37,6 +37,12 @@ under the License.
   implementation such as Istio) before submitting SparkApplications that request Gateway API
   routing. If the CRDs are missing, reconciliation of such SparkApplications will fail with a
   `no matches for kind "HTTPRoute"` error from the Kubernetes API.
+- **Kueue** (`workloads.kueue.x-k8s.io`, `resourceflavors.kueue.x-k8s.io`,
+  `workloadpriorityclasses.kueue.x-k8s.io`) — required only when `operatorRbac.kueue.enabled` is
+  set. Kueue is not bundled with the operator; install it from
+  [kueue.sigs.k8s.io](https://kueue.sigs.k8s.io/docs/installation/), and register
+  `SparkApplication.v1.spark.apache.org` and `SparkCluster.v1.spark.apache.org` in Kueue's
+  `integrations.externalFrameworks`.
 
 ## Spark Application Namespaces
 
@@ -108,6 +114,7 @@ following table:
 | operatorRbac.configManagement.create                             | Enable this to create a Role for operator configuration management (hot property loading and leader election).                                                                 | true                                                                                                    |
 | operatorRbac.configManagement.roleName                           | Role name for operator configuration management.                                                                                                                               | `spark-operator-config-role`                                                                            |
 | operatorRbac.configManagement.roleBinding                        | RoleBinding name for operator configuration management.                                                                                                                        | `"spark-operator-config-monitor-role-binding"`                                                          |
+| operatorRbac.kueue.enabled                                       | Grant the operator access to Kueue `workloads`, `resourceflavors`, `workloadpriorityclasses` and to `priorityclasses`. The cluster-scoped ones need `clusterRole.create`. See [Optional Prerequisites](#optional-prerequisites). | false                                                                                                   |
 | operatorRbac.labels                                              | Labels to be applied on all created `operatorRbac` resources.                                                                                                                  | `"app.kubernetes.io/component": "operator-rbac"`                                                        |
 | workloadResources.namespaces.create                              | Whether to create dedicated namespaces for Spark workload.                                                                                                                     | true                                                                                                    |
 | workloadResources.namespaces.overrideWatchedNamespaces           | When enabled, operator would by default only watch namespace(s) provided in data field.                                                                                        | true                                                                                                    |

tests/e2e/helm/helm-test-values/kueue/values.yaml

@@ -0,0 +1,25 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# Test values for validating the opt-in Kueue RBAC rules. The Kueue CRDs must be installed
+# on the cluster for the `helm test` assertions to run.
+#
+# Usage:
+#   helm install spark-operator . -f tests/e2e/helm/helm-test-values/kueue/values.yaml
+#   helm test spark-operator
+
+operatorRbac:
+  kueue:
+    enabled: true