Skip to content

Commit b5bcd6f

Browse files
peter-tothdongjoon-hyun
authored andcommitted
[SPARK-59578] Clarify that Kueue integrations.externalFrameworks is not needed in operations.md
### What changes were proposed in this pull request? This PR replaces the instruction to register `SparkApplication.v1.spark.apache.org` and `SparkCluster.v1.spark.apache.org` in Kueue's `integrations.externalFrameworks` in the Kueue entry of `docs/operations.md` with a sentence saying it is not needed, and why. ### Why are the changes needed? The registration has no effect for this operator, so the instruction asks users to configure something that does nothing. Traced against Kueue `v0.19.4`: - `integrations.externalFrameworks` only populates a lookup table. `registerExternal` (`pkg/controller/jobframework/integrationmanager.go:149-169`) stores a `PartialObjectMetadata` for the kind in `m.externalIntegrations`, and `setupControllers` (`pkg/controller/jobframework/setup.go:66-70`) calls `RegisterExternalJobType` for external frameworks and nothing else — no controller and no webhook is built for them. - The only owner-related consumer of that table is `getJobTypeForOwner` (`integrationmanager.go:230-244`). - Its caller `IsOwnerManagedByKueueForObject` (`integrationmanager.go:358-363`) starts with `metav1.GetControllerOf(obj)`, which returns the owner reference whose `controller` field is true. - The operator does not set a controller owner reference on the pods it creates: `ModelUtils.buildOwnerReferenceTo` sets `blockOwnerDeletion` only, and the single place the operator sets `controller: true` is `KueueWorkloadFactory`, on the `Workload` itself. So `GetControllerOf` returns nil for a driver or executor pod and the registry is never consulted. The operator also does not need Kueue to manage `Workload`s on its behalf: it builds them with `KueueWorkloadFactory` and deletes them in `AppCleanUpStep`. The same reasoning rules out the one case where the registration might have looked load-bearing. `defaultLocalQueueApplies` (`pkg/controller/jobframework/defaults.go:80-88`) has no feature gate, so in a namespace holding a `LocalQueue` named `default` Kueue can add `queue-name: default` to an object whose owner it does not manage. But that branch returns `!IsOwnerManagedByKueueForObject(jobObj)`, which evaluates the same way with or without the registration, again because there is no controller reference to resolve. ### Does this PR introduce _any_ user-facing change? No, documentation only. ### How was this patch tested? Manual review against the Kueue `v0.19.4` sources cited above. `integrations.externalFrameworks` had no other mention in the repository, so nothing else goes stale. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude Opus 5 Closes #844 from peter-toth/SPARK-59578-clarify-kueue-external-frameworks. Authored-by: Peter Toth <peter.toth@gmail.com> Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
1 parent d848ee2 commit b5bcd6f

1 file changed

Lines changed: 4 additions & 3 deletions

File tree

‎docs/operations.md‎

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -40,9 +40,10 @@ under the License.
4040
- **Kueue** (`workloads.kueue.x-k8s.io`, `resourceflavors.kueue.x-k8s.io`,
4141
`workloadpriorityclasses.kueue.x-k8s.io`) — required only when `operatorRbac.kueue.enabled` is
4242
set. Kueue is not bundled with the operator; install it from
43-
[kueue.sigs.k8s.io](https://kueue.sigs.k8s.io/docs/installation/), and register
44-
`SparkApplication.v1.spark.apache.org` and `SparkCluster.v1.spark.apache.org` in Kueue's
45-
`integrations.externalFrameworks`. `operatorRbac.kueue.enabled` also sets
43+
[kueue.sigs.k8s.io](https://kueue.sigs.k8s.io/docs/installation/). Kueue's
44+
`integrations.externalFrameworks` does not need to list `SparkApplication` or `SparkCluster`:
45+
the operator manages the `Workload` objects for them, so Kueue does not have to
46+
recognize the Spark custom resources as job kinds. `operatorRbac.kueue.enabled` also sets
4647
`spark.kubernetes.operator.kueue.workloadInformer.enabled`, so that the operator watches
4748
`Workload` resources and an admission starts the queued resource right away. Without the watch,
4849
an admission is noticed on the next periodic reconcile. The watch requires Kueue to be installed

0 commit comments

Comments
 (0)