Skip to content

[SPARK-60042] Support Kueue admission check podSetUpdates - #951

Closed
dongjoon-hyun wants to merge 2 commits into
apache:mainfrom
dongjoon-hyun:SPARK-60042
Closed

dongjoon-hyun wants to merge 2 commits into
apache:mainfrom
dongjoon-hyun:SPARK-60042

Conversation

@dongjoon-hyun

Copy link
Copy Markdown
Member

What changes were proposed in this pull request?

This PR aims to support Kueue admission check podSetUpdates like the Kueue built-in integrations. The labels, annotations, nodeSelector and tolerations which the admission checks report in status.admissionChecks[].podSetUpdates of a Workload are merged with the flavors and applied to the pods of SparkApplication and SparkCluster. A conflicting value fails the resource with SchedulingFailure, like a node label conflict of the flavors.

Why are the changes needed?

Previously, the operator applied only the flavors. So, the pods admitted through a ProvisioningRequest admission check lacked its annotations and node selector, and were not bound to the provisioned capacity.

Does this PR introduce any user-facing change?

No for the released versions, because the Kueue integration is not released yet in 1.0.0 (2026-07-23).

How was this patch tested?

Pass the CIs with the newly added test cases.

Was this patch authored or co-authored using generative AI tooling?

Generated-by: Claude Opus 5.5

@dongjoon-hyun

Copy link
Copy Markdown
Member Author

All tests passed. Could you review this PR when you have some time, @peter-toth ?

@peter-toth peter-toth left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR, @dongjoon-hyun!

This merges the podSetUpdates of the admission checks into the resolved flavors and applies them to the pod templates of both resources. Traced against Kueue v0.20.0, it matches getPodSetsInfoFromStatus and PodSetInfo.Merge in which update is used, in what order, and what counts as a conflict. Not checking the state of each check is safe. Kueue drops Admitted as soon as a check leaves Ready, and its webhook keeps podSetUpdates immutable while a check is Ready. I found nothing blocking.

Minor

  • 1. KueuePodSetFlavor name: the record now also carries the admission-check updates, so a name like Kueue's PodSetInfo would fit better. inline

* @param annotations The merged `annotations` of the podSetUpdates.
*/
public record KueuePodSetFlavor(Map<String, String> nodeSelector, List<Toleration> tolerations) {
public record KueuePodSetFlavor(

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Finding 1. The record now also holds the labels, annotations, nodeSelector and tolerations of the admission checks, so Flavor no longer describes what's in it. Kueue keeps the same merged shape in podset.PodSetInfo. Renaming it to KueuePodSetInfo would say what it holds. The Kueue integration isn't released yet, so a rename is cheap now. A follow-up PR is fine too.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you, @peter-toth. I renamed it to KueuePodSetInfo in this PR, along with the members which hold or produce it, e.g. setKueuePodSetInfos and resolvePodSetInfos.

@dongjoon-hyun dongjoon-hyun added this to the 1.1.0 milestone Oct 7, 2026
@dongjoon-hyun

Copy link
Copy Markdown
Member Author

Merged to main

@dongjoon-hyun
dongjoon-hyun deleted the SPARK-60042 branch October 7, 2026 18:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants