feat(tpu-inference/k8s): install JobSet and enable the Kueue integration - #464
Closed
theminghuang wants to merge 1 commit into
Closed
feat(tpu-inference/k8s): install JobSet and enable the Kueue integration#464theminghuang wants to merge 1 commit into
theminghuang wants to merge 1 commit into
Conversation
Collaborator
Author
|
This change is part of the following stack: Change managed by git-spice. |
This was referenced Aug 11, 2026
A batch/v1 Job cannot span hosts, so neither multi-host TPU slices (one pod
per host, gang admitted as a unit) nor prefill/decode disaggregation
(independent server pods plus a benchmark pod) can be expressed today. JobSet
covers both, and is the object Kueue gang-admits.
Install the operator on the manager and on every worker, and add
jobset.x-k8s.io/jobset to integrations.frameworks in both Kueue configs.
MultiKueue mirrors the JobSet onto the selected worker, so the CRD, the
operator version and the enabled framework list all have to match on both
sides - hence a single jobset_version variable driving both installs.
Kueue only registers the integration if the JobSet CRDs already exist when its
controller starts, so both Kueue releases now depend on the JobSet release.
The worker Kueue install also needed a trigger on its config content. A
null_resource local-exec only re-runs when a trigger changes, and the
triggers were version and cluster endpoint - so adding the JobSet framework
to worker-config.yaml would have left every worker on the old config while
terraform reported no changes.
Two adjacent fixes in the same files:
- kueue.tf pinned the chart version to a "0.19.0" literal in three places
while var.kueue_version existed and was unused, so bumping Kueue meant
editing the literals and the variable stayed silently wrong.
- waitForPodsReady is inert in both configs: the block sets blockAdmission
and timeout but never `enable: true`. Left off deliberately for now, but
annotated so it does not read as active configuration.
Signed-off-by: theminghuang <theminghuang@gmail.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
(cherry picked from commit 700917f)
theminghuang
force-pushed
the
kube-jobset-v2
branch
from
August 11, 2026 08:06
52ace1e to
6fdbd38
Compare
Collaborator
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A batch/v1 Job cannot span hosts, so neither multi-host TPU slices (one pod
per host, gang admitted as a unit) nor prefill/decode disaggregation
(independent server pods plus a benchmark pod) can be expressed today. JobSet
covers both, and is the object Kueue gang-admits.
Install the operator on the manager and on every worker, and add
jobset.x-k8s.io/jobset to integrations.frameworks in both Kueue configs.
MultiKueue mirrors the JobSet onto the selected worker, so the CRD, the
operator version and the enabled framework list all have to match on both
sides - hence a single jobset_version variable driving both installs.
Kueue only registers the integration if the JobSet CRDs already exist when its
controller starts, so both Kueue releases now depend on the JobSet release.
Two adjacent fixes in the same files:
while var.kueue_version existed and was unused, so bumping Kueue meant
editing the literals and the variable stayed silently wrong.
and timeout but never
enable: true. Left off deliberately for now, butannotated so it does not read as active configuration.
Signed-off-by: theminghuang theminghuang@gmail.com
Co-Authored-By: Claude Opus 5 noreply@anthropic.com
(cherry picked from commit 700917f)