Skip to content

refactor(tpu-inference/k8s): collapse duplicated Kueue queue templates - #462

Closed
theminghuang wants to merge 1 commit into
feature/tpu-multikueue-k8s-infrafrom
kube-merge-queue-templates
Closed

refactor(tpu-inference/k8s): collapse duplicated Kueue queue templates#462
theminghuang wants to merge 1 commit into
feature/tpu-multikueue-k8s-infrafrom
kube-merge-queue-templates

Conversation

@theminghuang

Copy link
Copy Markdown
Collaborator

queue_group_manager.yaml.tpl and queue_group_worker.yaml.tpl are identical
apart from the MultiKueue admissionChecksStrategy block. Every change to the
quota model - preemption policy, covered resources, flavor naming - has to be
made twice and kept in sync by hand, and there is nothing that catches a miss.

Replace both with queue_group.yaml.tpl plus an ${ADMISSION_CHECKS}
substitution the generator fills for the manager and leaves empty for workers,
and route both render paths through a shared render_queue_group() helper.

Verified as a pure refactor: regenerating from the committed prod.auto.tfvars
produces byte-identical output for every file under generated/, both the
modular parts and the consolidated bundles.

Also normalises hcl2 object keys. Keys written quoted in tfvars keep their
quotes through the parser and become Kubernetes object names, so a quoted
worker or pool key silently yields a ClusterQueue literally named
"v6e-1-1x1". The current tfvars uses bare identifiers so this is latent
rather than live, but it fails a long way from its cause. Confirmed both
spellings now generate identical manifests.

Adds scripts/requirements.txt; the generator imports hcl2 and nothing declared
it, so it fails with ModuleNotFoundError on a clean checkout.

Signed-off-by: theminghuang theminghuang@gmail.com

Co-Authored-By: Claude Opus 5 noreply@anthropic.com

queue_group_manager.yaml.tpl and queue_group_worker.yaml.tpl are identical
apart from the MultiKueue admissionChecksStrategy block. Every change to the
quota model - preemption policy, covered resources, flavor naming - has to be
made twice and kept in sync by hand, and there is nothing that catches a miss.

Replace both with queue_group.yaml.tpl plus an ${ADMISSION_CHECKS}
substitution the generator fills for the manager and leaves empty for workers,
and route both render paths through a shared render_queue_group() helper.

Verified as a pure refactor: regenerating from the committed prod.auto.tfvars
produces byte-identical output for every file under generated/, both the
modular parts and the consolidated bundles.

Also normalises hcl2 object keys. Keys written quoted in tfvars keep their
quotes through the parser and become Kubernetes object names, so a quoted
worker or pool key silently yields a ClusterQueue literally named
"v6e-1-1x1". The current tfvars uses bare identifiers so this is latent
rather than live, but it fails a long way from its cause. Confirmed both
spellings now generate identical manifests.

Adds scripts/requirements.txt; the generator imports hcl2 and nothing declared
it, so it fails with ModuleNotFoundError on a clean checkout.

Signed-off-by: theminghuang <theminghuang@gmail.com>

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@theminghuang

Copy link
Copy Markdown
Collaborator Author

Superseded by #466 (infra) and #467 (launcher), which present the same changes as two PRs instead of four.

@theminghuang
theminghuang deleted the kube-merge-queue-templates branch August 11, 2026 08:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant