Skip to content

Latest commit

 

History

History
163 lines (105 loc) · 8.53 KB

File metadata and controls

163 lines (105 loc) · 8.53 KB

2025 Annual Report: WG Batch

Current initiatives and Project Health

  1. What work did the WG do this year that should be highlighted?

See 2025 Highlights.

  1. Are there any areas and/or subprojects that your group needs help with (e.g. fewer than 2 active OWNERS)?

No, all subprojects have sufficient active owners.

2025 Highlights

We will break down our highlights into Sub Projects, KEPs, talks, community adoption.

Sub Projects

Kueue

Kueue has had 5 minor releases in 2025.

In 2025, the Kueue community would like to highlight Topology Aware Scheduling, MultiKueue, Admission Fair Sharing, Elastic Jobs, DRA Integration, v1beta2 API and KueueViz Dashboard.

Topology Aware Scheduling matured from alpha to beta (enabled by default in 0.14), facilitating scheduling of workloads that take into account data center topology. Workloads benefit from using interconnects that are physically close together.

MultiKueue expanded to support RayCluster, RayJob, and Pods. An external dispatcher API was introduced in 0.13 for nominating worker clusters. Security hardening with kubeconfig validation was added in 0.15.

Admission Fair Sharing progressed from alpha (0.12) to beta (0.15). This feature orders workloads based on recent LocalQueue usage rather than just priority, preventing queue manipulation and ensuring fair resource distribution.

Elastic Jobs via WorkloadSlices was introduced in 0.13 as an alpha feature. This enables dynamic job resizing without suspension or requeueing.

DRA Integration was introduced in 0.14 as an alpha feature, providing Dynamic Resource Allocation support for specialized hardware.

v1beta2 API was introduced in 0.15, representing API maturation toward stability.

KueueViz Dashboard was hardened for production with Helm charts for installation and rebranded with the CNCF logo.

JobSet

JobSet has had 3 minor releases in 2025.

In 2025, the JobSet community would like to highlight the Kubernetes blog post, Failure Policy, DependsOn API, and Coordinator.

The official Kubernetes blog post Introducing JobSet was published in March 2025.

Failure Policy added the onJobFailureMessagePatterns field to distinguish retriable from non-retriable Pod failure policies. Previously, all failures matching a Pod Failure Policy resulted in the same PodFailurePolicy job failure reason, making it impossible to apply different JobSet actions. With onJobFailureMessagePatterns, rules can match against failure message strings to differentiate between cases such as a non-retriable exit code (fail immediately) and a node eviction (restart without counting toward the limit).

DependsOn API defines execution dependencies between ReplicatedJobs within a JobSet, enabling serial/sequential execution. A ReplicatedJob can declare that it depends on another ReplicatedJob reaching either Ready or Complete status before it starts. This supports multi-stage workflows like data initialization followed by distributed training.

Coordinator designates a specific Pod within a JobSet as the coordinator for distributed ML/HPC workloads. A stable network endpoint for the coordinator is added as a label and annotation to every Job and Pod in the JobSet, enabling other pods to discover the coordinator without hardcoding addresses.

KJob

KJob had its first release (v0.1.0) in 2025, providing the base functionality for CLI-friendly batch job submission. KJob provides a template-based job execution with built-in SLURM support and kubectl plugin integration. The HPC/ML community tend to prefer CLI over YAML so the focus was to provide a templated solution for submitting batch jobs and a smooth transition for Slurm users.

KEPs

WG-Batch provided a series of Kubernetes enhancements that improved the experience of batch workloads on Kubernetes. In 2025, this group proposed/implemented/consulted the following KEPs.

Talks

  • Accelerate Your AI/ML Workloads With Topology-Aware Scheduling in Kueue

    • Speakers: Michal Wozniak and Yuki Iwai
    • KubeCon EU, London
    • Recording
  • Tutorial: Build, Operate, and Use a Multi-Tenant AI Cluster Based Entirely on Open Source

    • Speakers: Claudia Misale, Olivier Tardieu, and David Grove
    • KubeCon EU, London
    • Recording
  • Kueue: Save Some QPS for the Rest of Us! How To Manage 100k Updates Per Second

    • Speaker: Patryk Bundyra
    • KubeCon EU, London
    • Recording
  • WG-Batch Updates: What's New and What Is Next?

    • Speaker: Marcin Wielgus
    • KubeCon EU, London
    • Recording
  • From High Performance Computing to AI Workloads on Kubernetes: MPI Runtime in Kubeflow TrainJob

    • Speakers: Andrey Velichkevich and Yuki Iwai
    • KubeCon EU, London
    • Recording
  • Resource Fairness and Utilization for Heterogeneous Batch/ML Platforms With Kueue

    • Speakers: Yuki Iwai and Gabe Saba
    • KubeCon NA, Atlanta
    • Recording
  • WG-Batch Updates: What's New and What Is Next?

    • Speakers: Michal Wozniak and Yuki Iwai
    • KubeCon Japan, Tokyo
    • Recording

Community adoption

Operational

Operational tasks in wg-governance.md: