feat(job-manager): add Kueue scheduling option for user workloads #492

CameronMcClymont · 2025-08-12T13:09:19Z

Introduces Kueue as an alternative way to submit user jobs.

Kueue is disabled by default.

See related reana PR for testing instructions.

codecov · 2025-08-12T13:13:22Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 42.02%. Comparing base (128ca8d) to head (f014442).
⚠️ Report is 1 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #492      +/-   ##
==========================================
+ Coverage   41.87%   42.02%   +0.14%     
==========================================
  Files          18       18              
  Lines        1576     1580       +4     
==========================================
+ Hits          660      664       +4     
  Misses        916      916

Files with missing lines	Coverage Δ
reana_job_controller/config.py	`100.00% <100.00%> (ø)`
reana_job_controller/kubernetes_job_manager.py	`63.41% <ø> (ø)`

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

reana_job_controller/config.py

reana_job_controller/kubernetes_job_manager.py

Co-authored-by: Xavier Tintin <[email protected]>

Co-authored-by: Xavier Tintin <[email protected]> Closes reanahub/reana#800

…anahub#492) Introduces Kueue as an alternative way to submit user jobs. Kueue will schedule Kubernetes jobs representing user runtime batch and job workloads. Co-authored-by: Xavier Tintin <[email protected]>. Closes reanahub/reana#800

tiborsimko

Works well locally 👍

CameronMcClymont self-assigned this Aug 12, 2025

CameronMcClymont force-pushed the kueue-integration branch from 97233a7 to 0009e28 Compare August 12, 2025 13:10

CameronMcClymont added a commit to CameronMcClymont/reana-job-controller that referenced this pull request Aug 12, 2025

feat(kueue): add Kueue integration (reanahub#492)

76052e5

CameronMcClymont force-pushed the kueue-integration branch from 0009e28 to 76052e5 Compare August 12, 2025 13:10

CameronMcClymont added a commit to CameronMcClymont/reana-job-controller that referenced this pull request Aug 12, 2025

feat(kueue): add Kueue integration (reanahub#492)

dcb57f2

CameronMcClymont force-pushed the kueue-integration branch from 76052e5 to dcb57f2 Compare August 12, 2025 13:14

tiborsimko reviewed Aug 13, 2025

View reviewed changes

reana_job_controller/config.py Outdated Show resolved Hide resolved

reana_job_controller/kubernetes_job_manager.py Outdated Show resolved Hide resolved

tiborsimko reviewed Aug 13, 2025

View reviewed changes

reana_job_controller/kubernetes_job_manager.py Outdated Show resolved Hide resolved

CameronMcClymont added a commit to CameronMcClymont/reana-job-controller that referenced this pull request Aug 13, 2025

feat(kueue): add Kueue integration (reanahub#492)

5836091

Co-authored-by: Xavier Tintin <[email protected]>

CameronMcClymont force-pushed the kueue-integration branch from dcb57f2 to 5836091 Compare August 13, 2025 08:36

CameronMcClymont added a commit to CameronMcClymont/reana-job-controller that referenced this pull request Aug 13, 2025

feat(kueue): add Kueue integration (reanahub#492)

15a62fd

Co-authored-by: Xavier Tintin <[email protected]> Closes reanahub/reana#800

CameronMcClymont force-pushed the kueue-integration branch from 5836091 to 15a62fd Compare August 13, 2025 08:39

CameronMcClymont added a commit to CameronMcClymont/reana-job-controller that referenced this pull request Aug 13, 2025

feat(kueue): add Kueue integration (reanahub#492)

012b603

Co-authored-by: Xavier Tintin <[email protected]> Closes reanahub/reana#800

CameronMcClymont force-pushed the kueue-integration branch from 15a62fd to 012b603 Compare August 13, 2025 08:41

CameronMcClymont added a commit to CameronMcClymont/reana-job-controller that referenced this pull request Aug 13, 2025

feat(kueue): add Kueue integration (reanahub#492)

38585f5

Co-authored-by: Xavier Tintin <[email protected]> Closes reanahub/reana#800

CameronMcClymont force-pushed the kueue-integration branch from 012b603 to 38585f5 Compare August 13, 2025 11:43

CameronMcClymont added a commit to CameronMcClymont/reana-job-controller that referenced this pull request Aug 13, 2025

feat(kueue): add Kueue integration (reanahub#492)

019d9f5

Co-authored-by: Xavier Tintin <[email protected]> Closes reanahub/reana#800

CameronMcClymont force-pushed the kueue-integration branch from 38585f5 to 019d9f5 Compare August 13, 2025 11:43

CameronMcClymont added a commit to CameronMcClymont/reana-job-controller that referenced this pull request Aug 13, 2025

feat(kueue): add Kueue integration (reanahub#492)

88d31a3

Co-authored-by: Xavier Tintin <[email protected]> Closes reanahub/reana#800

CameronMcClymont force-pushed the kueue-integration branch from 019d9f5 to 88d31a3 Compare August 13, 2025 12:09

CameronMcClymont added a commit to CameronMcClymont/reana-job-controller that referenced this pull request Aug 13, 2025

feat(kueue): add Kueue integration (reanahub#492)

95b236f

Co-authored-by: Xavier Tintin <[email protected]> Closes reanahub/reana#800

CameronMcClymont force-pushed the kueue-integration branch from 88d31a3 to 95b236f Compare August 13, 2025 12:18

CameronMcClymont added a commit to CameronMcClymont/reana-job-controller that referenced this pull request Aug 14, 2025

feat(kueue): add Kueue integration (reanahub#492)

8a2a140

Co-authored-by: Xavier Tintin <[email protected]> Closes reanahub/reana#800

CameronMcClymont force-pushed the kueue-integration branch from 95b236f to 8a2a140 Compare August 14, 2025 07:57

CameronMcClymont added a commit to CameronMcClymont/reana-job-controller that referenced this pull request Aug 14, 2025

feat(kueue): add Kueue integration (reanahub#492)

92730db

Co-authored-by: Xavier Tintin <[email protected]> Closes reanahub/reana#800

CameronMcClymont force-pushed the kueue-integration branch from 8a2a140 to 92730db Compare August 14, 2025 07:58

CameronMcClymont added a commit to CameronMcClymont/reana-job-controller that referenced this pull request Aug 14, 2025

feat(kueue): add Kueue integration (reanahub#492)

b73662a

Co-authored-by: Xavier Tintin <[email protected]> Closes reanahub/reana#800

CameronMcClymont force-pushed the kueue-integration branch from 92730db to b73662a Compare August 14, 2025 08:05

CameronMcClymont marked this pull request as ready for review August 14, 2025 08:22

CameronMcClymont mentioned this pull request Aug 14, 2025

feat(helm): add Kueue scheduling option for user workloads reanahub/reana#903

Merged

CameronMcClymont added a commit to CameronMcClymont/reana-job-controller that referenced this pull request Aug 14, 2025

feat(job-manager): add Kueue batch scheduling option (reanahub#492)

abca601

Co-authored-by: Xavier Tintin <[email protected]> Closes reanahub/reana#800

CameronMcClymont force-pushed the kueue-integration branch from b73662a to abca601 Compare August 14, 2025 12:15

CameronMcClymont added a commit to CameronMcClymont/reana-job-controller that referenced this pull request Aug 14, 2025

feat(job-manager): add Kueue batch scheduling option (reanahub#492)

adf9fbb

Co-authored-by: Xavier Tintin <[email protected]> Closes reanahub/reana#800

CameronMcClymont force-pushed the kueue-integration branch from abca601 to adf9fbb Compare August 14, 2025 12:42

CameronMcClymont changed the title ~~feat(kueue): add Kueue integration~~ feat(job-manager): add Kueue batch scheduling option Aug 14, 2025

CameronMcClymont force-pushed the kueue-integration branch from adf9fbb to 5cbe5ed Compare August 19, 2025 12:25

CameronMcClymont changed the title ~~feat(job-manager): add Kueue batch scheduling option~~ feat(job-manager): add Kueue scheduling option for user workloads Aug 19, 2025

tiborsimko force-pushed the kueue-integration branch from 5cbe5ed to f014442 Compare August 20, 2025 15:34

tiborsimko approved these changes Aug 20, 2025

View reviewed changes

tiborsimko merged commit f014442 into reanahub:master Aug 20, 2025
19 checks passed

tiborsimko mentioned this pull request Aug 20, 2025

system: Introduce Kueue as an optional alternative approach how to submit user jobs #453

Closed

github-actions bot mentioned this pull request Jul 24, 2025

chore(master): release 0.9.5 #446

Open

CameronMcClymont deleted the kueue-integration branch August 21, 2025 07:12

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat(job-manager): add Kueue scheduling option for user workloads #492

feat(job-manager): add Kueue scheduling option for user workloads #492

Uh oh!

CameronMcClymont commented Aug 12, 2025 •

edited

Loading

Uh oh!

codecov bot commented Aug 12, 2025 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

tiborsimko left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

feat(job-manager): add Kueue scheduling option for user workloads #492

feat(job-manager): add Kueue scheduling option for user workloads #492

Uh oh!

Conversation

CameronMcClymont commented Aug 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov bot commented Aug 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

Uh oh!

Uh oh!

tiborsimko left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

CameronMcClymont commented Aug 12, 2025 •

edited

Loading

codecov bot commented Aug 12, 2025 •

edited

Loading