Add E2E tests for DRA #8421

sohankunkerkar · 2025-12-25T00:02:35Z

This change adds end-to-end tests for the DRA feature, validating that Kueue correctly manages workloads using DRA.

What type of PR is this?

/kind cleanup

What this PR does / why we need it:

Which issue(s) this PR fixes:

Fixes #

Special notes for your reviewer:

Part of #8243 and we need this before pushing the Kueue + DRA feature to Beta.

Does this PR introduce a user-facing change?

None

netlify · 2025-12-25T00:02:42Z

✅ Deploy Preview for kubernetes-sigs-kueue canceled.

Name	Link
🔨 Latest commit	`4bff381`
🔍 Latest deploy log	https://app.netlify.com/projects/kubernetes-sigs-kueue/deploys/69605de451179000082fc811

Copilot

Pull request overview

This PR adds comprehensive end-to-end tests for the Dynamic Resource Allocation (DRA) feature in Kueue, validating that workloads using DRA resources are correctly managed. The changes include test infrastructure setup, DRA driver installation scripts, multiple test scenarios covering quota management, and helper functions for creating jobs with DRA resource claims.

Key Changes

Adds E2E test suite for DRA with 5 test scenarios covering admission, quota enforcement, and multi-pod jobs
Implements infrastructure to build and deploy the DRA example driver from source in the test environment
Adds configuration for enabling DRA feature gates in Kind clusters and Kueue controllers

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
test/e2e/dra/suite_test.go	Test suite setup with DRA driver availability checks
test/e2e/dra/dra_test.go	Five comprehensive test scenarios validating DRA resource management
test/e2e/config/dra/kustomization.yaml	Kustomization configuration for DRA E2E environment
test/e2e/config/dra/controller_manager_config.yaml	Controller configuration enabling DRA feature gate and device class mappings
test/e2e/config/dra/manager_e2e_patch.yaml	Image pull policy patch for E2E testing
pkg/util/testingjobs/job/wrappers.go	Helper function to add DRA resource claim templates to test jobs
hack/e2e-common.sh	Installation and configuration functions for DRA example driver
Makefile-test.mk	Make target for running DRA E2E tests
Makefile-deps.mk	DRA example driver version configuration

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

hack/e2e-common.sh

Makefile-test.mk

mimowo · 2026-01-07T08:19:16Z

hack/e2e-common.sh

+    dra_driver_temp_dir=$(mktemp -d)
+    # shellcheck disable=SC2064 # Intentionally expand now to capture the temp dir path
+    trap "rm -rf '$dra_driver_temp_dir'" RETURN
+    git clone --depth 1 --branch main "${DRA_EXAMPLE_DRIVER_REPO}" "$dra_driver_temp_dir"


This is using the main branch so it can potentially break our builds, also releases, if there is a bug upstream. I think it would be better to somehow depend on the released version, and ideally if this gets managed by dependabot.

cc @sohankunkerkar @mbobrovskyi is there a way to achieve it?

I'm ok temporarily to use "main" branch, but I would like to see a proper path forward.

cc @sohankunkerkar @mbobrovskyi is there a way to achieve it?

I think so. The repository https://github.com/kubernetes-sigs/dra-example-driver.git
is in Go, so we could probably add it as a dependency in hack/internal/tools. That way, we can bump it with each release.

I'm ok temporarily to use "main" branch, but I would like to see a proper path forward.

👍

This is using the main branch so it can potentially break our builds, also releases, if there is a bug upstream. I think it would be better to somehow depend on the released version, and ideally if this gets managed by dependabot.

I can create an issue in https://github.com/kubernetes-sigs/dra-example-driver.git to ask for the new release.
I've not see any activity for the last 3 months so I'm not sure if the repo is actively managed.

I see, please do then. It would be better to test released version.

I think we can switch to the release version when we get there. Are we going to block this PR until then?

kubernetes-sigs/dra-example-driver#135

I think we can switch to the release version when we get there. Are we going to block this PR until then?

Thank you, no need to block on that since there is already consensus, and it should be done in coming days.

Let's merge as is so that we can in parallel:

start running the new tests in another CI job

update to released version when that is ready

Makefile-test.mk

mimowo · 2026-01-08T16:47:17Z

/lgtm
/approve
Thank you 👍

k8s-ci-robot · 2026-01-08T16:47:26Z

LGTM label has been added.

Details

Git tree hash: 216035182a1ddefba2f6ff85c9c6ce443bd10afe

k8s-ci-robot · 2026-01-08T16:47:26Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: mimowo, sohankunkerkar

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

~~OWNERS~~ [mimowo]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

mimowo · 2026-01-08T17:37:09Z

/hold
as https://prow.k8s.io/view/gs/kubernetes-ci-logs/pr-logs/pull/kubernetes-sigs_kueue/8421/pull-kueue-test-e2e-main-1-34/2009311567799128064 looks relatwd. Lest make sure the tests are deflaked first

k8s-ci-robot · 2026-01-08T19:58:32Z

New changes are detected. LGTM label has been removed.

sohankunkerkar · 2026-01-08T20:06:56Z

/hold as https://prow.k8s.io/view/gs/kubernetes-ci-logs/pr-logs/pull/kubernetes-sigs_kueue/8421/pull-kueue-test-e2e-main-1-34/2009311567799128064 looks relatwd. Lest make sure the tests are deflaked first

Actually the tests run with ParallelTotal: 2, and both processes were racing to create cluster-scoped resources (dra-flavor, dra-cq) with fixed names, causing "already exists" errors when they started simultaneously. I fixed that part in the recent push.

This change adds end-to-end tests for the DRA feature, validating that Kueue correctly manages workloads using DRA. Signed-off-by: Sohan Kunkerkar <[email protected]>

Copilot AI review requested due to automatic review settings December 25, 2025 00:02

k8s-ci-robot requested review from mbobrovskyi and mimowo December 25, 2025 00:02

k8s-ci-robot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels Dec 25, 2025

Copilot started reviewing on behalf of sohankunkerkar December 25, 2025 00:03 View session

Copilot AI reviewed Dec 25, 2025

View reviewed changes

hack/e2e-common.sh Outdated Show resolved Hide resolved

sohankunkerkar force-pushed the dra-e2e branch 2 times, most recently from 83263a0 to 8e6e90e Compare December 25, 2025 00:46

sohankunkerkar mentioned this pull request Jan 5, 2026

Tracking DRA Support for Beta #8243

Open

sohankunkerkar changed the title ~~[WIP] Add E2E tests for DRA~~ Add E2E tests for DRA Jan 5, 2026

k8s-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jan 5, 2026

mbobrovskyi reviewed Jan 6, 2026

View reviewed changes

Makefile-test.mk Show resolved Hide resolved

sohankunkerkar force-pushed the dra-e2e branch 4 times, most recently from 6b70be0 to a8a46d3 Compare January 6, 2026 23:53

mimowo reviewed Jan 7, 2026

View reviewed changes

Makefile-test.mk Show resolved Hide resolved

mbobrovskyi reviewed Jan 7, 2026

View reviewed changes

Makefile-test.mk Show resolved Hide resolved

k8s-ci-robot assigned mimowo Jan 8, 2026

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jan 8, 2026

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jan 8, 2026

k8s-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jan 8, 2026

sohankunkerkar force-pushed the dra-e2e branch from a8a46d3 to 542919a Compare January 8, 2026 19:58

k8s-ci-robot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jan 8, 2026

k8s-ci-robot requested a review from mimowo January 8, 2026 19:58

sohankunkerkar force-pushed the dra-e2e branch from 542919a to 08b068f Compare January 8, 2026 20:31

Add E2E tests for DRA

4bff381

This change adds end-to-end tests for the DRA feature, validating that Kueue correctly manages workloads using DRA. Signed-off-by: Sohan Kunkerkar <[email protected]>

sohankunkerkar force-pushed the dra-e2e branch from 08b068f to 4bff381 Compare January 9, 2026 01:46

Add E2E tests for DRA #8421

Are you sure you want to change the base?

Add E2E tests for DRA #8421

Conversation

sohankunkerkar commented Dec 25, 2025

What type of PR is this?

What this PR does / why we need it:

Which issue(s) this PR fixes:

Special notes for your reviewer:

Does this PR introduce a user-facing change?

Uh oh!

netlify bot commented Dec 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Deploy Preview for kubernetes-sigs-kueue canceled.

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Key Changes

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

mimowo Jan 7, 2026

Choose a reason for hiding this comment

Uh oh!

mbobrovskyi Jan 7, 2026

Choose a reason for hiding this comment

Uh oh!

sohankunkerkar Jan 7, 2026

Choose a reason for hiding this comment

Uh oh!

mimowo Jan 7, 2026

Choose a reason for hiding this comment

Uh oh!

sohankunkerkar Jan 7, 2026

Choose a reason for hiding this comment

Uh oh!

sohankunkerkar Jan 7, 2026

Choose a reason for hiding this comment

Uh oh!

mimowo Jan 8, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

mimowo commented Jan 8, 2026

Uh oh!

k8s-ci-robot commented Jan 8, 2026

Uh oh!

k8s-ci-robot commented Jan 8, 2026

Uh oh!

mimowo commented Jan 8, 2026

Uh oh!

k8s-ci-robot commented Jan 8, 2026

Uh oh!

sohankunkerkar commented Jan 8, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

netlify bot commented Dec 25, 2025 •

edited

Loading