test: Parallelize the End to End tests - Part two enablement by shmuelk · Pull Request #2033 · llm-d/llm-d-router

shmuelk · 2026-07-15T14:45:01Z

What type of PR is this?
/kind test

What this PR does / why we need it:
Currently the llm-d-router End to End tests run for quite a long time (about sixteen minutes on my Mac M4 Max).

This long time has affected the CI pipeline in the past. A work around was done to run the tests in parts in parallel in separate Kind clusters. This splitting up of the tests needs to be managed manually and has overhead as prepetory steps are done over and over again for each test "slice'

A better solution is to run tests in parallel on the same Kind cluster using K8S Namespaces for separation as needed.

This PR enables the End to End tests to run in parallel. The tests are broken up into groups, mostly by Ginkgo When blocks. By default five such groups run in parallel, all on the same Kind cluster, each using its own namespace with an envoy instance and its own nodePort.

On my laptop a Mac M4 (Max) this PR has reduced the time to run the tests from about sixteen minutes, to between five and six minutes, a significant improvement.

A future PR will update CI to exploit this PR and remove the "test labels"

Which issue(s) this PR fixes:
Refs #1931

Release note (write NONE if no user-facing change):

NONE

Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>

elevran · 2026-07-20T14:36:53Z

lint not passing

elevran

Parallelizing the e2e suite via Ginkgo's multi-process runner is a good direction and the 10m saving is significant - thanks!

Two things need attention before merge:

CI lint is currently failing (staticcheck QF1001 in setup_test.go) - see inline comment.
The new NodePort computation ignores any overrides in E2E_PORT/E2E_METRICS_PORT and would no longer match the actual NodePort of the created Service.

elevran · 2026-07-20T14:56:34Z

+
+		ginkgo.AfterAll(func() {
+			// Only cleanup if the test succeeded
+			if !(ginkgo.CurrentSpecReport().Failed() && keepClusterOnFailure) {


CI lint fails

Suggested change

if !(ginkgo.CurrentSpecReport().Failed() && keepClusterOnFailure) {

if !ginkgo.CurrentSpecReport().Failed() || !keepClusterOnFailure {

elevran · 2026-07-20T14:56:34Z

+
+			infraSubs := map[string]string{
+				"${EPP_NAME}":          "e2e-epp",
+				"${METRICS_NODE_PORT}": strconv.Itoa(32090 + 100*(ginkgo.GinkgoParallelProcess()-1)),


This hardcodes 32090 instead of using baseMetricsPort, diverging from getMetricsPort() (via testutils.ProcessPort(baseMetricsPort)) used elsewhere in this file.
If a user overrides E2E_METRICS_PORT, the client dials the overridden port but the NodePort Service created here still binds to 32090-based values.

elevran · 2026-07-20T14:56:34Z

+func createEnvoy(nsName string) ([]string, *gexec.Session) {
+	infraSubs := map[string]string{
+		"${NAMESPACE}":       nsName,
+		"${ENVOY_NODE_PORT}": strconv.Itoa(30080 + 100*(ginkgo.GinkgoParallelProcess()-1)),


Same issue as the metrics port: this hardcodes 30080 instead of basePort.
Overriding E2E_PORT would break the NodePort/client-port match.

elevran · 2026-07-20T14:56:34Z

+
+// testWrapper wraps tests with the setup and teardown code needed.
+// It is used as a wrapper of the function passed to ginkgo.When calls that
+// setup the tests. It is important that the gink.Ordered decorator is used


Suggested change

// setup the tests. It is important that the gink.Ordered decorator is used

// setup the tests. It is important that the ginkgo.Ordered decorator is used

elevran · 2026-07-20T14:56:34Z

+    E2E_NUM_PROCS=1 go test -v -timeout 45m "${pkg}" -ginkgo.v -ginkgo.fail-fast "-ginkgo.label-filter=${E2E_LABEL_FILTER}"
  else
-    go test -v -timeout 45m "${pkg}" -ginkgo.v -ginkgo.fail-fast
+    ginkgo run --procs=${E2E_NUM_PROCS} --timeout 45m -v --fail-fast "${pkg}"


nit:
${E2E_NUM_PROCS} is unquoted here, unlike the other variable expansions in this function (e.g., "${pkg}", "${E2E_LABEL_FILTER}"). Low risk but worth being consistent.

Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>

shmuelk · 2026-07-21T06:12:59Z

@elevran Thanks for the thorough review. I believe all of theissues raised have been fixed.

shmuelk requested a review from a team as a code owner July 15, 2026 14:45

shmuelk requested review from elevran and vMaroon July 15, 2026 14:45

github-actions Bot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Jul 15, 2026

shmuelk force-pushed the e2e-parallel-2 branch from e129cbe to e4cb43c Compare July 16, 2026 09:11

github-actions Bot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Jul 16, 2026

shmuelk force-pushed the e2e-parallel-2 branch from 78824d3 to 6a3ae27 Compare July 19, 2026 08:54

shmuelk added 21 commits July 19, 2026 13:59

Updated YAML templates to not hard code nodePorts

e7e4082

Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>

Added test wrapper function with common BeforeAll and AfterAll functions

367d7ed

Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>

Improved wait for EPP to start

afe010c

Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>

Removed setup/teardown code that is now part of the test wrapper

6e132cb

Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>

Create envoy service with appropriate nodePort

fd88d73

Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>

No need to delete inference pools in tests

228cc2c

Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>

High level setup done once and just some done per process

af08689

Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>

Namespaces, if created, managed by test wrapper

1d84009

Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>

Various utilities now work via namespace

137c023

Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>

Each test creates and deletes its own InferencePool

7a7fbea

Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>

Do not use DeferedCleanup, interferes with test wrapper's AfterAll

d3ef51a

Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>

Exploit test wrapper in tests

d82326c

Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>

Updated to use ginkgo CLI to enable parallel tests

e6928e1

Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>

Set default number of processes for parallel tests

97b3bea

Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>

Updated E2E documentation WRT parallel tests

16d97a3

Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>

Fixed lint issues

e60a826

Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>

Force E2E_NUM_PROCS to equal one, when not running tests in parallel

920b299

Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>

Handle creating the vLLM renderer when testing in parallel

a9b5ac2

Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>

Rebase conflict cleanup

1100f6b

Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>

Correct the URL of the vLLM Renderer

574cc03

Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>

Fixed lint issue

0f6c834

Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>

shmuelk force-pushed the e2e-parallel-2 branch from 6a3ae27 to 0f6c834 Compare July 19, 2026 11:08

shmuelk added 2 commits July 19, 2026 14:39

Fixes to rebased code

dac9999

Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>

Compute the namespace when the test actually runs

026fcb6

Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>

elevran self-assigned this Jul 20, 2026

On failure, don't cleanup only if flag is set

167cfc9

Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>

elevran reviewed Jul 20, 2026

View reviewed changes

shmuelk added 3 commits July 21, 2026 09:10

Use proper port for nodePort

1f02708

Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>

Fix lint issues

65c0546

Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>

Review comment

52755fd

Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>

elevran approved these changes Jul 21, 2026

View reviewed changes

elevran merged commit feafdf4 into llm-d:main Jul 21, 2026
35 checks passed

shmuelk mentioned this pull request Jul 21, 2026

CI: Exploit parallel Router End to End testing in CI #2115

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

test: Parallelize the End to End tests - Part two enablement#2033

test: Parallelize the End to End tests - Part two enablement#2033
elevran merged 27 commits into
llm-d:mainfrom
shmuelk:e2e-parallel-2

shmuelk commented Jul 15, 2026

Uh oh!

elevran commented Jul 20, 2026

Uh oh!

elevran left a comment

Uh oh!

elevran Jul 20, 2026

Uh oh!

shmuelk Jul 21, 2026

Uh oh!

elevran Jul 20, 2026

Uh oh!

elevran Jul 20, 2026

Uh oh!

elevran Jul 20, 2026

Uh oh!

elevran Jul 20, 2026

Uh oh!

shmuelk commented Jul 21, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

	if !(ginkgo.CurrentSpecReport().Failed() && keepClusterOnFailure) {
	if !ginkgo.CurrentSpecReport().Failed() \|\| !keepClusterOnFailure {

	// setup the tests. It is important that the gink.Ordered decorator is used
	// setup the tests. It is important that the ginkgo.Ordered decorator is used

Uh oh!

Conversation

shmuelk commented Jul 15, 2026

Uh oh!

elevran commented Jul 20, 2026

Uh oh!

elevran left a comment

Choose a reason for hiding this comment

Uh oh!

elevran Jul 20, 2026

Choose a reason for hiding this comment

Uh oh!

shmuelk Jul 21, 2026

Choose a reason for hiding this comment

Uh oh!

elevran Jul 20, 2026

Choose a reason for hiding this comment

Uh oh!

elevran Jul 20, 2026

Choose a reason for hiding this comment

Uh oh!

elevran Jul 20, 2026

Choose a reason for hiding this comment

Uh oh!

elevran Jul 20, 2026

Choose a reason for hiding this comment

Uh oh!

shmuelk commented Jul 21, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants