Skip to content

chore: Added end to end tests#310

Merged
github-actions[bot] merged 18 commits intollm-d:mainfrom
shmuelk:e2e-tests
Aug 26, 2025
Merged

chore: Added end to end tests#310
github-actions[bot] merged 18 commits intollm-d:mainfrom
shmuelk:e2e-tests

Conversation

@shmuelk
Copy link
Collaborator

@shmuelk shmuelk commented Aug 21, 2025

This PR adds an initial set of end to end tests for the llm-d-inference-scheduler. The test causes the llm-d-inference-scheduler to be built. The tests are run under a Kubernetes Kind cluster that is setup and torn down as part of the tests. The llm-d-inference-sim is used for the vLLM servers in all of the tests.

These tests include:

  1. A simple EPP configuration with a single vLLM simulator.
  2. A setup that uses Prefill/Decode disambiguation with one prefill instance and four decode instances.
  3. A setup that uses the precise prefix scorer exploiting KV events with a single vLLM simulator.

There is a set of utility functions that possibly can be upstreamed for common use in tests both in the llm-d-inference-scheduler and in the upstream Inference Gateway. This will investigated at a later time.

@elevran
Copy link
Collaborator

elevran commented Aug 25, 2025

Ref #209

@shmuelk shmuelk requested a review from elevran August 26, 2025 10:04
@elevran
Copy link
Collaborator

elevran commented Aug 26, 2025

/approve
/lgtm

github-actions[bot]
github-actions bot previously approved these changes Aug 26, 2025
shmuelk added 18 commits August 26, 2025 13:59
Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>
Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>
Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>
Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>
Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>
Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>
Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>
Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>
Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>
Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>
Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>
Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>
Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>
Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>
Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>
Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>
Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>
Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>
@elevran
Copy link
Collaborator

elevran commented Aug 26, 2025

/approve
/lgtm

@github-actions github-actions bot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Aug 26, 2025
@github-actions github-actions bot merged commit 836ea52 into llm-d:main Aug 26, 2025
5 checks passed
@shmuelk shmuelk deleted the e2e-tests branch October 23, 2025 11:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

lgtm "Looks good to me", indicates that a PR is ready to be merged.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants