Skip to content

test: Scale up and down the model server during an end to end test#354

Merged
shmuelk merged 8 commits intollm-d:mainfrom
shmuelk:scale-up-down-test
Nov 10, 2025
Merged

test: Scale up and down the model server during an end to end test#354
shmuelk merged 8 commits intollm-d:mainfrom
shmuelk:scale-up-down-test

Conversation

@shmuelk
Copy link
Collaborator

@shmuelk shmuelk commented Sep 17, 2025

This PR adds a test to the end to tests.

In particular it adds a test in which:

  1. The "system" is brought up
  2. Inference requests are sent.
  3. The model server deployment is scaled up by one pod
  4. Additional inference requests are made
  5. The test validates that some of the requests go to the added pod.
  6. The model server deployment is scaled down by one
  7. Additional inference requests are made

Fixes: #347

@elevran
Copy link
Collaborator

elevran commented Nov 6, 2025

@shmuelk - please resolve conflicts

@shmuelk shmuelk marked this pull request as draft November 6, 2025 15:15
Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>
Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>
Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>
Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>
Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>
Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>
Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>
Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>
@shmuelk
Copy link
Collaborator Author

shmuelk commented Nov 10, 2025

/unhold

@shmuelk shmuelk marked this pull request as ready for review November 10, 2025 13:05
@shmuelk shmuelk requested a review from elevran November 10, 2025 13:05
@elevran
Copy link
Collaborator

elevran commented Nov 10, 2025

/lgtm
/approve

@github-actions github-actions bot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Nov 10, 2025
@shmuelk shmuelk merged commit 9755367 into llm-d:main Nov 10, 2025
6 checks passed
@github-project-automation github-project-automation bot moved this from In review to Done in llm-d-inference-scheduler Nov 10, 2025
@shmuelk shmuelk deleted the scale-up-down-test branch November 10, 2025 13:29
lionelvillard pushed a commit to lionelvillard/llm-d-inference-scheduler that referenced this pull request Nov 10, 2025
…lm-d#354)

* Added a helper to scale up/down deployments

Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>

* Added a test in which the model server is scaled up and down

Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>

* Fixed typo

Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>

* Fixed lint issue

Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>

* Restored code commented out for debugging

Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>

* Updates due to review comments

Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>

* Added comments for clarification

Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>

* Changes due to refactoring done in prior PRs

Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>

---------

Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

lgtm "Looks good to me", indicates that a PR is ready to be merged.

Projects

Archived in project

Development

Successfully merging this pull request may close these issues.

Create scale up/down test

2 participants