Skip to content

Wait for storage pods to exist before moving on#138

Merged
elevran merged 1 commit intollm-d:mainfrom
david-martin:wait-for-storage-pods
May 25, 2025
Merged

Wait for storage pods to exist before moving on#138
elevran merged 1 commit intollm-d:mainfrom
david-martin:wait-for-storage-pods

Conversation

@david-martin
Copy link
Contributor

Fix for this intermittent error (race condition) when bring up the local dev environment:

+ kubectl --context kind-llm-d-inference-scheduler-dev -n local-path-storage get pods
+ '[' 0 -ne 0 ']'
+ kubectl --context kind-llm-d-inference-scheduler-dev -n local-path-storage wait --for=condition=Ready --all pods --timeout=300s
error: no matching resources found
make: *** [env-dev-kind] Error 1

@nirrozenbaum nirrozenbaum requested review from elevran and shaneutt May 22, 2025 19:41
@elevran elevran requested a review from shmuelk May 23, 2025 09:53
@elevran elevran force-pushed the wait-for-storage-pods branch from 2c96244 to 76ac5cb Compare May 23, 2025 09:53
Signed-off-by: David Martin <davmarti@redhat.com>
@elevran elevran force-pushed the wait-for-storage-pods branch from 76ac5cb to 9005fc3 Compare May 25, 2025 14:21
@elevran elevran enabled auto-merge (squash) May 25, 2025 18:12
@elevran elevran disabled auto-merge May 25, 2025 18:13
@elevran elevran merged commit 60dd83d into llm-d:main May 25, 2025
1 check passed
@david-martin david-martin deleted the wait-for-storage-pods branch May 26, 2025 07:09
shmuelk pushed a commit to shmuelk/llm-d-inference-scheduler that referenced this pull request May 29, 2025
Signed-off-by: David Martin <davmarti@redhat.com>
shmuelk added a commit that referenced this pull request May 29, 2025
* Removed unused make targets and set default tag to dev

Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>

* Changed kind development deployer to use dev tag for EPP and Simulator

Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>

* Updated documentation to reflect changes

Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>

* Removed local change

Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>

* Use correct variable in the Makefile

Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>

* Mention the vLLM simulator in the kind environment documentation

Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>

* update GIE upstream version + code adaptations (#135)

* update GIE upstream version + code adaptations

Signed-off-by: Nir Rozenbaum <nirro@il.ibm.com>

* addressed code review comments

Signed-off-by: Nir Rozenbaum <nirro@il.ibm.com>

---------

Signed-off-by: Nir Rozenbaum <nirro@il.ibm.com>

* test: Improve scheduler test (#139)

* Check headers, more tests for pd scheduler

Signed-off-by: Ira <IRAR@il.ibm.com>

* Moved config and context out of the loop, added logger to the context

Signed-off-by: Ira <IRAR@il.ibm.com>

* Removed comment and debug prints

Signed-off-by: Ira <IRAR@il.ibm.com>

---------

Signed-off-by: Ira <IRAR@il.ibm.com>

* Wait for storage pods to exist before moving on (#138)

Signed-off-by: David Martin <davmarti@redhat.com>

* dependabot configuration for Go, GH acions and Docker

Signed-off-by: Etai Lev Ran <elevran@gmail.com>

* link checker - fails PR if links are broken (#130)

* link checker - fails PR if links are broken

Signed-off-by: Andy Anderson <andy@clubanderson.com>

* replace markdown-link-check tool with lychee
* remove markdown-link-check tool custom action
* make README links relative
* increase timeouts and ignore gnu's website since it throttles by timeouts

Signed-off-by: Etai Lev Ran <elevran@gmail.com>

---------

Signed-off-by: Etai Lev Ran <elevran@gmail.com>
Co-authored-by: Etai Lev Ran <elevran@gmail.com>

* Corrected name of vLLM simulator

Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>

---------

Signed-off-by: Shmuel Kallner <kallner@il.ibm.com>
Signed-off-by: Nir Rozenbaum <nirro@il.ibm.com>
Signed-off-by: Ira <IRAR@il.ibm.com>
Signed-off-by: David Martin <davmarti@redhat.com>
Signed-off-by: Etai Lev Ran <elevran@gmail.com>
Co-authored-by: Nir Rozenbaum <nirro@il.ibm.com>
Co-authored-by: Ira Rosen <irar@il.ibm.com>
Co-authored-by: David Martin <david-martin@users.noreply.github.com>
Co-authored-by: Etai Lev Ran <elevran@gmail.com>
Co-authored-by: Andy Anderson <andy@clubanderson.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants