Skip to content

feat: deploy vllm-render as standalone service; use simulated render in e2e#1601

Merged
elevran merged 8 commits into
llm-d:mainfrom
mayabar:e2e-test-single-render
Jul 16, 2026
Merged

feat: deploy vllm-render as standalone service; use simulated render in e2e#1601
elevran merged 8 commits into
llm-d:mainfrom
mayabar:e2e-test-single-render

Conversation

@mayabar

@mayabar mayabar commented Jun 11, 2026

Copy link
Copy Markdown
Contributor

What type of PR is this?
/kind feature
/kind test

What this PR does / why we need it:

Extracts the vllm-render container from the EPP Deployment sidecar into its own standalone Deployment + Service. This makes the render service independently scalable — operators can tune replica count without touching the EPP deployment. It also removes the tight coupling between EPP and render lifecycle.

For e2e tests, replaces the real vllm-render instance with the inference simulator image running with --force-dummy-tokenizer.

This eliminates:

  • the HuggingFace API call on every test-case startup (which required network access and hit an external dependency unrelated to what routing tests validate)
  • the per-test render container startup wait
  • the runtime YAML mutation logic (removeRenderSidecar, isModelReal, removePodSpecListItem) that patched out the sidecar for simulated-model tests

The EPP config is updated to point at http://vllm-render:8082 (the new Service) instead of http://localhost:8000, and simulators receive --force-dummy-tokenizer to match.

Which issue(s) this PR fixes:

Ref #1595

Release note (write NONE if no user-facing change):

vllm-render is now deployed as an independent Deployment and Service rather than a sidecar in the EPP pod, allowing it to be scaled independently.

@mayabar
mayabar requested a review from a team as a code owner June 11, 2026 11:28
@mayabar
mayabar requested review from ahg-g and liu-cong June 11, 2026 11:28
@mayabar
mayabar marked this pull request as draft June 11, 2026 11:28
@github-actions github-actions Bot added kind/feature Categorizes issue or PR as related to a new feature. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Jun 11, 2026
@mayabar
mayabar force-pushed the e2e-test-single-render branch from cb95763 to e8049e7 Compare June 17, 2026 11:10
@mayabar
mayabar marked this pull request as ready for review June 17, 2026 13:05
@mayabar
mayabar force-pushed the e2e-test-single-render branch from d499a34 to bbb0e0b Compare June 21, 2026 10:44

@elevran elevran left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good PR - thanks!
It clean removal of external test dependencies and YAML mutation machinery. A few suggestions/nits before merging are noted inlnie.
Also - please resolve conflicts,

Comment thread deploy/environments/dev/base-kind-istio/vllm-render.yaml Outdated
Comment thread deploy/components/vllm-decode/deployment.yaml Outdated
Comment thread deploy/environments/dev/e2e-infra/vllm-render.yaml
Comment thread deploy/environments/dev/base-kind-istio/vllm-render.yaml Outdated
Comment thread test/e2e/setup_test.go
Comment thread test/e2e/configs_test.go Outdated
Comment thread test/e2e/configs_test.go Outdated
@ahg-g

ahg-g commented Jun 22, 2026

Copy link
Copy Markdown
Collaborator

Not that users use the helm charts for deployment, so this PR doesn't practically enable vllm-render as a standalone for end users.

@mayabar
mayabar force-pushed the e2e-test-single-render branch from 7d90a62 to abba7a5 Compare June 24, 2026 12:19
@mayabar
mayabar force-pushed the e2e-test-single-render branch from abba7a5 to 200efe7 Compare July 6, 2026 10:39
@mayabar

mayabar commented Jul 6, 2026

Copy link
Copy Markdown
Contributor Author

Not that users use the helm charts for deployment, so this PR doesn't practically enable vllm-render as a standalone for end users.

@ahg-g That's correct. The scope of this specific PR is strictly limited to updating the e2e tests and the kind deployment. Is this scope ok from your point of view?

@mayabar
mayabar force-pushed the e2e-test-single-render branch from 9fe0355 to b50eb64 Compare July 7, 2026 11:55
@mayabar
mayabar requested a review from elevran July 7, 2026 12:56
@mayabar
mayabar force-pushed the e2e-test-single-render branch from b50eb64 to c986495 Compare July 12, 2026 12:54
@github-actions github-actions Bot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Jul 12, 2026
@mayabar
mayabar force-pushed the e2e-test-single-render branch from c986495 to 407c6fd Compare July 15, 2026 11:22
@github-actions github-actions Bot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Jul 15, 2026
@mayabar
mayabar force-pushed the e2e-test-single-render branch 2 times, most recently from 1f48d63 to ca097d6 Compare July 15, 2026 18:11
…ulator pods

Signed-off-by: Maya Barnea <mayab@il.ibm.com>
mayabar added 7 commits July 15, 2026 22:01
Signed-off-by: Maya Barnea <mayab@il.ibm.com>
Signed-off-by: Maya Barnea <mayab@il.ibm.com>
Signed-off-by: Maya Barnea <mayab@il.ibm.com>
Signed-off-by: Maya Barnea <mayab@il.ibm.com>
Signed-off-by: Maya Barnea <mayab@il.ibm.com>
Signed-off-by: Maya Barnea <mayab@il.ibm.com>
Signed-off-by: Maya Barnea <mayab@il.ibm.com>
@mayabar
mayabar force-pushed the e2e-test-single-render branch from 6ffbe6b to f74e466 Compare July 15, 2026 19:01
@elevran
elevran merged commit 7794541 into llm-d:main Jul 16, 2026
29 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

kind/feature Categorizes issue or PR as related to a new feature. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants