Skip to content

[Tests] Skip model weight download for render-only test server#36813

Open
sagearc wants to merge 4 commits intovllm-project:mainfrom
sagearc:fix-render-server-model-download
Open

[Tests] Skip model weight download for render-only test server#36813
sagearc wants to merge 4 commits intovllm-project:mainfrom
sagearc:fix-render-server-model-download

Conversation

@sagearc
Copy link
Contributor

@sagearc sagearc commented Mar 11, 2026

Purpose

RemoteLaunchRenderServer inherits the full model weight download from RemoteVLLMServer.__init__, but the render server only needs the tokenizer — no weights are loaded at runtime.

Changes

  • Extract _pre_download_model hook from RemoteVLLMServer.__init__
  • Override it as a no-op in RemoteLaunchRenderServer

Test Plan

Existing render tests (test_launch_render.py) cover this path.


Essential Elements of an Effective PR Description Checklist
  • The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
  • The test plan, such as providing test command.
  • The test results, such as pasting the results comparison before and after, or e2e results
  • (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
  • (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors the model pre-download logic to allow skipping it for the render-only test server, which is a sensible optimization. However, the new implementation for the render server completely skips any pre-download. Since the render server still requires the tokenizer, this could lead to test flakiness if the server startup times out while downloading it. I've provided a suggestion to explicitly pre-download only the tokenizer for the render server to make the tests more robust.

@sagearc
Copy link
Contributor Author

sagearc commented Mar 11, 2026

cc @DarkLight1337 @NickLucche

sagearc added 2 commits March 11, 2026 21:55
Signed-off-by: Sage Ahrac <sagiahrak@gmail.com>
Signed-off-by: Sage Ahrac <sagiahrak@gmail.com>
@sagearc sagearc force-pushed the fix-render-server-model-download branch from 3bfce13 to 4e3d23e Compare March 11, 2026 19:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants