[kimi2.6] enable model text only initial - #1574
Open
sureshnam wants to merge 1 commit into
Open
Conversation
Adds VLLM_SKIP_MM_WARMUP=1 to skip only the multimodal-graph warm-up while still running the prompt and decode graph warm-up. Motivation: some multimodal models include vision towers whose upstream implementation hits a torch._inductor code path that has no 'hpu' backend registered, causing engine init to crash in warmup_multimodal_graphs. Today the only workaround is VLLM_SKIP_WARMUP, which disables all warmup and causes runtime JIT-compile stalls on every new bucket size hit. With this knob, users running such models text-only can keep the normal prompt/decode bucket warmup and avoid the runtime stalls, while skipping just the vision compile that they don't need. Pattern follows existing in-file env-var gates such as VLLM_SKIP_MARK_UNBACKED. Documents the new variable under Developer Mode Parameters next to VLLM_SKIP_WARMUP. Signed-off-by: Suresh Nampalli <suresh.b.nampalli@intel.com>
sureshnam
requested review from
PatrykWo,
adobrzyn,
afierka-intel,
iboiko-habana,
jbyczkow,
mgawarkiewicz-intel,
michalkuligowski and
xuechendi
as code owners
June 28, 2026 22:05
sureshnam
had a problem deploying
to
pre-merge-approval
June 28, 2026 22:05 — with 
GitHub Actions
Error
GitHub Actions
Error
Contributor
There was a problem hiding this comment.
Pull request overview
Adds a new env var gate to selectively skip multimodal (vision) warmup graph compilation on Gaudi, allowing text-only usage of multimodal models to keep prompt/decode warmup (and avoid runtime JIT stalls) while bypassing vision tower compilation failures.
Changes:
- Add
VLLM_SKIP_MM_WARMUPhandling inHPUModelRunner.warmup_model()to skipwarmup_multimodal_graphs()while preserving prompt/decode warmup. - Add documentation for
VLLM_SKIP_MM_WARMUPunder Developer Mode parameters.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
vllm_gaudi/v1/worker/hpu_model_runner.py |
Adds an env-var gate to skip only multimodal warmup graphs in both lazy-mode and torch.compile warmup flows. |
docs/configuration/env_variables.md |
Documents the new VLLM_SKIP_MM_WARMUP variable next to VLLM_SKIP_WARMUP. |
| # uses a torch._inductor path without an 'hpu' backend), and when the | ||
| # user is running the model text-only and does not need the vision | ||
| # path. Differs from VLLM_SKIP_WARMUP, which disables all warmup. | ||
| skip_mm_warmup = os.environ.get('VLLM_SKIP_MM_WARMUP', '0').lower() in ('1', 'true') |
adobrzyn
reviewed
Jun 29, 2026
| # uses a torch._inductor path without an 'hpu' backend), and when the | ||
| # user is running the model text-only and does not need the vision | ||
| # path. Differs from VLLM_SKIP_WARMUP, which disables all warmup. | ||
| skip_mm_warmup = os.environ.get('VLLM_SKIP_MM_WARMUP', '0').lower() in ('1', 'true') |
Collaborator
There was a problem hiding this comment.
Can we add it to features and use get_config?
Collaborator
|
@sureshnam do we need this PR or it can be closed |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds VLLM_SKIP_MM_WARMUP=1 to skip only the multimodal-graph warm-up while still running the prompt and decode graph warm-up.
Motivation: some multimodal models include vision towers whose upstream implementation hits a torch._inductor code path that has no 'hpu' backend registered, causing engine init to crash in warmup_multimodal_graphs. Today the only workaround is VLLM_SKIP_WARMUP, which disables all warmup and causes runtime JIT-compile stalls on every new bucket size hit. With this knob, users running such models text-only can keep the normal prompt/decode bucket warmup and avoid the runtime stalls, while skipping just the vision compile that they don't need.
Pattern follows existing in-file env-var gates such as VLLM_SKIP_MARK_UNBACKED. Documents the new variable under Developer Mode Parameters next to VLLM_SKIP_WARMUP.