[Fix] Improve ROCm detection in WSL environments

[yiz-liu]

Purpose

Adds fallback detection for ROCm platforms running in WSL by checking torch.version.hip as amd-smi and rocm-smi is unreachable in WSL now. This enables proper platform identification for ROCm setups in WSL where traditional detection methods may not work reliably.

Also replaces warning_once with warning in GCN architecture detection to avoid circular import issues that arise when the distributed module attempts to query the current platform.

Test Plan

Currently only tested with one device and VLLM_​ATTENTION_​BACKEND=TRITON_ATTN

Test Result

Details

~$ python scripts/offline-inference.py
WARNING 03-28 23:31:21 [rocm.py:38] Failed to import from amdsmi with ModuleNotFoundError("No module named 'amdsmi'")
WARNING 03-28 23:31:21 [rocm.py:135] Failed to get GCN arch via amdsmi, falling back to torch.cuda. This will initialize CUDA and may cause issues if CUDA_VISIBLE_DEVICES is not set yet.
INFO 03-28 23:31:24 [utils.py:233] non-default args: {'dtype': 'float16', 'max_model_len': 64, 'gpu_memory_utilization': 0.5, 'max_num_seqs': 4, 'disable_log_stats': True, 'enforce_eager': True, 'language_model_only': True, 'model': 'Qwen3.5-9B-AWQ-4bit'}
INFO 03-28 23:31:24 [model.py:549] Resolved architecture: Qwen3_5ForConditionalGeneration
WARNING 03-28 23:31:24 [model.py:2003] Casting torch.bfloat16 to torch.float16.
INFO 03-28 23:31:24 [model.py:1665] Using max model len 64
WARNING 03-28 23:31:25 [quark_ocp_mx.py:151] AITER is not found or QuarkOCP_MX is not supported on the current platform. QuarkOCP_MX quantization will not be available.
INFO 03-28 23:31:25 [scheduler.py:238] Chunked prefill is enabled with max_num_batched_tokens=256.
INFO 03-28 23:31:25 [config.py:228] Setting attention block size to 528 tokens to ensure that attention page size is >= mamba page size.
INFO 03-28 23:31:25 [config.py:259] Padding mamba page size by 0.76% to ensure that mamba page size and attention page size are exactly equal.
INFO 03-28 23:31:25 [vllm.py:786] Asynchronous scheduling is enabled.
WARNING 03-28 23:31:25 [vllm.py:844] Enforce eager set, disabling torch.compile and CUDAGraphs. This is equivalent to setting -cc.mode=none -cc.cudagraph_mode=none
WARNING 03-28 23:31:25 [vllm.py:855] Inductor compilation was disabled by user settings, optimizations settings that are only active during inductor compilation will be ignored.
INFO 03-28 23:31:25 [vllm.py:1021] Cudagraph is disabled under eager mode
INFO 03-28 23:31:25 [compilation.py:290] Enabled custom fusions: norm_quant, act_quant
INFO 03-28 23:31:25 [registry.py:126] All limits of multimodal modalities supported by the model are set to 0, running in text-only mode.
WARNING 03-28 23:31:26 [system_utils.py:152] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: CUDA is initialized; WSL is detected and NVML is not compatible with fork
WARNING 03-28 23:31:28 [rocm.py:38] Failed to import from amdsmi with ModuleNotFoundError("No module named 'amdsmi'")
WARNING 03-28 23:31:28 [rocm.py:135] Failed to get GCN arch via amdsmi, falling back to torch.cuda. This will initialize CUDA and may cause issues if CUDA_VISIBLE_DEVICES is not set yet.
  warnings.warn(
(EngineCore pid=60466) INFO 03-28 23:31:31 [core.py:105] Initializing a V1 LLM engine (v0.18.1rc1.dev211+g6dad4c572.d20260328) with config: model='Qwen3.5-9B-AWQ-4bit', speculative_config=None, tokenizer='Qwen3.5-9B-AWQ-4bit', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, tokenizer_revision=None, trust_remote_code=False, dtype=torch.float16, max_seq_len=64, download_dir=None, load_format=auto, tensor_parallel_size=1, pipeline_parallel_size=1, data_parallel_size=1, decode_context_parallel_size=1, dcp_comm_backend=ag_rs, disable_custom_all_reduce=True, quantization=compressed-tensors, enforce_eager=True, enable_return_routed_experts=False, kv_cache_dtype=auto, device_config=cuda, structured_outputs_config=StructuredOutputsConfig(backend='auto', disable_any_whitespace=False, disable_additional_properties=False, reasoning_parser='', reasoning_parser_plugin='', enable_in_reasoning=False), observability_config=ObservabilityConfig(show_hidden_metrics_for_version=None, otlp_traces_endpoint=None, collect_detailed_traces=None, kv_cache_metrics=False, kv_cache_metrics_sample=0.01, cudagraph_metrics=False, enable_layerwise_nvtx_tracing=False, enable_mfu_metrics=False, enable_mm_processor_stats=False, enable_logging_iteration_details=False), seed=0, served_model_name=Qwen3.5-9B-AWQ-4bit, enable_prefix_caching=False, enable_chunked_prefill=True, pooler_config=None, compilation_config={'mode': <CompilationMode.NONE: 0>, 'debug_dump_path': None, 'cache_dir': '', 'compile_cache_save_format': 'binary', 'backend': 'inductor', 'custom_ops': ['+sparse_attn_indexer', 'all'], 'splitting_ops': [], 'compile_mm_encoder': False, 'cudagraph_mm_encoder': False, 'encoder_cudagraph_token_budgets': [], 'encoder_cudagraph_max_images_per_batch': 0, 'compile_sizes': [], 'compile_ranges_endpoints': [256], 'inductor_compile_config': {'enable_auto_functionalized_v2': False, 'size_asserts': False, 'alignment_asserts': False, 'scalar_asserts': False, 'combo_kernels': True, 'benchmark_combo_kernel': True}, 'inductor_passes': {}, 'cudagraph_mode': <CUDAGraphMode.NONE: 0>, 'cudagraph_num_of_warmups': 0, 'cudagraph_capture_sizes': [], 'cudagraph_copy_inputs': False, 'cudagraph_specialize_lora': True, 'use_inductor_graph_partition': False, 'pass_config': {'fuse_norm_quant': True, 'fuse_act_quant': True, 'fuse_attn_quant': False, 'enable_sp': False, 'fuse_gemm_comms': False, 'fuse_allreduce_rms': False}, 'max_cudagraph_capture_size': 0, 'dynamic_shapes_config': {'type': <DynamicShapesType.BACKED: 'backed'>, 'evaluate_guards': False, 'assume_32_bit_indexing': False}, 'local_cache_dir': None, 'fast_moe_cold_start': True, 'static_all_moe_layers': []}
(EngineCore pid=60466) WARNING 03-28 23:31:31 [interface.py:525] Using 'pin_memory=False' as WSL is detected. This may slow down the performance.
(EngineCore pid=60466) INFO 03-28 23:31:32 [registry.py:126] All limits of multimodal modalities supported by the model are set to 0, running in text-only mode.
(EngineCore pid=60466) INFO 03-28 23:31:32 [parallel_state.py:1400] world_size=1 rank=0 local_rank=0 distributed_init_method=tcp://192.168.31.57:46841 backend=nccl
(EngineCore pid=60466) INFO 03-28 23:31:33 [parallel_state.py:1716] rank 0 in world size 1 is assigned as DP rank 0, PP rank 0, PCP rank 0, TP rank 0, EP rank N/A, EPLB rank N/A
(EngineCore pid=60466) INFO 03-28 23:31:34 [gpu_model_runner.py:4723] Starting to load model Qwen3.5-9B-AWQ-4bit...
(EngineCore pid=60466) INFO 03-28 23:31:34 [rocm.py:540] Using Torch SDPA backend for ViT model.
(EngineCore pid=60466) WARNING 03-28 23:31:34 [activation.py:643] [ROCm] PyTorch's native GELU with tanh approximation is unstable. Falling back to GELU(approximate='none').
(EngineCore pid=60466) INFO 03-28 23:31:34 [mm_encoder_attention.py:230] Using AttentionBackendEnum.TORCH_SDPA for MMEncoderAttention.
(EngineCore pid=60466) INFO 03-28 23:31:34 [compressed_tensors_wNa16.py:112] Using ExllamaLinearKernel for CompressedTensorsWNA16
(EngineCore pid=60466) INFO 03-28 23:31:34 [gdn_linear_attn.py:147] Using Triton/FLA GDN prefill kernel
(EngineCore pid=60466) INFO 03-28 23:31:34 [rocm.py:480] Using ROCM_ATTN attention backend out of potential backends: ['ROCM_ATTN', 'TRITON_ATTN'].
(EngineCore pid=60466) WARNING 03-28 23:31:35 [compilation.py:1220] Op 'sparse_attn_indexer' not present in model, enabling with '+sparse_attn_indexer' has no effect
Loading safetensors checkpoint shards:   0% Completed | 0/2 [00:00<?, ?it/s]
Loading safetensors checkpoint shards:  50% Completed | 1/2 [00:03<00:03,  3.85s/it]
Loading safetensors checkpoint shards: 100% Completed | 2/2 [00:06<00:00,  2.88s/it]
Loading safetensors checkpoint shards: 100% Completed | 2/2 [00:06<00:00,  3.02s/it]
(EngineCore pid=60466)
(EngineCore pid=60466) INFO 03-28 23:31:41 [default_loader.py:384] Loading weights took 6.09 seconds
(EngineCore pid=60466) INFO 03-28 23:31:41 [gpu_model_runner.py:4808] Model loading took 7.81 GiB memory and 6.954194 seconds
(EngineCore pid=60466) INFO 03-28 23:31:46 [gpu_worker.py:436] Available KV cache memory: 0.55 GiB
(EngineCore pid=60466) INFO 03-28 23:31:46 [kv_cache_utils.py:1319] GPU KV cache size: 512 tokens
(EngineCore pid=60466) INFO 03-28 23:31:46 [kv_cache_utils.py:1324] Maximum concurrency for 64 tokens per request: 8.25x
(EngineCore pid=60466) INFO 03-28 23:31:46 [core.py:283] init engine (profile, create kv cache, warmup model) took 5.15 seconds
(EngineCore pid=60466) INFO 03-28 23:31:47 [vllm.py:786] Asynchronous scheduling is enabled.
(EngineCore pid=60466) WARNING 03-28 23:31:47 [vllm.py:844] Enforce eager set, disabling torch.compile and CUDAGraphs. This is equivalent to setting -cc.mode=none -cc.cudagraph_mode=none
WARNING 03-28 23:31:47 [interface.py:525] Using 'pin_memory=False' as WSL is detected. This may slow down the performance.
(EngineCore pid=60466) WARNING 03-28 23:31:47 [vllm.py:855] Inductor compilation was disabled by user settings, optimizations settings that are only active during inductor compilation will be ignored.
(EngineCore pid=60466) INFO 03-28 23:31:47 [vllm.py:1021] Cudagraph is disabled under eager mode
(EngineCore pid=60466) INFO 03-28 23:31:47 [compilation.py:290] Enabled custom fusions: norm_quant, act_quant
Rendering prompts: 100%|█████████████████████████████████████████████████████████████████| 4/4 [00:00<00:00, 444.89it/s]
Processed prompts: 100%|█████████████| 4/4 [00:04<00:00,  1.20s/it, est. speed input: 4.58 toks/s, output: 13.31 toks/s]

Generated Outputs:
------------------------------------------------------------
Prompt:    'Hello, my name is'
Output:    ' Alex. I have a query for you. Could you please explain the meaning of'
------------------------------------------------------------
Prompt:    'The president of the United States is'
Output:    ' the head of state and head of government of the United States. This political office'
------------------------------------------------------------
Prompt:    'The capital of France is'
Output:    ' Paris.\nA. True\nB. False\n\n<think>\n\n</think>\n\nA'
------------------------------------------------------------
Prompt:    'The future of AI is'
Output:    ' not just about smarter models—it’s about systems that can think with purpose, adapt'
------------------------------------------------------------

---

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
• (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[gemini-code-assist]

Code Review

This pull request introduces ROCm platform detection for WSL environments and resolves a circular import issue in the ROCm platform module by replacing warning_once with warning. Feedback suggests refining the ROCm detection logic to ensure it does not activate when vLLM is built for CPU-only, which prevents potential runtime conflicts between detected platforms.

[gemini-code-assist]

This fallback detection for ROCm is a good addition for WSL. However, similar to cuda_platform_plugin, we should avoid activating the ROCm platform if vLLM was built for CPU-only, even if ROCm is available on the system. Otherwise, this can lead to conflicts where both CPU and ROCm platforms are detected as active, causing a runtime error.

Let's add a check for CPU-only builds here. Note that the existing amdsmi detection path also seems to be missing this check, which should probably be fixed in a separate change to ensure consistency.

Suggested change

	if getattr(torch.version, "hip", None):
	is_rocm = True
	logger.debug("Confirmed ROCm platform is available in WSL via torch.version.hip.")
	if getattr(torch.version, "hip", None):
	if not vllm_version_matches_substr("cpu"):
	is_rocm = True
	logger.debug("Confirmed ROCm platform is available in WSL via torch.version.hip.")
	else:
	logger.debug(
	"ROCm platform is not available because vLLM is built with CPU."
	)

[SuibianP]

Possibly related: ROCm/ROCm#5007 (comment)

[MengqingCao]

LGTM, thanks for this fix, @tjtanaa could you take a look at this fix?

[Soluchann]

Hey @yiz-liu ,

Poked around a bit and wanted to flag a couple things in case it helps get this over the line:

The torch.version.hip check alone doesn't confirm a device is actually visible (no is_available() / device_count() check), which is basically the CPU-only-build false-positive risk @gemini-code-assist called out above — probably worth tightening before merge.
This is showing as conflicting against main now and hasn't been updated since June, so it might just need a rebase to get CI running again.
Also heads up, the warning_once → warning part here is kind of moot now — #46516 already dropped that call entirely upstream for the same circular-import reason.

For reference, I put together an extended version of this same idea (device-visibility check, a VLLM_ROCM_GCN_ARCH env override, rocminfo fallback, plus tests) here if it's useful as a reference:

https://github.com/Soluchann/vllm/tree/fix/rocm-wsl-platform-detection