A local change is ready. The test result is as below:

$ rm -rf ~/.cache/tvm-ffi/ && pytest . -v
================================================ test session starts =================================================
platform linux -- Python 3.10.12, pytest-9.0.2, pluggy-1.6.0 -- /opt/venv/bin/python
cachedir: .pytest_cache
hypothesis profile 'default'
rootdir: /opt/venv/lib/python3.10/site-packages/sglang/jit_kernel/tests
plugins: hypothesis-6.150.2, anyio-4.13.0
collected 19108 items / 6 errors

======================================================= ERRORS =======================================================
____________________________ ERROR collecting diffusion/test_diffusion_nvfp4_scaled_mm.py ____________________________
ImportError while importing test module '/opt/venv/lib/python3.10/site-packages/sglang/jit_kernel/tests/diffusion/test_diffusion_nvfp4_scaled_mm.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/lib/python3.10/importlib/__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
diffusion/test_diffusion_nvfp4_scaled_mm.py:3: in <module>
    import flashinfer
E   ModuleNotFoundError: No module named 'flashinfer'
_____________________________ ERROR collecting diffusion/test_fused_norm_scale_shift.py ______________________________
ImportError while importing test module '/opt/venv/lib/python3.10/site-packages/sglang/jit_kernel/tests/diffusion/test_fused_norm_scale_shift.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/lib/python3.10/importlib/__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
diffusion/test_fused_norm_scale_shift.py:9: in <module>
    from sglang.jit_kernel.diffusion.cutedsl.scale_residual_norm_scale_shift import (
../diffusion/cutedsl/scale_residual_norm_scale_shift.py:3: in <module>
    import cuda.bindings.driver as cuda
E   ModuleNotFoundError: No module named 'cuda'
__________________________ ERROR collecting diffusion/test_norm_tanh_mul_add_norm_scale.py ___________________________
ImportError while importing test module '/opt/venv/lib/python3.10/site-packages/sglang/jit_kernel/tests/diffusion/test_norm_tanh_mul_add_norm_scale.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/lib/python3.10/importlib/__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
diffusion/test_norm_tanh_mul_add_norm_scale.py:4: in <module>
    from sglang.jit_kernel.diffusion.cutedsl.norm_tanh_mul_add_norm_scale import (
../diffusion/cutedsl/norm_tanh_mul_add_norm_scale.py:3: in <module>
    import cuda.bindings.driver as cuda
E   ModuleNotFoundError: No module named 'cuda'
______________________________ ERROR collecting diffusion/test_qwen_image_modulation.py ______________________________
ImportError while importing test module '/opt/venv/lib/python3.10/site-packages/sglang/jit_kernel/tests/diffusion/test_qwen_image_modulation.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/lib/python3.10/importlib/__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
diffusion/test_qwen_image_modulation.py:7: in <module>
    from sglang.jit_kernel.diffusion.triton.norm import norm_infer
../diffusion/triton/norm.py:8: in <module>
    from sglang.multimodal_gen.runtime.platforms import current_platform
../../multimodal_gen/__init__.py:2: in <module>
    from sglang.multimodal_gen.configs.pipeline_configs import PipelineConfig
../../multimodal_gen/configs/pipeline_configs/__init__.py:3: in <module>
    from sglang.multimodal_gen.configs.pipeline_configs.base import (
../../multimodal_gen/configs/pipeline_configs/base.py:16: in <module>
    from sglang.multimodal_gen.configs.models import (
../../multimodal_gen/configs/models/__init__.py:4: in <module>
    from sglang.multimodal_gen.configs.models.dits.base import DiTConfig
../../multimodal_gen/configs/models/dits/__init__.py:3: in <module>
    from sglang.multimodal_gen.configs.models.dits.helios import HeliosConfig
../../multimodal_gen/configs/models/dits/helios.py:4: in <module>
    from sglang.multimodal_gen.configs.models.dits.base import DiTArchConfig, DiTConfig
../../multimodal_gen/configs/models/dits/base.py:8: in <module>
    from sglang.multimodal_gen.runtime.layers.quantization import QuantizationConfig
../../multimodal_gen/runtime/layers/quantization/__init__.py:8: in <module>
    from sglang.multimodal_gen.runtime.layers.quantization.fp8 import Fp8Config
../../multimodal_gen/runtime/layers/quantization/fp8.py:33: in <module>
    from sglang.srt.layers.quantization.fp8_kernel import (
../../srt/layers/quantization/__init__.py:41: in <module>
    from sglang.srt.layers.quantization.quark.quark import QuarkConfig
../../srt/layers/quantization/quark/quark.py:18: in <module>
    from sglang.srt.layers.quantization.quark.schemes import (
../../srt/layers/quantization/quark/schemes/__init__.py:4: in <module>
    from .quark_w4a4_mxfp4 import QuarkW4A4MXFP4
../../srt/layers/quantization/quark/schemes/quark_w4a4_mxfp4.py:13: in <module>
    from aiter.ops.triton.gemm.fused.fused_gemm_afp4wfp4_split_cat import (
E   ModuleNotFoundError: No module named 'aiter'
_____________________________________ ERROR collecting test_flash_attention_3.py _____________________________________
test_flash_attention_3.py:470: in <module>
    not is_fa3_supported(),
test_flash_attention_3.py:33: in is_fa3_supported
    return (torch.version.cuda >= "12.3") and (
E   TypeError: '>=' not supported between instances of 'NoneType' and 'str'
_____________________________________ ERROR collecting test_moe_wna16_marlin.py ______________________________________
ImportError while importing test module '/opt/venv/lib/python3.10/site-packages/sglang/jit_kernel/tests/test_moe_wna16_marlin.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/lib/python3.10/importlib/__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
test_moe_wna16_marlin.py:9: in <module>
    from sglang.srt.layers.moe.fused_moe_triton import moe_align_block_size
../../srt/layers/moe/fused_moe_triton/__init__.py:9: in <module>
    from sglang.srt.layers.moe.fused_moe_triton.layer import (
../../srt/layers/moe/fused_moe_triton/layer.py:10: in <module>
    from sglang.srt.batch_overlap.two_batch_overlap import MaybeTboDeepEPDispatcher
../../srt/batch_overlap/two_batch_overlap.py:15: in <module>
    from sglang.srt.batch_overlap.operations_strategy import OperationsStrategy
../../srt/batch_overlap/operations_strategy.py:8: in <module>
    from sglang.srt.layers.moe.token_dispatcher import DeepEPConfig
../../srt/layers/moe/token_dispatcher/__init__.py:11: in <module>
    from sglang.srt.layers.moe.token_dispatcher.deepep import (
../../srt/layers/moe/token_dispatcher/deepep.py:21: in <module>
    from sglang.srt.layers.moe.topk import TopKOutput
../../srt/layers/moe/topk.py:47: in <module>
    from sglang.srt.eplb import expert_location_dispatch
../../srt/eplb/expert_location_dispatch.py:20: in <module>
    from sglang.srt.eplb.expert_location import get_global_expert_location_metadata
../../srt/eplb/expert_location.py:28: in <module>
    from sglang.srt.eplb import eplb_algorithms
../../srt/eplb/eplb_algorithms/__init__.py:6: in <module>
    from sglang.srt.elastic_ep.elastic_ep import ElasticEPStateManager
../../srt/elastic_ep/elastic_ep.py:8: in <module>
    from sglang.srt.managers.schedule_batch import ServerArgs
../../srt/managers/schedule_batch.py:3: in <module>
    from sglang.srt.dllm.config import DllmConfig
../../srt/dllm/config.py:3: in <module>
    from sglang.srt.configs.model_config import ModelConfig
../../srt/configs/model_config.py:27: in <module>
    from sglang.srt.layers.quantization import QUANTIZATION_METHODS
../../srt/layers/quantization/__init__.py:41: in <module>
    from sglang.srt.layers.quantization.quark.quark import QuarkConfig
../../srt/layers/quantization/quark/quark.py:18: in <module>
    from sglang.srt.layers.quantization.quark.schemes import (
../../srt/layers/quantization/quark/schemes/__init__.py:4: in <module>
    from .quark_w4a4_mxfp4 import QuarkW4A4MXFP4
../../srt/layers/quantization/quark/schemes/quark_w4a4_mxfp4.py:13: in <module>
    from aiter.ops.triton.gemm.fused.fused_gemm_afp4wfp4_split_cat import (
E   ModuleNotFoundError: No module named 'aiter'
================================================== warnings summary ==================================================
<frozen importlib._bootstrap>:241
  <frozen importlib._bootstrap>:241: DeprecationWarning: builtin type SwigPyPacked has no __module__ attribute

<frozen importlib._bootstrap>:241
  <frozen importlib._bootstrap>:241: DeprecationWarning: builtin type SwigPyObject has no __module__ attribute

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
============================================== short test summary info ===============================================
ERROR diffusion/test_diffusion_nvfp4_scaled_mm.py
ERROR diffusion/test_fused_norm_scale_shift.py
ERROR diffusion/test_norm_tanh_mul_add_norm_scale.py
ERROR diffusion/test_qwen_image_modulation.py
ERROR test_flash_attention_3.py - TypeError: '>=' not supported between instances of 'NoneType' and 'str'
ERROR test_moe_wna16_marlin.py
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 6 errors during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
=========================================== 2 warnings, 6 errors in 15.92s ===========================================
sys:1: DeprecationWarning: builtin type swigvarlink has no __module__ attribute

Action items:

the 2 No module named 'aiter' should be fixed.
@hubertlu-tw If you have a chance, can you help to comment if the result looks reasonable to you other than the two cases?

[AMD] Support JIT kernel for wheels #19917

Description

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions