Fail to run DeepSeek V3.2 on MI 300x

[LoadingZhang]

docker: atom-dev:sglang-v0.5.10-nightly_20260514

run command:
SGLANG_USE_AITER=1 SGLANG_EXTERNAL_MODEL_PACKAGE=atom.plugin.sglang.models NCCL_DEBUG=WARN python3 -m sglang.launch_server --model-path /root/DeepSeek-V3.2/ --host 0.0.0.0 --port 80 --trust-remote-code --tensor-parallel-size 8 --kv-cache-dtype fp8_e4m3 --mem-fraction-static 0.8 --page-size 1 --disable-radix-cache

error log:

[2026-05-15 05:24:42 TP6] Scheduler hit an exception: Traceback (most recent call last):
  File "/app/sglang/python/sglang/srt/managers/scheduler.py", line 3600, in run_scheduler_process
    scheduler = Scheduler(
                ^^^^^^^^^^
  File "/app/sglang/python/sglang/srt/managers/scheduler.py", line 385, in __init__
    self.init_model_worker()
  File "/app/sglang/python/sglang/srt/managers/scheduler.py", line 632, in init_model_worker
    self.init_tp_model_worker()
  File "/app/sglang/python/sglang/srt/managers/scheduler.py", line 600, in init_tp_model_worker
    self.tp_worker = TpModelWorker(**worker_kwargs)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/sglang/python/sglang/srt/managers/tp_worker.py", line 261, in __init__
    self._init_model_runner()
  File "/app/sglang/python/sglang/srt/managers/tp_worker.py", line 344, in _init_model_runner
    self._model_runner = ModelRunner(
                         ^^^^^^^^^^^^
  File "/app/sglang/python/sglang/srt/model_executor/model_runner.py", line 427, in __init__
    self.initialize(pre_model_load_memory)
  File "/app/sglang/python/sglang/srt/model_executor/model_runner.py", line 507, in initialize
    self.load_model()
  File "/app/sglang/python/sglang/srt/model_executor/model_runner.py", line 1159, in load_model
    self.model = self.loader.load_model(
                 ^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/sglang/python/sglang/srt/model_loader/loader.py", line 683, in load_model
    model = _initialize_model(
            ^^^^^^^^^^^^^^^^^^
  File "/app/sglang/python/sglang/srt/model_loader/loader.py", line 277, in _initialize_model
    return model_class(**kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/app/ATOM/atom/plugin/sglang/models/base_model_wrapper.py", line 330, in __init__
    self.model = atom.prepare_model(config=config, engine="sglang")
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/ATOM/atom/plugin/prepare.py", line 107, in prepare_model
    model = model_cls(atom_config=atom_config)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/ATOM/atom/models/deepseek_v2.py", line 1935, in __init__
    self.model = DeepseekV2Model(
                 ^^^^^^^^^^^^^^^^
  File "/app/ATOM/atom/utils/decorators.py", line 436, in __init__
    old_init(self, atom_config=atom_config, **kwargs)
  File "/app/ATOM/atom/models/deepseek_v2.py", line 1819, in __init__
    self.start_layer, self.end_layer, self.layers = make_layers(
                                                    ^^^^^^^^^^^^
  File "/app/ATOM/atom/models/utils.py", line 135, in make_layers
    layer_fn(prefix=f"{prefix}.{idx}", layer_num=layer_num_offset + idx)
  File "/app/ATOM/atom/models/deepseek_v2.py", line 1821, in <lambda>
    lambda prefix, layer_num=None: DeepseekV2DecoderLayer(
                                   ^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/ATOM/atom/models/deepseek_v2.py", line 1599, in __init__
    self.self_attn = DeepseekV2MLAAttention(
                     ^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/ATOM/atom/models/deepseek_v2.py", line 1424, in __init__
    self.indexer = Indexer(
                   ^^^^^^^^
  File "/app/ATOM/atom/plugin/attention_mla_sparse.py", line 605, in new_init
    orig_init(self, *args, **kwargs)
  File "/app/ATOM/atom/models/deepseek_v2.py", line 1180, in __init__
    self.max_total_seq_len = atom_config.max_num_seqs * self.max_model_len
                             ~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~
TypeError: unsupported operand type(s) for *: 'NoneType' and 'int'

[LoadingZhang]

workaround by adding --max-running-requests N

[LoadingZhang]

vLLM worked.