V1.17.0-cu129 运行qwen3-next-80B-A3B-Instruct报错——'Qwen3NextConfig' object has no attribute 'dtype'-4张L20

[gaojef]

System Info / 系統信息

ubuntu 22.04
docker

Running Xinference with Docker? / 是否使用 Docker 运行 Xinfernece？

• docker / docker
• pip install / 通过 pip install 安装
• installation from source / 从源码安装

Version info / 版本信息

V1.17.0-cu129

The command used to start Xinference / 用以启动 xinference 的命令

xinference launch --model-engin vllm --model-name Qwen3-Next-Instruct --size-in-billon 80 --quantization none --model-format pytorch --model-path /opt/xxxx/Qwen3-Next-80B-A3B-Instruct --n-gpu 4 --n-worker 1 --max-model-len 3000 --gpu-memory-utilization 0.8

Reproduction / 复现过程

直接加载模型后，运行到一段时间后报错

'Qwen3NextConfig' object has no attribute 'dtype'

比较奇怪的是，我是用vllm的运行引擎，因为transformers版本不够，但是在运行到这个报错之前，调用的方法是

/transformers/configuration_util.py，是在这报出的错误。

我看了看模型的config.json文件，里面是有“torch_type”:"bfloat16"的字段，和别的模型是一样的，但是别的模型能够正常加载，这个就不行。

Expected behavior / 期待表现

能够成功加载模型

[OliverBryant]

能够提供一下日志文件么

[gaojef]

下面是具体日志，谢谢。

^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.12/concurrent/futures/_base.py", line 401, in _get_result
raise self._exception
File "/usr/local/lib/python3.12/dist-packages/xoscar/backends/context.py", line 262, in send
return self._process_result_message(result)
^^^^^^^^^^^^^^^^^^^
File"/usr/local/lib/python3.12/dist-packages/xoscar/backends/context.py", line 111, in _process_result_message
raise message.as_instanceof_cause()
File "/usr/local/lib/python3.12/dist-packages/xoscar/backends/pool.py",line 689,in send
result = await self._run_coro(message.message_id,coro)
^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/xoscar/backends/pool.py", line 389, in _run_coro
return await coro
File "/usr/local/lib/python3.12/dist-packages/xoscar/api.py", line 418, in on_receive
return await super().on_receive(message) # type: ignore
^^^^^^^^^^^^
File"xoscar/core.pyx", line 564, in on_receive
raise ex
File "xoscar/core.pyx", line 526, in xoscar.core._BaseActor.on_receive
async with self._lock:
^^^^^^^^^^^^
File "xoscar/core.pyx", line 527, in xoscar.core._BaseActor.on_receive
with debug_async_timeout('actor_lock_timeout',
^^^^^^^^^^^^^^^^^^^
File "xoscar/core.pyx", line 530, in xoscar.core._BaseActor.on_receive
result= func(*args,**kwargs)
^^^^^^^^^^^^^^^^^^^
File "/opt/inference/xinference/model/llm/vllm/distributed_executor_v1.py", line 81, in execute_method
return getattr(self._worker, method)(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/vllm/v1/worker/gpu_worker.py", line 273, in load_model
self.model_runner.load_model(eep_scale_up=eep_scale_up)
^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/vllm/v1/worker/gpu_model_runner.py", line 3276, in load_model
self.model = model_loader.load_model(
^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/vllm/model_executor/model_loader/base_loader.py", line 49, in load_model
model = initialize_model(
^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/vllm/model_executor/model_loader/utils.py", line 55, in initialize_model
return model_class(vllm_config-vllm_config, prefix=prefix)
^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/vllm/model_executor/models/qwen3_next.py", line 1203, in init
self.model = Qwen3NextModel(
^^^^^^^^^^^^^^^^^^^
File"/usr/local/lib/python3.12/dist-packages/vllm/compilation/decorators.py", line 276, in init
old_init(self,**kwargs)
^^^^^^^^^^^^^^^^^^^
File"/usr/local/lib/python3.12/dist-packages/vllm/model_executor/models/qwen3_next.py", line 989, in init
self.start_layer, self.end_layer, self.layers = make_layers(
^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/vllm/model_executor/models/utils.py", line 651, in make_layers maybe_offload_to_cpu(layer_fn(prefix=f"{prefix}.{idx}"))
^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/vllm/model_executor/models/qwen3_next.py", line 983, in get_layer
return Qwen3NextDecoderLayer(
^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/vllm/model_executor/models/qwen3_next.py", line 838, in init
self.linear_attn = Qwen3NextGatedDeltaNet(
^^^^^^^^^^^^^^^^^^^
File"/usr/local/lib/python3.12/dist-packages/vllm/model_executor/models/qwen3_next.py", line 350, in init
dtype=config.dtype,
^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/transformers/configuration_utils.py", line 211, in getattribute
return super().getattribute(key)
^^^^^^^^^^^^^^^^^^^
AttributeError: [address=0.0.0.0:38939, pid=9237] 'Qwen3NextConfig' object has no attribute 'dtype'