国产算力平台部署问题反馈

[myhloli]

我们新增了一些国产算力平台的支持，包括以下平台：

• 昇腾（NPU）
• 平头哥（PPU）
• 沐曦（MACA）
• 海光 （DCU）

您可点击链接跳转至文档页查阅各平台的详细部署方案

国产硬件适配是一项复杂且具有挑战性的工作。我们已尽最大努力确保兼容性、功能完整性与运行稳定性，但仍可能存在个别场景下的稳定性问题、兼容性差异或精度对齐偏差。建议您在使用前，参考文档页面中的「红绿灯」状态标识，结合自身需求选择合适的平台与使用场景。

如在使用过程中遇到文档未覆盖的问题，欢迎在本帖中反馈——这将帮助更多用户快速定位和解决问题。
📌 发帖前请注意：
请先浏览已有回复，确认是否已有相同问题的讨论；
若为新问题，请单独开一个楼层发布，便于后续追踪与查阅。

感谢您的理解与支持！

[Xiaochaomeng]

大佬想请教下命令行的npu中vllm框架的红灯是体现在精度上吗，我在用旧版本的mineru后端vlm-vllm-engine调用do_parse已经适配好了会对精度有影响嘛

[Yikun]

@Xiaochaomeng 方便的话，提个issue以及复现用例，我们看下。争取能在0.11.0版本解决啊。

cc @wangxiyuan @gcanlin @shen-shanshan

[myhloli]

@Xiaochaomeng 方便的话，提个issue以及复现用例，我们看下。争取能在0.11.0版本解决啊。

cc @wangxiyuan @gcanlin @shen-shanshan

vllm-project/vllm-ascend#4502

[imwhyyyy]

好的,大佬我现在是修改部分vllm源码使用同步engine已经可以大批量解析了，请问会有什么隐患嘛
请问可以提供下您的源码修改吗

[myhloli]

@Yikun 目前通过预配显存参数跳过了命令行中的显存检测过程，可以在命令行中使用vllm的同步engine了。目前还有一个问题就是在fast-api中，如果先调用pipeline backend解析一本pdf后再调用vllm-engine，又会出现engine初始化失败的情况，目前解决方案是先用vllm-engine解析一本pdf，完成engine初始化后，再切换pipeline就没问题了。这种方法可以凑合着用，但不是很优雅，还是希望贵团队能够修复此问题。🙏

[myhloli]

环境变量指定spwan模式有效，已更新文档

[myhloli]

默认的基础镜像对应的是A2服务器910b卡的，300i可以尝试将基础镜像的tag更换为v0.11.0rc2-310pv0.10.0rc1-310p重新build再试一下

[trappc406-coder]

可以了，感谢 @myhloli

[bandiangou001]

怎么实现Mineru使用多张910B的npu显卡跑啊

[myhloli]

可以使用数据并行功能启动服务

[rade-f]

不好意思，请问910A设备有没有可以尝试的路子可以运行？

[myhloli]

不支持910a

[rade-f]

感谢回复

[zx1259]

请问 可以支持 310P3 的设备进行运行嘛

[myhloli]

基础镜像tag换成v0.11.0rc2-310pv0.10.0rc1-310p即可

[myhloli]

可以参考https://docs.vllm.ai/projects/ascend/en/latest/tutorials/single_node_300i.html

[myhloli]

可以参考https://docs.vllm.ai/projects/ascend/en/latest/tutorials/single_node_300i.html

[liuzeyu4201]

谢谢，尝试了文档的方法。
我尝试了直接拉起vllm server
命令如下：
root@user:/workspace# python -m vllm.entrypoints.openai.api_server
--model /root/.cache/modelscope/hub/models/OpenDataLab/MinerU2.5-2509-1.2B
--tensor-parallel-size 1
--enforce-eager
--dtype half
--block-size 128
--max-model-len 4096
--gpu-memory-utilization 0.3
--host 0.0.0.0
--port 9237
--disable-chunked-prefill
--disable-prefix-caching

不走mineru 也会出现同样的错误如下：
(EngineCore_DP0 pid=33771) WARNING 12-01 07:41:29 [platform.py:275] If chunked prefill or prefix caching is enabled, block size must be set to 128.
Loading safetensors checkpoint shards: 0% Completed | 0/1 [00:00<?, ?it/s]
Loading safetensors checkpoint shards: 100% Completed | 1/1 [00:01<00:00, 1.55s/it]
Loading safetensors checkpoint shards: 100% Completed | 1/1 [00:01<00:00, 1.55s/it]
(EngineCore_DP0 pid=33771)
(EngineCore_DP0 pid=33771) INFO 12-01 07:41:31 [default_loader.py:267] Loading weights took 1.73 seconds
(EngineCore_DP0 pid=33771) INFO 12-01 07:41:32 [model_runner_v1.py:2667] Loading model weights took 2.1678 GB
(EngineCore_DP0 pid=33771) WARNING 12-01 07:41:34 [cudagraph_dispatcher.py:106] cudagraph dispatching keys are not initialized. No cudagraph will be used.
(EngineCore_DP0 pid=33771) INFO 12-01 07:41:35 [worker_v1.py:256] Available memory: 9547198873, total memory: 46381125632
(EngineCore_DP0 pid=33771) INFO 12-01 07:41:35 [kv_cache_utils.py:1087] GPU KV cache size: 776,832 tokens
(EngineCore_DP0 pid=33771) INFO 12-01 07:41:35 [kv_cache_utils.py:1091] Maximum concurrency for 4,096 tokens per request: 189.66x
[rank0]:[W1201 07:41:36.262587355 compiler_depend.ts:62] Warning: Cannot create tensor with NZ format while dim < 2, tensor will be created with ND format. (function operator())
[rank0]:[W1201 07:41:36.573299850 compiler_depend.ts:57] Warning: EH9999: Inner Error!
The error from device(0), serial number is 16. there is a sdma error, sdma channel is 0, the channel exist the following problems: The SMMU returns a Terminate error during page table translation.. the value of CQE status is 2. the description of CQE status: When the SQE translates a page table, the SMMU returns a Terminate error.it's config include: setting1=0xc000000880e0000, setting2=0xff009000ff004c, setting3=0, sq base addr=0x800d00001003d000[FUNC:ProcessSdmaErrorInfo][FILE:device_error_proc.cc][LINE:780]
rtStreamSynchronize execute failed, reason=[sdma copy error][FUNC:FuncErrorReason][FILE:error_message_manage.cc][LINE:53]
EH9999[PID: 33771] 2025-12-01-07:41:36.369.861 (EH9999): synchronize stream failed, runtime result = 507013[FUNC:ReportCallError][FILE:log_inner.cpp][LINE:162]
TraceBack (most recent call last):

DEVICE[0] PID[33771]:
EXCEPTION STREAM:
Exception info:TGID=2652498, model id=65535, stream id=3, stream phase=SCHEDULE
Message info[0]:RTS_HWTS: hwts sdma error, slot_id=3, stream_id=3
Other info[0]:time=2025-12-01-15:41:35.667.162, function=int_process_hwts_sdma_error, line=2070, error code=0x20b (function copy_between_host_and_device_opapi)

[liuzeyu4201]

问题解决了：vllm-ascend:v0.11.0rc2-310p对300Iduo不支持。降级到quay.io/ascend/vllm-ascend:v0.10.0rc1-310p就行了
vllm-project/vllm-ascend#4026

[myhloli]

310p需要在命令行中添加如下两个参数--enforce-eager --dtype float16

[dutianyu]

310p需要在命令行中添加如下两个参数--enforce-eager --dtype float16

及时雨👍️感谢感谢，以下命令的确可行

mineru-gradio --server-name 0.0.0.0 --server-port 9191 --enable-vllm-engine true --dtype float16 --enforce-eager

[Juruobudong]

请问下Atlas 300V Pro有适配吗，有什么启动参数需要注意的

[myhloli]

没有明确支持该型号，已经明确支持的型号是300i duo，您可以参考文档中300i duo的方式尝试部署 https://opendatalab.github.io/MinerU/zh/usage/acceleration_cards/Ascend/

[Juruobudong]

感谢回复@myhloli
请问下是参考这条嘛Atlas 300I Duo适配的版本,请将第三行的tag修改为 v0.10.0rc1-310p

[zerobing]

请问支持 海光 K100 么

[zerobing]

@myhloli 麻烦帮确认下

[myhloli]

目前没有适配

[myhloli]

据vllm-ascend官方文档说明，300i为实验性支持，具体表现可能与其他npu卡（如910b）等不一致，您可以在vllm-ascend项目获得更多信息与帮助。

[ymw123]

vlm-transformers支持吗，使用vllm双卡没有成功，想使用双卡，后端换成vlm-transformers，解析时还是有BFloat16的问题，已经在命令中添加了--dtype float16的。

root@ununtu:/workspace# mineru-gradio --server-name 0.0.0.0 --server-port 7860 --enforce-eager
--dtype float16 --tensor-parallel-size 1 --data-parallel-size 2 --gpu-memory-utilization 0.
6

onnxruntime cpuid_info warning: Unknown CPU vendor. cpuinfo_vendor value: 15

• Running on local URL: http://0.0.0.0:7860
• To create a public link, set share=True in launch().
  2025-12-22 12:14:46.907 | WARNING | mineru.utils.pdf_page_id:get_end_page_id:8 - end_page_id is out of range, use images length
  2025-12-22 12:15:09.287 | INFO | mineru.backend.vlm.utils:set_default_batch_size:92 - gpu_memory: 43 GB, batch_size: 8
  2025-12-22 12:15:09.289 | INFO | mineru.backend.vlm.vlm_analyze:get_model:189 - get transformers predictor cost: 22.32s
  Layout Preparation: 100%|████████████████████████████████████████| 85/85 [00:00<00:00, 333.67it/s]
  Predict: 0%| | 0/85 [00:00<?, ?it/s].[W1222 12:15:19.609799468 compiler_depend.ts:47] Warning: The oprator of ne is executed, Currently High Accuracy but Low Performance OP with 64-bit has been used, Please Do Some Cast at Python Functions with 32-bit for Better Performance! (function operator())
  .[W1222 12:15:29.610186043 compiler_depend.ts:81] Warning: [Check][offset] Check input storage_offset[%ld] = 0 failed, result is untrustworthy239898084 (function operator())
  Predict: 0%| | 0/85 [00:15<?, ?it/s]
  [W1222 12:15:33.042176971 compiler_depend.ts:164] Warning: Device do not support double dtype now, dtype cast replace with float. (function operator())
  ./usr/local/python3.11.13/lib/python3.11/site-packages/torch/_tensor_str.py:397: UserWarning: current tensor is running as_strided, don't perform inplace operations on the returned value. If you encounter this warning and have precision issues, you can try torch.npu.config.allow_internal_format = False to resolve precision issues. (Triggered internally at build/CMakeFiles/torch_npu.dir/compiler_depend.ts:128.)
  start = [self[i] for i in range(0, PRINT_OPTS.edgeitems)]
  2025-12-22 12:15:30.150 | ERROR | mineru.cli.gradio_app:parse_pdf:50 - expected dtype Float but got dtype BFloat16
  [ERROR] 2025-12-22-12:15:30 (PID:37968, Device:0, RankID:-1) ERR01002 OPS invalid type

[myhloli]

transformes不支持vllm的命令行参数，也不支持数据并行，在vllm可用的情况下不推荐使用transformers推理

[ymw123]

文件上传已经快10分钟了，还在Added request这一步，这是正常的吗？为什么这么慢？[DeepWiki]搜索的时候提示使用vllm的时候不用理会“Using a slow image processor as use_fast is unset”，是这样吗？

2025-12-22 15:03:01.650 | WARNING | mineru.utils.pdf_page_id:get_end_page_id:8 - end_page_id is out of range, use images length
Two Step Extraction: 0%| | 0/439 [00:00<?, ?it/s]Using a slow image processor as use_fast is unset and a slow processor was saved with this model. use_fast=True will be the default behavior in v4.52, even if the model was saved with a slow processor. This will result in minor differences in outputs. You'll still be able to use a slow processor with use_fast=False.
INFO 12-22 15:03:32 [async_llm.py:269] Added request e43dc054-1121-4a84-86c8-477d191a95e6.
('Warning: torch.save with "_use_new_zipfile_serialization = False" is not recommended for npu tensor, which may bring unexpected errors and hopefully set "_use_new_zipfile_serialization = True"', 'if it is necessary to use this, please convert the npu tensor to cpu tensor for saving')
INFO 12-22 15:03:32 [async_llm.py:269] Added request 0dc1af3e-bb28-4407-8d59-7a05a337b7a1.
INFO 12-22 15:03:32 [async_llm.py:269] Added request e6f07548-e929-4938-9dd7-8583581a532e.
..INFO 12-22 15:03:32 [async_llm.py:269] Added request 9686c51d-3036-47c8-8603-731b10c29116.
INFO 12-22 15:03:33 [async_llm.py:269] Added request 69a338e5-42ee-4b6c-89fc-a8662693d49d.
INFO 12-22 15:03:33 [async_llm.py:269] Added request 1578b02b-57b8-44da-be27-68bc0b2a2764.
INFO 12-22 15:03:33 [async_llm.py:269] Added request 1061fd01-4672-49ea-a63a-3fcf49abae39.
INFO 12-22 15:03:33 [async_llm.py:269] Added request 2f83d898-64ad-4f0c-8b8b-27b5bc8c9634.

[pengxin99]

请问在昇腾平台上有什么性能优化吗？ @Yikun

[Yikun]

https://opendatalab.github.io/MinerU/zh/usage/acceleration_cards/Ascend/

按这个教程来，其中dockerfile中的版本，可以更新为FROM quay.m.daocloud.io/ascend/vllm-ascend:v0.11.0