File tree Expand file tree Collapse file tree 2 files changed +22
-8
lines changed
docs/pipeline_usage/tutorials/ocr_pipelines Expand file tree Collapse file tree 2 files changed +22
-8
lines changed Original file line number Diff line number Diff line change @@ -287,6 +287,7 @@ comments: true
287287 * 由于我们没有收集NPU和XPU的设备内存数据,因此表中相应位置的数据标记为N/A。
288288
289289## 2. 快速开始
290+
290291PaddleX 所提供的模型产线均可以快速体验效果,你可以在本地使用命令行或 Python 体验通用通用版面解析v3产线的效果。
291292
292293在本地使用通用版面解析v3产线前,请确保您已经按照[ PaddleX本地安装教程] ( ../../../installation/installation.md ) 完成了PaddleX的wheel包安装。如果您希望选择性安装依赖,请参考安装教程中的相关说明。该产线对应的依赖分组为 ` ocr ` 。此外,为了使用飞桨框架读取 safetensors 格式模型,请执行如下命令安装 safetensors:
@@ -907,6 +908,20 @@ docker run \
907908 paddlex_genai_server --model_name PaddleOCR-VL-0.9B --host 0.0.0.0 --port 8118 --backend vllm
908909```
909910
911+ 若您使用的是 NVIDIA 50 系显卡 (Compute Capacity >= 12),需要在启动服务前安装指定版本的 FlashAttention:
912+
913+ ```
914+ docker run \
915+ -it \
916+ --rm \
917+ --gpus all \
918+ --network host \
919+ ccr-2vdh3abv-pub.cnc.bj.baidubce.com/paddlepaddle/paddlex-genai-vllm-server \
920+ /bin/bash
921+ python -m pip install flash-attn==2.8.3
922+ paddlex_genai_server --model_name PaddleOCR-VL-0.9B --backend vllm --port 8118
923+ ```
924+
910925#### 3.1.2 通过 PaddleX CLI 和启动
911926
912927由于推理加速框架可能与飞桨框架存在依赖冲突,建议在虚拟环境中安装。示例如下:
@@ -924,6 +939,12 @@ paddlex --install genai-vllm-server
924939# paddlex --install genai-sglang-server
925940```
926941
942+ 若您使用的是 NVIDIA 50 系显卡 (Compute Capacity >= 12),需要在启动服务前安装指定版本的 flash-attn:
943+
944+ ```
945+ python -m pip install flash-attn==2.8.3
946+ ```
947+
927948安装完成后,可通过 ` paddlex_genai_server ` 命令启动服务:
928949
929950``` bash
Original file line number Diff line number Diff line change @@ -370,14 +370,7 @@ def _install_genai_deps(plugin_types):
370370 if "vllm" in plugin_type or "sglang" in plugin_type :
371371 try :
372372 install_packages (["wheel" ], constraints = "required" )
373- if "vllm" in plugin_type :
374- install_packages (
375- ["flash-attn == 2.8.3" ], constraints = "required"
376- )
377- elif "sglang" in plugin_type :
378- install_packages (
379- ["flash-attn == 2.8.2" ], constraints = "required"
380- )
373+ install_packages (["flash-attn == 2.8.2" ], constraints = "required" )
381374 except Exception :
382375 logging .error ("Installation failed" , exc_info = True )
383376 sys .exit (1 )
You can’t perform that action at this time.
0 commit comments