Skip to content

Commit 9d4473d

Browse files
committed
refactor: optimize openvino code
1 parent eee48af commit 9d4473d

14 files changed

Lines changed: 408 additions & 112 deletions

File tree

README.md

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -37,10 +37,9 @@ pip install rapid-layout onnxruntime
3737
### 📋 使用
3838

3939
```python
40-
from rapid_layout import RapidLayout, RapidLayoutInput
40+
from rapid_layout import RapidLayout
4141

42-
cfg = RapidLayoutInput()
43-
layout_engine = RapidLayout(cfg=cfg)
42+
layout_engine = RapidLayout()
4443

4544
img_path = "https://raw.githubusercontent.com/RapidAI/RapidLayout/718b60e927ab893c2fad67c98f753b2105a6f421/tests/test_files/layout.jpg"
4645
results = layout_engine(img_path)

demo.py

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,11 @@
11
# -*- encoding: utf-8 -*-
22
# @Author: SWHL
33
# @Contact: liekkaskono@163.com
4-
from rapid_layout import EngineType, ModelType, RapidLayout, RapidLayoutInput
4+
from rapid_layout import EngineType, ModelType, RapidLayout
55

6-
cfg = RapidLayoutInput()
7-
layout_engine = RapidLayout(cfg=cfg)
6+
layout_engine = RapidLayout(
7+
engine_type=EngineType.OPENVINO, model_type=ModelType.PP_LAYOUT_CDLA
8+
)
89

910
img_path = "https://raw.githubusercontent.com/RapidAI/RapidLayout/718b60e927ab893c2fad67c98f753b2105a6f421/tests/test_files/layout.jpg"
1011
results = layout_engine(img_path)

docs/install_usage/how_to_use_other_engine.md

Lines changed: 33 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -7,54 +7,77 @@ hide:
77

88
## 引言
99

10-
因为版面分析模型输入图像尺寸固定,故可使用`onnxruntime-gpu`来提速。因为`rapid_layout`库默认依赖是CPU版`onnxruntime`,如果想要使用GPU推理,需要手动安装`onnxruntime-gpu`。详细使用和评测可参见[AI Studio](https://aistudio.baidu.com/projectdetail/8094594)
10+
版面分析支持多种推理引擎与设备:
1111

12-
## 使用GPU
12+
- **ONNX Runtime**:默认引擎,支持 CPU / CUDA / DirectML / CANN,需按需安装对应包。
13+
- **OpenVINO**:可选,`pip install openvino` 后通过 `engine_type=EngineType.OPENVINO` 使用。
14+
15+
默认依赖为 CPU 版 `onnxruntime`;使用 GPU 推理需手动安装 `onnxruntime-gpu`。详细使用和评测可参见 [AI Studio](https://aistudio.baidu.com/projectdetail/8094594)
16+
17+
## 使用 ONNX Runtime + GPU (CUDA)
1318

1419
```bash
1520
pip install rapid_layout
16-
1721
# 请确保 onnxruntime-gpu 与当前 GPU/CUDA 版本对应
1822
# 参见 https://onnxruntime.ai/docs/execution-providers/CUDA-ExecutionProvider.html#requirements
1923
pip install onnxruntime-gpu
2024
```
2125

22-
## 使用
23-
2426
```python linenums="1"
2527
from rapid_layout import EngineType, ModelType, RapidLayout, RapidLayoutInput
2628

2729
cfg = RapidLayoutInput(
2830
model_type=ModelType.PP_LAYOUT_CDLA,
2931
engine_type=EngineType.ONNXRUNTIME,
30-
engine_cfg={"use_cuda": True, "cuda_ep_cfg.gpu_id": 1},
32+
engine_cfg={"use_cuda": True, "cuda_ep_cfg": {"device_id": 0}},
3133
)
3234
layout_engine = RapidLayout(cfg=cfg)
3335

3436
img_path = "https://raw.githubusercontent.com/RapidAI/RapidLayout/718b60e927ab893c2fad67c98f753b2105a6f421/tests/test_files/layout.jpg"
3537
results = layout_engine(img_path)
3638
print(results)
39+
results.vis("layout_res.png")
40+
```
41+
42+
多卡时可通过 `cuda_ep_cfg.device_id` 指定卡号(与 [engine_cfg.yaml](https://github.com/RapidAI/RapidLayout/blob/main/rapid_layout/configs/engine_cfg.yaml)`cuda_ep_cfg.device_id` 一致)。
43+
44+
## 使用 OpenVINO
45+
46+
```bash
47+
pip install rapid-layout onnxruntime openvino
48+
```
49+
50+
```python linenums="1"
51+
from rapid_layout import EngineType, ModelType, RapidLayout
3752

53+
layout_engine = RapidLayout(
54+
model_type=ModelType.PP_LAYOUT_CDLA,
55+
engine_type=EngineType.OPENVINO,
56+
)
57+
img_path = "https://raw.githubusercontent.com/RapidAI/RapidLayout/718b60e927ab893c2fad67c98f753b2105a6f421/tests/test_files/layout.jpg"
58+
results = layout_engine(img_path)
59+
print(results)
3860
results.vis("layout_res.png")
3961
```
4062

41-
## 使用NPU
63+
OpenVINO 设备与线程等配置见 [engine_cfg.yaml](https://github.com/RapidAI/RapidLayout/blob/main/rapid_layout/configs/engine_cfg.yaml)`openvino` 段。
4264

43-
详细配置参数参见:[engine_cfg.yaml](https://github.com/RapidAI/RapidLayout/blob/a7ab63ff291bd72e1a98ac2bb11860575514f432/rapid_layout/configs/engine_cfg.yaml)
65+
## 使用 NPU (CANN)
66+
67+
详细配置参数参见:[engine_cfg.yaml](https://github.com/RapidAI/RapidLayout/blob/main/rapid_layout/configs/engine_cfg.yaml)
4468

4569
```python linenums="1"
4670
from rapid_layout import EngineType, ModelType, RapidLayout, RapidLayoutInput
4771

4872
cfg = RapidLayoutInput(
4973
model_type=ModelType.PP_LAYOUT_CDLA,
5074
engine_type=EngineType.ONNXRUNTIME,
51-
engine_cfg={"use_cann": True, "cann_ep_cfg.gpu_id": 0},
75+
engine_cfg={"use_cann": True, "cann_ep_cfg": {"device_id": 0}},
5276
)
5377
layout_engine = RapidLayout(cfg=cfg)
5478

5579
img_path = "https://raw.githubusercontent.com/RapidAI/RapidLayout/718b60e927ab893c2fad67c98f753b2105a6f421/tests/test_files/layout.jpg"
5680
results = layout_engine(img_path)
5781
print(results)
58-
5982
results.vis("layout_res.png")
6083
```

docs/install_usage/usage.md

Lines changed: 48 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -5,28 +5,71 @@ hide:
55
- toc
66
---
77

8-
98
## Python 脚本运行
109

11-
```python
12-
from rapid_layout import EngineType, ModelType, RapidLayout, RapidLayoutInput
10+
**默认用法**(默认模型 `pp_layout_cdla` + `onnxruntime` 引擎):
1311

14-
cfg = RapidLayoutInput()
15-
layout_engine = RapidLayout(cfg=cfg)
12+
```python
13+
from rapid_layout import RapidLayout
1614

15+
layout_engine = RapidLayout()
1716
img_path = "https://raw.githubusercontent.com/RapidAI/RapidLayout/718b60e927ab893c2fad67c98f753b2105a6f421/tests/test_files/layout.jpg"
1817
results = layout_engine(img_path)
1918
print(results)
19+
results.vis("layout_res.png")
20+
```
21+
22+
**指定模型与引擎**(关键字参数):
23+
24+
```python
25+
from rapid_layout import EngineType, ModelType, RapidLayout
26+
27+
layout_engine = RapidLayout(
28+
model_type=ModelType.PP_LAYOUT_CDLA,
29+
engine_type=EngineType.ONNXRUNTIME,
30+
conf_thresh=0.5,
31+
iou_thresh=0.5,
32+
)
33+
results = layout_engine(img_path)
34+
print(results)
35+
results.vis("layout_res.png")
36+
```
37+
38+
**使用配置对象**(与上方等价):
39+
40+
```python
41+
from rapid_layout import EngineType, ModelType, RapidLayout, RapidLayoutInput
2042

43+
cfg = RapidLayoutInput(
44+
model_type=ModelType.PP_LAYOUT_CDLA,
45+
engine_type=EngineType.ONNXRUNTIME,
46+
conf_thresh=0.5,
47+
iou_thresh=0.5,
48+
)
49+
layout_engine = RapidLayout(cfg=cfg)
50+
results = layout_engine(img_path)
51+
print(results)
2152
results.vis("layout_res.png")
2253
```
2354

2455
## 终端运行
2556

2657
```bash
2758
rapid_layout test_images/layout.png
59+
rapid_layout test_images/layout.png -m pp_layout_cdla --conf_thresh 0.5 --iou_thresh 0.5
2860
```
2961

62+
## 构造函数参数(RapidLayout / RapidLayoutInput)
63+
64+
| 参数 | 类型 | 默认值 | 说明 |
65+
|------|------|--------|------|
66+
| `model_type` | ModelType / str | `pp_layout_cdla` | 模型类型 |
67+
| `model_dir_or_path` | str / Path / None | None | 模型路径,不传则按 model_type 解析 |
68+
| `engine_type` | EngineType / str | `onnxruntime` | 推理引擎:`onnxruntime``openvino` |
69+
| `engine_cfg` | dict | `{}` | 引擎额外配置 |
70+
| `conf_thresh` | float | 0.5 | 框置信度阈值 [0, 1] |
71+
| `iou_thresh` | float | 0.5 | IoU 阈值 [0, 1] |
72+
3073
## 可视化结果
3174

3275
<div align="center">

docs/quickstart.md

Lines changed: 55 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -12,29 +12,80 @@ hide:
1212
pip install rapid-layout onnxruntime
1313
```
1414

15+
如需使用 OpenVINO 引擎,请额外安装:`pip install openvino`
16+
1517
## 运行
1618

17-
=== "Python脚本运行"
19+
=== "Python 脚本(默认)"
20+
21+
不传参数时使用默认模型 `pp_layout_cdla` 与 `onnxruntime` 引擎:
1822

1923
```python linenums="1"
20-
from rapid_layout import EngineType, ModelType, RapidLayout, RapidLayoutInput
24+
from rapid_layout import RapidLayout
2125

22-
cfg = RapidLayoutInput()
23-
layout_engine = RapidLayout(cfg=cfg)
26+
layout_engine = RapidLayout()
27+
img_path = "https://raw.githubusercontent.com/RapidAI/RapidLayout/718b60e927ab893c2fad67c98f753b2105a6f421/tests/test_files/layout.jpg"
28+
results = layout_engine(img_path)
29+
print(results)
30+
results.vis("layout_res.png")
31+
```
32+
33+
=== "Python 脚本(指定模型与引擎)"
34+
35+
通过关键字参数指定 `model_type`、`engine_type`、`conf_thresh` 等:
2436

37+
```python linenums="1"
38+
from rapid_layout import EngineType, ModelType, RapidLayout
39+
40+
layout_engine = RapidLayout(
41+
model_type=ModelType.PP_LAYOUT_CDLA,
42+
engine_type=EngineType.ONNXRUNTIME,
43+
conf_thresh=0.5,
44+
)
2545
img_path = "https://raw.githubusercontent.com/RapidAI/RapidLayout/718b60e927ab893c2fad67c98f753b2105a6f421/tests/test_files/layout.jpg"
2646
results = layout_engine(img_path)
2747
print(results)
48+
results.vis("layout_res.png")
49+
```
50+
51+
=== "Python 脚本(使用配置对象)"
2852

53+
```python linenums="1"
54+
from rapid_layout import EngineType, ModelType, RapidLayout, RapidLayoutInput
55+
56+
cfg = RapidLayoutInput(
57+
model_type=ModelType.PP_LAYOUT_CDLA,
58+
engine_type=EngineType.ONNXRUNTIME,
59+
)
60+
layout_engine = RapidLayout(cfg=cfg)
61+
img_path = "https://raw.githubusercontent.com/RapidAI/RapidLayout/718b60e927ab893c2fad67c98f753b2105a6f421/tests/test_files/layout.jpg"
62+
results = layout_engine(img_path)
63+
print(results)
2964
results.vis("layout_res.png")
3065
```
3166

3267
=== "终端运行"
3368

3469
```bash linenums="1"
3570
rapid_layout test_images/layout.png
71+
rapid_layout test_images/layout.png -m pp_layout_cdla --conf_thresh 0.5
3672
```
3773

74+
## 构造函数参数说明
75+
76+
`RapidLayout(cfg=None, **kwargs)` 支持以下关键字参数(与 `RapidLayoutInput` 一致):
77+
78+
| 参数 | 类型 | 默认值 | 说明 |
79+
|------|------|--------|------|
80+
| `model_type` | `ModelType` 或 str | `pp_layout_cdla` | 模型类型,见 [模型列表](models.md) |
81+
| `model_dir_or_path` | str / Path / None | None | 模型路径,不传则按 `model_type` 自动解析 |
82+
| `engine_type` | `EngineType` 或 str | `onnxruntime` | 推理引擎:`onnxruntime``openvino` |
83+
| `engine_cfg` | dict | `{}` | 引擎额外配置,见 [engine_cfg.yaml](https://github.com/RapidAI/RapidLayout/blob/main/rapid_layout/configs/engine_cfg.yaml) |
84+
| `conf_thresh` | float | 0.5 | 框置信度阈值 [0, 1] |
85+
| `iou_thresh` | float | 0.5 | IoU 阈值 [0, 1] |
86+
87+
传入 `cfg` 时,`kwargs` 会覆盖同名字段。
88+
3889
## 可视化结果
3990

4091
<div align="center">

mkdocs.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -146,7 +146,7 @@ markdown_extensions:
146146
- admonition
147147

148148
nav:
149-
- 概览: index.md
149+
# - 概览: index.md
150150
- 快速开始: quickstart.md
151151
- 模型列表: models.md
152152
- 安装及使用:
Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
# -*- encoding: utf-8 -*-
2+
# @Author: SWHL
3+
# @Contact: liekkaskono@163.com
4+
import os
5+
from typing import Any, Dict
6+
7+
from omegaconf import DictConfig
8+
9+
from ...utils.logger import logger
10+
11+
12+
class OpenVINOConfig:
13+
def __init__(self, engine_cfg: DictConfig):
14+
self.cfg = engine_cfg
15+
16+
def get_config(self) -> Dict[str, Any]:
17+
config = {}
18+
19+
infer_num_threads = self.cfg.get("inference_num_threads", -1)
20+
if infer_num_threads != -1 and 1 <= infer_num_threads <= os.cpu_count():
21+
config["INFERENCE_NUM_THREADS"] = str(infer_num_threads)
22+
23+
performance_hint = self.cfg.get("performance_hint", None)
24+
if performance_hint is not None:
25+
config["PERFORMANCE_HINT"] = str(performance_hint)
26+
27+
performance_num_requests = self.cfg.get("performance_num_requests", -1)
28+
if performance_num_requests != -1:
29+
config["PERFORMANCE_HINT_NUM_REQUESTS"] = str(performance_num_requests)
30+
31+
enable_cpu_pinning = self.cfg.get("enable_cpu_pinning", None)
32+
if enable_cpu_pinning is not None:
33+
config["ENABLE_CPU_PINNING"] = str(enable_cpu_pinning)
34+
35+
num_streams = self.cfg.get("num_streams", -1)
36+
if num_streams != -1:
37+
config["NUM_STREAMS"] = str(num_streams)
38+
39+
enable_hyper_threading = self.cfg.get("enable_hyper_threading", None)
40+
if enable_hyper_threading is not None:
41+
config["ENABLE_HYPER_THREADING"] = str(enable_hyper_threading)
42+
43+
scheduling_core_type = self.cfg.get("scheduling_core_type", None)
44+
if scheduling_core_type is not None:
45+
config["SCHEDULING_CORE_TYPE"] = str(scheduling_core_type)
46+
47+
logger.info(f"Using OpenVINO config: {config}")
48+
return config

0 commit comments

Comments
 (0)