diff --git a/docs/installation/paddlepaddle_install.en.md b/docs/installation/paddlepaddle_install.en.md index 4e6b693004..e7935fdd9b 100644 --- a/docs/installation/paddlepaddle_install.en.md +++ b/docs/installation/paddlepaddle_install.en.md @@ -46,7 +46,7 @@ nvidia-docker run --name paddlex -v $PWD:/paddle --shm-size=8G --network=host - To use [Paddle Inference TensorRT Subgraph Engine](https://www.paddlepaddle.org.cn/documentation/docs/en/install/pip/linux-pip_en.html#gpu), install TensorRT by executing the following instructions in the 'paddlex' container that has just been started ```bash -python -m pip install /usr/local/TensorRT-8.6.1.6/python/tensorrt-8.6.1-cp310-none-linux_x86_64.whl +python -m pip install /usr/local/TensorRT-*/python/tensorrt-*-cp310-none-linux_x86_64.whl ``` ## Installing PaddlePaddle via pip @@ -94,7 +94,7 @@ tar xvf TensorRT-8.6.1.6.Linux.x86_64-gnu.cuda-11.8.tar.gz # Install TensorRT wheel package python -m pip install TensorRT-8.6.1.6/python/tensorrt-8.6.1-cp310-none-linux_x86_64.whl # Add the absolute path of TensorRT's `lib` directory to LD_LIBRARY_PATH -export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:TensorRT-8.6.1.6/lib +export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:TensorRT-8.6.1.6/lib" ``` > ❗ Note: If you encounter any issues during the installation process, feel free to [submit an issue](https://github.com/PaddlePaddle/Paddle/issues) in the Paddle repository. diff --git a/docs/installation/paddlepaddle_install.md b/docs/installation/paddlepaddle_install.md index 06aeb190f1..dd16cead5f 100644 --- a/docs/installation/paddlepaddle_install.md +++ b/docs/installation/paddlepaddle_install.md @@ -47,7 +47,7 @@ nvidia-docker run --name paddlex -v $PWD:/paddle --shm-size=8G --network=host -i 在刚刚启动的 `paddlex` 容器中执行下面指令安装 TensorRT，即可使用 [Paddle Inference TensorRT 子图引擎](https://www.paddlepaddle.org.cn/documentation/docs/zh/guides/paddle_v3_features/paddle_trt_cn.html)： ```bash -python -m pip install /usr/local/TensorRT-8.6.1.6/python/tensorrt-8.6.1-cp310-none-linux_x86_64.whl +python -m pip install /usr/local/TensorRT-*/python/tensorrt-*-cp310-none-linux_x86_64.whl ``` ## 基于 pip 安装飞桨 @@ -94,7 +94,7 @@ tar xvf TensorRT-8.6.1.6.Linux.x86_64-gnu.cuda-11.8.tar.gz # 安装 TensorRT wheel 包 python -m pip install TensorRT-8.6.1.6/python/tensorrt-8.6.1-cp310-none-linux_x86_64.whl # 添加 TensorRT 的 `lib` 目录的绝对路径到 LD_LIBRARY_PATH 中 -export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:TensorRT-8.6.1.6/lib +export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:TensorRT-8.6.1.6/lib" ``` > ❗ 注：如果在安装的过程中，出现任何问题，欢迎在Paddle仓库中[提Issue](https://github.com/PaddlePaddle/Paddle/issues)。 diff --git a/docs/module_usage/instructions/model_python_API.en.md b/docs/module_usage/instructions/model_python_API.en.md index f99d8789e4..71cc956bf1 100644 --- a/docs/module_usage/instructions/model_python_API.en.md +++ b/docs/module_usage/instructions/model_python_API.en.md @@ -39,7 +39,6 @@ In short, just three steps: * `use_hpip`：`bool` type, whether to enable the high-performance inference plugin; * `hpi_config`：`dict | None` type, high-performance inference configuration; * _`inference hyperparameters`_: used to set common inference hyperparameters. Please refer to specific model description document for details. - * Return Value: `BasePredictor` type. ### 2. Perform Inference Prediction by Calling the `predict()` Method of the Prediction Model Object diff --git a/docs/module_usage/instructions/model_python_API.md b/docs/module_usage/instructions/model_python_API.md index 238ab6c0ce..9cfa1b3377 100644 --- a/docs/module_usage/instructions/model_python_API.md +++ b/docs/module_usage/instructions/model_python_API.md @@ -40,7 +40,6 @@ for res in output: * `use_hpip`：`bool` 类型，是否启用高性能推理插件； * `hpi_config`：`dict | None` 类型，高性能推理配置； * _`推理超参数`_：支持常见推理超参数的修改，具体参数说明详见具体模型文档； - * 返回值：`BasePredictor` 类型。 ### 2. 调用预测模型对象的`predict()`方法进行推理预测 diff --git a/docs/pipeline_deploy/edge_deploy.en.md b/docs/pipeline_deploy/edge_deploy.en.md index aaf9e1730b..564accfe4f 100644 --- a/docs/pipeline_deploy/edge_deploy.en.md +++ b/docs/pipeline_deploy/edge_deploy.en.md @@ -190,7 +190,7 @@ This guide applies to 8 models across 6 modules: Note: - `{Pipeline_Name}` and `{Demo_Name}` are placeholders. Refer to the table at the end of this section for specific values. - `download.sh` and `run.sh` support passing in model names to specify models. If not specified, the default model will be used. Refer to the `Model_Name` column in the table at the end of this section for currently supported models. - - To use your own trained model, refer to the [Model Conversion Method](https://paddlepaddle.github.io/Paddle-Lite/develop/model_optimize_tool/) to obtain the `.nb` model, place it in the `PaddleX_Lite_Deploy/{Pipeline_Name}/assets/{Model_Name}` directory, where `{Model_Name}` is the model name, e.g., `PaddleX_Lite_Deploy/object_detection/assets/PicoDet-L`. + - To use your own trained model, refer to the [Model Conversion Method](https://paddlepaddle.github.io/Paddle-Lite/develop/model_optimize_tool/) to obtain the `.nb` model, place it in the `PaddleX_Lite_Deploy/{Pipeline_Name}/assets/{Model_Name}` directory, where `{Model_Name}` is the model name, e.g., `PaddleX_Lite_Deploy/object_detection/assets/PicoDet-L`. Please note that converting static graph models in `.json` format to `.nb` format is currently not supported. When exporting a static graph model using PaddleX, please set the environment variable `FLAGS_json_format_model` to `0`. - Before running the `build.sh` script, change the path specified by `NDK_ROOT` to the actual installed NDK path. - Keep ADB connected when running the `build.sh` script. - On Windows systems, you can use Git Bash to execute the deployment steps. @@ -305,6 +305,7 @@ This section describes the deployment steps applicable to the demos listed in th Note + - Currently, there is no demo for deploying the Layout Area Detection module on the edge, so the `picodet_detection` demo is reused to deploy the `PicoDet_layout_1x` model. ## Reference Materials diff --git a/docs/pipeline_deploy/edge_deploy.md b/docs/pipeline_deploy/edge_deploy.md index 26dc0b290d..4ffc556b27 100644 --- a/docs/pipeline_deploy/edge_deploy.md +++ b/docs/pipeline_deploy/edge_deploy.md @@ -190,7 +190,7 @@ comments: true 注意： - `Pipeline_Name` 和 `Demo_Name` 为占位符，具体值可参考本节最后的表格。 - `download.sh` 和 `run.sh` 支持传入模型名来指定模型，若不指定则使用默认模型。目前适配的模型可参考本节最后表格的 `Model_Name` 列。 - - 若想使用自己训练的模型，参考 [模型转换方法](https://paddlepaddle.github.io/Paddle-Lite/develop/model_optimize_tool/) 得到 `.nb` 模型，放到`PaddleX_Lite_Deploy/{Pipeline_Name}/assets/{Model_Name}`目录下， `Model_Name`为模型名，例如 `PaddleX_Lite_Deploy/object_detection/assets/PicoDet-L`。 + - 若想使用自己训练的模型，参考 [模型转换方法](https://paddlepaddle.github.io/Paddle-Lite/develop/model_optimize_tool/) 得到 `.nb` 模型，放到`PaddleX_Lite_Deploy/{Pipeline_Name}/assets/{Model_Name}`目录下， `Model_Name`为模型名，例如 `PaddleX_Lite_Deploy/object_detection/assets/PicoDet-L`。请注意，目前暂不支持将 `.json` 格式的静态图模型转换为 `.nb` 格式。在使用 PaddleX 导出静态图模型时，请设置环境变量 `FLAGS_json_format_model` 为 `0`。 - 在运行 `build.sh` 脚本前，需要更改 `NDK_ROOT` 指定的路径为实际安装的 NDK 路径。 - 在运行 `build.sh` 脚本时需保持 ADB 连接。 - 在 Windows 系统上可以使用 Git Bash 执行部署步骤。 @@ -307,7 +307,8 @@ detection, image size: 768, 576, detect object: dog, score: 0.731584, location: 备注 -- 目前没有版面区域检测模块的端侧部署 demo，因此复用 `picodet_detection`demo 来部署`PicoDet_layout_1x`模型。 + +- 目前没有版面区域检测模块的端侧部署 demo，因此复用 `picodet_detection` demo 来部署 `PicoDet_layout_1x` 模型。 ## 参考资料 diff --git a/docs/pipeline_deploy/high_performance_inference.en.md b/docs/pipeline_deploy/high_performance_inference.en.md index bc4e510fa0..1b082224d9 100644 --- a/docs/pipeline_deploy/high_performance_inference.en.md +++ b/docs/pipeline_deploy/high_performance_inference.en.md @@ -4,7 +4,7 @@ comments: true # PaddleX High-Performance Inference Guide -In real production environments, many applications impose strict performance metrics—especially in response time—on deployment strategies to ensure system efficiency and a smooth user experience. To address this, PaddleX offers a high-performance inference plugin that, through automatic configuration and multi-backend inference capabilities, enables users to significantly accelerate model inference without concerning themselves with complex configurations and low-level details. +In real production environments, many applications impose strict performance metrics—especially in response time—on deployment strategies to ensure system efficiency and a smooth user experience. To address this, PaddleX offers a high-performance inference plugin that, through automatic configuration and multi-backend inference capabilities, enables users to significantly accelerate model inference without concerning themselves with complex configurations and low-level details. In addition to supporting inference acceleration on pipelines, the PaddleX high-performance inference plugin can also be used to accelerate inference when modules are used standalone. ## Table of Contents @@ -24,7 +24,7 @@ In real production environments, many applications impose strict performance met Before using the high-performance inference plugin, please ensure that you have completed the PaddleX installation according to the [PaddleX Local Installation Tutorial](../installation/installation.en.md) and have run the quick inference using the PaddleX pipeline command line or the PaddleX pipeline Python script as described in the usage instructions. -The high-performance inference plugin supports handling multiple model formats, including **PaddlePaddle static graph (`.pdmodel`, `.json`)**, **ONNX (`.onnx`)** and **Huawei OM (`.om`)**, among others. For ONNX models, you can convert them using the [Paddle2ONNX Plugin](./paddle2onnx.en.md). If multiple model formats are present in the model directory, PaddleX will automatically choose the appropriate one as needed, and automatic model conversion may be performed. **It is recommended to install the Paddle2ONNX plugin first before installing the high-performance inference plugin, so that PaddleX can convert model formats when needed.** +The high-performance inference plugin supports handling multiple model formats, including **PaddlePaddle static graph (`.pdmodel`, `.json`)**, **ONNX (`.onnx`)** and **Huawei OM (`.om`)**, among others. For ONNX models, you can convert them using the [Paddle2ONNX Plugin](./paddle2onnx.en.md). If multiple model formats are present in the model directory, PaddleX will automatically choose the appropriate one as needed, and automatic model conversion may be performed. ### 1.1 Installing the High-Performance Inference Plugin @@ -86,12 +86,14 @@ Refer to [Get PaddleX based on Docker](../installation/installation.en.md#21-obt -In the official PaddleX Docker image, TensorRT is installed by default. The high-performance inference plugin can then accelerate inference using the Paddle Inference TensorRT subgraph engine. +The official PaddleX Docker images come with the Paddle2ONNX plugin pre-installed, allowing PaddleX to convert model formats on demand. In addition, the GPU version of the image includes TensorRT, so the high-performance inference plugin can leverage the Paddle Inference TensorRT subgraph engine for accelerated inference. **Please note that the aforementioned Docker image refers to the official PaddleX image described in [Get PaddleX via Docker](../installation/installation.en.md#21-get-paddlex-based-on-docker), rather than the PaddlePaddle official image described in [PaddlePaddle Local Installation Tutorial](../installation/paddlepaddle_install.en.md#installing-paddlepaddle-via-docker). For the latter, please refer to the local installation instructions for the high-performance inference plugin.** #### 1.1.2 Installing the High-Performance Inference Plugin Locally +**It is recommended to install the Paddle2ONNX plugin first before installing the high-performance inference plugin, so that PaddleX can convert model formats when needed.** + **To install the CPU version of the high-performance inference plugin:** Run: @@ -322,7 +324,7 @@ The available configuration items for `backend_config` vary for different backen ### 2.3 Modifying the High-Performance Inference Configuration -Due to the diversity of actual deployment environments and requirements, the default configuration might not meet all needs. In such cases, manual adjustment of the high-performance inference configuration may be necessary. Users can modify the configuration by editing the **pipeline/module configuration file** or by passing the `hpi_config` field in the parameters via **CLI** or **Python API**. **Parameters passed via CLI or Python API will override the settings in the pipeline/module configuration file.** Different levels of configurations in the config file are automatically merged, and the deepest-level settings take the highest priority. The following examples illustrate how to modify the configuration. +When the model is initialized, the log will, by default, record the high-performance inference configuration that is about to be used. Due to the diversity of actual deployment environments and requirements, the default configuration might not meet all needs. In such cases, manual adjustment of the high-performance inference configuration may be necessary. Users can modify the configuration by editing the pipeline/module configuration file or by passing the `hpi_config` field in the parameters via CLI or Python API. Parameters passed via CLI or Python API will override the settings in the pipeline/module configuration file. Different levels of configurations in the config file are automatically merged, and the deepest-level settings take the highest priority. The following examples illustrate how to modify the configuration. **For the general OCR pipeline, use the `onnxruntime` backend for all models:** @@ -566,3 +568,11 @@ For the GPU version of the high-performance inference plugin, the official Paddl **4. Why does the program freeze during runtime or display some "WARNING" and "ERROR" messages after using the high-performance inference feature? What should be done in such cases?** When initializing the model, operations such as subgraph optimization may take longer and may generate some "WARNING" and "ERROR" messages. However, as long as the program does not exit automatically, it is recommended to wait patiently, as the program usually continues to run to completion. + +**5. When using GPU for inference, enabling the high-performance inference plugin increases memory usage and causes OOM. How can this be resolved?** + +Some acceleration methods trade off memory usage to support a broader range of inference scenarios. If memory becomes a bottleneck, consider the following optimization strategies: + +* **Adjust pipeline configurations**: Disable unnecessary features to avoid loading redundant models. Appropriately reduce the batch size based on business requirements to balance throughput and memory usage. +* **Switch inference backends**: Different inference backends have varying memory management strategies. Try benchmarking various backends to compare memory usage and performance. +* **Optimize dynamic shape configurations**: For modules using TensorRT or Paddle Inference TensorRT subgraph engine, narrow the dynamic shape range based on the actual distribution of input data. diff --git a/docs/pipeline_deploy/high_performance_inference.md b/docs/pipeline_deploy/high_performance_inference.md index 4ee284a274..d7757a8913 100644 --- a/docs/pipeline_deploy/high_performance_inference.md +++ b/docs/pipeline_deploy/high_performance_inference.md @@ -4,7 +4,7 @@ comments: true # PaddleX 高性能推理指南 -在实际生产环境中，许多应用对部署策略的性能指标（尤其是响应速度）有着较严苛的标准，以确保系统的高效运行与用户体验的流畅性。为此，PaddleX 提供高性能推理插件，通过自动配置和多后端推理功能，让用户无需关注复杂的配置和底层细节，即可显著提升模型的推理速度。 +在实际生产环境中，许多应用对部署策略的性能指标（尤其是响应速度）有着较严苛的标准，以确保系统的高效运行与用户体验的流畅性。为此，PaddleX 提供高性能推理插件，通过自动配置和多后端推理功能，让用户无需关注复杂的配置和底层细节，即可显著提升模型的推理速度。除了支持产线的推理加速外，PaddleX 高性能推理插件也可用于单独使用模块时的推理加速。 ## 目录 @@ -22,9 +22,9 @@ comments: true ## 1. 安装与基础使用方法 -使用高性能推理插件前，请确保您已经按照 [PaddleX本地安装教程](../installation/installation.md) 完成了PaddleX的安装，且按照PaddleX产线命令行使用说明或PaddleX产线Python脚本使用说明跑通了产线的快速推理。 +使用高性能推理插件前，请确保您已经按照 [PaddleX本地安装教程](../installation/installation.md) 完成了 PaddleX 的安装，且按照 PaddleX 产线命令行使用说明或 PaddleX 产线 Python 脚本使用说明跑通了产线的快速推理。 -高性能推理插件支持处理 **PaddlePaddle 静态图（`.pdmodel`、 `.json`）**、**ONNX（`.onnx`）**、**华为 OM（`.om`）** 等多种模型格式。对于 ONNX 模型，可以使用 [Paddle2ONNX 插件](./paddle2onnx.md) 转换得到。如果模型目录中存在多种格式的模型，PaddleX 会根据需要自动选择，并可能进行自动模型转换。**建议在安装高性能推理插件前，首先安装 Paddle2ONNX 插件，以便 PaddleX 可以在需要时转换模型格式。** +高性能推理插件支持处理 **飞桨静态图（`.pdmodel`、 `.json`）**、**ONNX（`.onnx`）**、**华为 OM（`.om`）** 等多种模型格式。对于 ONNX 模型，可以使用 [Paddle2ONNX 插件](./paddle2onnx.md) 转换得到。如果模型目录中存在多种格式的模型，PaddleX 会根据需要自动选择，并可能进行自动模型转换。 ### 1.1 安装高性能推理插件 @@ -86,12 +86,14 @@ comments: true -PaddleX 官方 Docker 镜像中默认安装了 TensorRT，高性能推理插件可以使用 Paddle Inference TensorRT 子图引擎进行推理加速。 +PaddleX 官方 Docker 镜像中预装了 Paddle2ONNX 插件，以便 PaddleX 可以在需要时转换模型格式。此外，GPU 版本的镜像中安装了 TensorRT，高性能推理插件可以使用 Paddle Inference TensorRT 子图引擎进行推理加速。 **请注意，以上提到的镜像指的是 [基于Docker获取PaddleX](../installation/installation.md#21-基于docker获取paddlex) 中描述的 PaddleX 官方镜像，而非 [飞桨PaddlePaddle本地安装教程](../installation/paddlepaddle_install.md#基于-docker-安装飞桨) 中描述的飞桨框架官方镜像。对于后者，请参考高性能推理插件本地安装说明。** #### 1.1.2 本地安装高性能推理插件 +**建议在安装高性能推理插件前，首先安装 Paddle2ONNX 插件，以便 PaddleX 可以在需要时转换模型格式。** + **安装 CPU 版本的高性能推理插件：** 执行： @@ -107,7 +109,7 @@ paddlex --install hpi-cpu - [安装 CUDA 11.8](https://developer.nvidia.com/cuda-11-8-0-download-archive) - [安装 cuDNN 8.9](https://docs.nvidia.com/deeplearning/cudnn/archives/cudnn-890/install-guide/index.html) -如果使用的是飞桨框架官方镜像，则镜像中的 CUDA 和 cuDNN 版本已经是满足要求的，无需重新安装。 +如果使用的是飞桨框架官方镜像，则镜像中的 CUDA 和 cuDNN 版本已经是满足要求的，无需额外安装。 如果通过 pip 安装飞桨，通常 CUDA、cuDNN 的相关 Python 包将被自动安装。在这种情况下，**仍需要通过安装非 Python 专用的 CUDA 与 cuDNN**。同时，建议安装的 CUDA 和 cuDNN 版本与环境中存在的 Python 包版本保持一致，以避免不同版本的库共存导致的潜在问题。可以通过如下方式可以查看 CUDA 和 cuDNN 相关 Python 包的版本： @@ -132,7 +134,7 @@ paddlex --install hpi-gpu **注意：** -1. **目前 PaddleX 官方仅提供 CUDA 11.8 + cuDNN 8.9 的预编译包**；CUDA 12 已经在支持中。 +1. **目前 PaddleX 官方仅提供 CUDA 11.8 + cuDNN 8.9 的预编译包**。CUDA 12 已经在支持中。 2. 同一环境中只应该存在一个版本的高性能推理插件。 @@ -246,7 +248,7 @@ output = model.predict("https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/ auto_paddle2onnx - 是否将 PaddlePaddle 静态图模型自动转换为 ONNX 模型。当 Paddle2ONNX 插件不可用时，不执行转换。 + 是否将飞桨静态图模型自动转换为 ONNX 模型。当 Paddle2ONNX 插件不可用时，不执行转换。 bool True @@ -323,7 +325,7 @@ output = model.predict("https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/ ### 2.3 修改高性能推理配置 -由于实际部署环境和需求的多样性，默认配置可能无法满足所有要求。这时，可能需要手动调整高性能推理配置。用户可以通过修改**产线/模块配置文件**、**CLI**或**Python API**所传递参数中的 `hpi_config` 字段内容来修改配置。**通过 CLI 或 Python API 传递的参数将覆盖产线/模块配置文件中的设置**。配置文件中不同层级的配置将自动合并，最深层的配置具有最高的优先级。以下将结合一些例子介绍如何修改配置。 +模型初始化时，日志中默认会记录将要使用的高性能推理配置。由于实际部署环境和需求的多样性，默认配置可能无法满足所有要求。这时，可能需要手动调整高性能推理配置。用户可以通过修改产线/模块配置文件、CLI或Python API所传递参数中的 `hpi_config` 字段内容来修改配置。通过 CLI 或 Python API 传递的参数将覆盖产线/模块配置文件中的设置。配置文件中不同层级的配置将自动合并，最深层的配置具有最高的优先级。以下将结合一些例子介绍如何修改配置。 **通用OCR产线的所有模型使用 `onnxruntime` 后端：** @@ -566,3 +568,11 @@ python -m pip install ../../python/dist/ultra_infer*.whl **4. 为什么使用高性能推理功能后，程序在运行过程中会卡住或者显示一些“WARNING”和“ERROR”信息？这种情况下应该如何处理？** 在初始化模型时，子图优化等操作可能会导致程序耗时较长，并生成一些“WARNING”和“ERROR”信息。然而，只要程序没有自动退出，建议耐心等待，程序通常会继续运行至完成。 + +**5. 使用 GPU 推理时，启用高性能推理插件后显存占用增大并导致 OOM，如何解决？** + +部分提速手段会以牺牲显存为代价，以支持更广泛的推理场景。如果显存成为瓶颈，可参考以下优化思路： + +- 调整产线配置：禁用不需要使用的功能，避免加载多余模型；根据业务需求合理降低 batch size，平衡吞吐与显存使用。 +- 切换推理后端：不同推理后端在显存管理策略上各有差异，可尝试各种后端测评显存占用与性能。 +- 优化动态形状配置：对于使用 TensorRT 或 Paddle Inference TensorRT 子图引擎的模块，根据实际输入数据的分布，缩小动态形状范围。 diff --git a/docs/pipeline_deploy/paddle2onnx.en.md b/docs/pipeline_deploy/paddle2onnx.en.md index 3ceb9ad7ad..11291833f6 100644 --- a/docs/pipeline_deploy/paddle2onnx.en.md +++ b/docs/pipeline_deploy/paddle2onnx.en.md @@ -34,7 +34,7 @@ paddlex --install paddle2onnx opset_version int - The ONNX opset version to use. Defaults to 7. + The ONNX opset version to use. If a lower-version opset cannot complete the conversion, a higher-version opset will be automatically selected for the conversion. Defaults to 7. diff --git a/docs/pipeline_deploy/paddle2onnx.md b/docs/pipeline_deploy/paddle2onnx.md index 3ec5fd042b..84255f15e3 100644 --- a/docs/pipeline_deploy/paddle2onnx.md +++ b/docs/pipeline_deploy/paddle2onnx.md @@ -1,7 +1,7 @@ # Paddle2ONNX 插件的安装与使用 -PaddleX 的 Paddle2ONNX 插件提供了将 PaddlePaddle 静态图模型转化到 ONNX 格式模型的能力，底层使用[Paddle2ONNX](https://github.com/PaddlePaddle/Paddle2ONNX)。 +PaddleX 的 Paddle2ONNX 插件提供了将飞桨静态图模型转化到 ONNX 格式模型的能力，底层使用 [Paddle2ONNX](https://github.com/PaddlePaddle/Paddle2ONNX)。 ## 1. 安装 @@ -25,17 +25,17 @@ paddlex --install paddle2onnx paddle_model_dir str - 包含Paddle模型的目录。 + 包含 Paddle 模型的目录。 onnx_model_dir str - ONNX模型的输出目录，可以与Paddle模型目录相同。默认为onnx。 + ONNX 模型的输出目录，可以与 Paddle 模型目录相同。默认为 onnx。 opset_version int - 使用的ONNX opset版本。默认为7。 + 使用的 ONNX opset 版本。当使用低版本 opset 无法完成转换时，将自动选择更高版本的 opset 进行转换。默认为 7。 @@ -47,9 +47,9 @@ paddlex --install paddle2onnx ```bash paddlex \ --paddle2onnx \ # 使用paddle2onnx功能 - --paddle_model_dir /your/paddle_model/dir \ # 指定Paddle模型所在的目录 - --onnx_model_dir /your/onnx_model/output/dir \ # 指定转换后ONNX模型的输出目录 - --opset_version 7 # 指定要使用的ONNX opset版本 + --paddle_model_dir /your/paddle_model/dir \ # 指定 Paddle 模型所在的目录 + --onnx_model_dir /your/onnx_model/output/dir \ # 指定转换后 ONNX 模型的输出目录 + --opset_version 7 # 指定要使用的 ONNX opset 版本 ``` 以 image_classification 模块中的 ResNet18 模型为例： diff --git a/docs/pipeline_deploy/serving.en.md b/docs/pipeline_deploy/serving.en.md index 730fc378fb..a20dd474dc 100644 --- a/docs/pipeline_deploy/serving.en.md +++ b/docs/pipeline_deploy/serving.en.md @@ -76,11 +76,11 @@ The command-line options related to serving are as follows: --host -Hostname or IP address the server binds to. Defaults to `0.0.0.0`. +Hostname or IP address the server binds to. Defaults to 0.0.0.0. --port -Port number the server listens on. Defaults to `8080`. +Port number the server listens on. Defaults to 8080. --use_hpip @@ -323,8 +323,8 @@ With the image prepared, navigate to the `server` directory and execute the foll docker run \ -it \ -e PADDLEX_HPS_DEVICE_TYPE={deployment device type} \ - -v "$(pwd)":/workspace \ - -w /workspace \ + -v "$(pwd)":/app \ + -w /app \ --rm \ --gpus all \ --init \ diff --git a/docs/pipeline_deploy/serving.md b/docs/pipeline_deploy/serving.md index e1900f459d..28cca2c63f 100644 --- a/docs/pipeline_deploy/serving.md +++ b/docs/pipeline_deploy/serving.md @@ -76,11 +76,11 @@ INFO: Uvicorn running on http://0.0.0.0:8080 (Press CTRL+C to quit) --host -服务器绑定的主机名或 IP 地址。默认为 `0.0.0.0`。 +服务器绑定的主机名或 IP 地址。默认为 0.0.0.0。 --port -服务器监听的端口号。默认为 `8080`。 +服务器监听的端口号。默认为 8080。 --use_hpip @@ -323,8 +323,8 @@ paddlex --serve --pipeline image_classification --use_hpip docker run \ -it \ -e PADDLEX_HPS_DEVICE_TYPE={部署设备类型} \ - -v "$(pwd)":/workspace \ - -w /workspace \ + -v "$(pwd)":/app \ + -w /app \ --rm \ --gpus all \ --init \ diff --git a/docs/pipeline_usage/instructions/parallel_inference.en.md b/docs/pipeline_usage/instructions/parallel_inference.en.md new file mode 100644 index 0000000000..2e2231865f --- /dev/null +++ b/docs/pipeline_usage/instructions/parallel_inference.en.md @@ -0,0 +1,196 @@ +# Pipeline Parallel Inference + +## Specifying Multiple Inference Devices + +For some pipelines in both the CLI and Python API, PaddleX supports specifying multiple inference devices simultaneously. If multiple devices are specified, at initialization each device will host its own instance of the underlying pipeline class, and incoming inputs will be inferred in parallel across them. For example, for the PP-StructureV3 pipeline: + +```bash +paddlex --pipeline PP-StructureV3 \ + --input input_images/ \ + --use_doc_orientation_classify False \ + --use_doc_unwarping False \ + --use_textline_orientation False \ + --save_path ./output \ + --device 'gpu:0,1,2,3' +``` + +```python +pipeline = create_pipeline(pipeline="PP-StructureV3", device="gpu:0,1,2,3") +output = pipeline.predict( + input="input_images/", + use_doc_orientation_classify=False, + use_doc_unwarping=False, + use_textline_orientation=False, +) +``` + +In both examples above, four GPUs (IDs 0, 1, 2, 3) are used to perform parallel inference on all files in the `input_images` directory. + +When specifying multiple devices, the inference interface remains the same as when specifying a single device. Please refer to the pipeline usage guide to check whether a given pipeline supports multiple-device inference. + +## Example of Multi-Process Parallel Inference + +Beyond PaddleX’s built-in multi-GPU parallel inference, users can also implement parallelism by wrapping PaddleX pipeline API calls themselves according to their specific scenario, with a view to achieving a better speedup. Below is an example of using Python’s `multiprocessing` to run multiple cards and multiple pipeline instances in parallel over the files in an input directory: + +```python +import argparse +import sys +from multiprocessing import Manager, Process +from pathlib import Path +from queue import Empty + +from paddlex import create_pipeline +from paddlex.utils.device import constr_device, parse_device + + +def worker(pipeline_name_or_config_path, device, task_queue, batch_size, output_dir): + pipeline = create_pipeline(pipeline_name_or_config_path, device=device) + + should_end = False + batch = [] + + while not should_end: + try: + input_path = task_queue.get_nowait() + except Empty: + should_end = True + else: + batch.append(input_path) + + if batch and (len(batch) == batch_size or should_end): + try: + for result in pipeline.predict(batch): + input_path = Path(result["input_path"]) + if result.get("page_index") is not None: + output_path = f"{input_path.stem}_{result['page_index']}.json" + else: + output_path = f"{input_path.stem}.json" + output_path = str(Path(output_dir, output_path)) + result.save_to_json(output_path) + print(f"Processed {repr(str(input_path))}") + except Exception as e: + print( + f"Error processing {batch} on {repr(device)}: {e}", file=sys.stderr + ) + batch.clear() + + +def main(): + parser = argparse.ArgumentParser() + parser.add_argument( + "--pipeline", type=str, required=True, help="Pipeline name or config path." + ) + parser.add_argument("--input_dir", type=str, required=True, help="Input directory.") + parser.add_argument( + "--device", + type=str, + required=True, + help="Specifies the devices for performing parallel inference.", + ) + parser.add_argument( + "--output_dir", type=str, default="output", help="Output directory." + ) + parser.add_argument( + "--instances_per_device", + type=int, + default=1, + help="Number of pipeline instances per device.", + ) + parser.add_argument( + "--batch_size", + type=int, + default=1, + help="Inference batch size for each pipeline instance.", + ) + parser.add_argument( + "--input_glob_pattern", + type=str, + default="*", + help="Pattern to find the input files.", + ) + args = parser.parse_args() + + input_dir = Path(args.input_dir) + if not input_dir.exists(): + print(f"The input directory does not exist: {input_dir}", file=sys.stderr) + return 2 + if not input_dir.is_dir(): + print(f"{repr(str(input_dir))} is not a directory.", file=sys.stderr) + return 2 + + output_dir = Path(args.output_dir) + if output_dir.exists() and not output_dir.is_dir(): + print(f"{repr(str(output_dir))} is not a directory.", file=sys.stderr) + return 2 + output_dir.mkdir(parents=True, exist_ok=True) + + device_type, device_ids = parse_device(args.device) + if device_ids is None or len(device_ids) == 1: + print( + "Please specify at least two devices for performing parallel inference.", + file=sys.stderr, + ) + sys.exit(2) + + if args.batch_size <= 0: + print("Batch size must be greater than 0.", file=sys.stderr) + sys.exit(2) + + manager = Manager() + task_queue = manager.Queue() + for img_path in input_dir.glob(args.input_glob_pattern): + task_queue.put(str(img_path)) + + processes = [] + for device_id in device_ids: + for _ in range(args.instances_per_device): + device = constr_device(device_type, [device_id]) + p = Process( + target=worker, + args=( + args.pipeline, + device, + task_queue, + args.batch_size, + str(output_dir), + ), + ) + p.start() + processes.append(p) + + for p in processes: + p.join() + + print("All done") + + return 0 + + +if __name__ == "__main__": + sys.exit(main()) +``` + +Assuming you save the script above as `mp_infer.py`, here are some example invocations: + +```bash +# PP-StructureV3 pipeline +# Process all files in `input_images` +# Use GPUs 0,1,2,3 with 1 pipeline instance per GPU and batch size 1 +python mp_infer.py \ + --pipeline PP-StructureV3 \ + --input_dir input_images/ \ + --device 'gpu:0,1,2,3' \ + --output_dir output1 + +# PP-StructureV3 pipeline +# Process all `.jpg` files in `input_images` +# Use GPUs 0 and 2 with 2 pipeline instances per GPU and batch size 4 +python mp_infer.py \ + --pipeline PP-StructureV3 \ + --input_dir input_images/ \ + --device 'gpu:0,2' \ + --output_dir output2 \ + --instances_per_device 2 \ + --batch_size 4 \ + --input_glob_pattern '*.jpg' +``` diff --git a/docs/pipeline_usage/instructions/parallel_inference.md b/docs/pipeline_usage/instructions/parallel_inference.md new file mode 100644 index 0000000000..db101afcc2 --- /dev/null +++ b/docs/pipeline_usage/instructions/parallel_inference.md @@ -0,0 +1,196 @@ +# 产线并行推理 + +## 指定多个推理设备 + +对于部分产线的 CLI 和 Python API，PaddleX 支持同时指定多个推理设备。如果指定了多个设备，产线初始化时将在每个设备上创建一个底层产线类对象的实例，并对接收到的输入进行并行推理。例如，对于通用版面解析 v3 产线： + +```bash +paddlex --pipeline PP-StructureV3 \ + --input input_images/ \ + --use_doc_orientation_classify False \ + --use_doc_unwarping False \ + --use_textline_orientation False \ + --save_path ./output \ + --device 'gpu:0,1,2,3' +``` + +```python +pipeline = create_pipeline(pipeline="PP-StructureV3", device="gpu:0,1,2,3") +output = pipeline.predict( + input="input_images/", + use_doc_orientation_classify=False, + use_doc_unwarping=False, + use_textline_orientation=False, +) +``` + +以上两个例子均使用 4 块 GPU（编号为 0、1、2、3）对 `input_images` 目录中的文件进行并行推理。 + +指定多个设备时，推理接口仍然与指定单设备时保持一致。请查看产线使用教程以了解某一产线是否支持指定多个推理设备。 + +## 多进程并行推理示例 + +除了使用 PaddleX 内置的多设备并行推理功能外，用户也可以结合实际场景，通过包装 PaddleX 产线推理 API 调用逻辑等手段实现并行推理加速，以期达到更优的加速比。如下是使用 Python 多进程实现多卡、多实例并行处理输入目录中的文件的示例代码： + +```python +import argparse +import sys +from multiprocessing import Manager, Process +from pathlib import Path +from queue import Empty + +from paddlex import create_pipeline +from paddlex.utils.device import constr_device, parse_device + + +def worker(pipeline_name_or_config_path, device, task_queue, batch_size, output_dir): + pipeline = create_pipeline(pipeline_name_or_config_path, device=device) + + should_end = False + batch = [] + + while not should_end: + try: + input_path = task_queue.get_nowait() + except Empty: + should_end = True + else: + batch.append(input_path) + + if batch and (len(batch) == batch_size or should_end): + try: + for result in pipeline.predict(batch): + input_path = Path(result["input_path"]) + if result.get("page_index") is not None: + output_path = f"{input_path.stem}_{result['page_index']}.json" + else: + output_path = f"{input_path.stem}.json" + output_path = str(Path(output_dir, output_path)) + result.save_to_json(output_path) + print(f"Processed {repr(str(input_path))}") + except Exception as e: + print( + f"Error processing {batch} on {repr(device)}: {e}", file=sys.stderr + ) + batch.clear() + + +def main(): + parser = argparse.ArgumentParser() + parser.add_argument( + "--pipeline", type=str, required=True, help="Pipeline name or config path." + ) + parser.add_argument("--input_dir", type=str, required=True, help="Input directory.") + parser.add_argument( + "--device", + type=str, + required=True, + help="Specifies the devices for performing parallel inference.", + ) + parser.add_argument( + "--output_dir", type=str, default="output", help="Output directory." + ) + parser.add_argument( + "--instances_per_device", + type=int, + default=1, + help="Number of pipeline instances per device.", + ) + parser.add_argument( + "--batch_size", + type=int, + default=1, + help="Inference batch size for each pipeline instance.", + ) + parser.add_argument( + "--input_glob_pattern", + type=str, + default="*", + help="Pattern to find the input files.", + ) + args = parser.parse_args() + + input_dir = Path(args.input_dir) + if not input_dir.exists(): + print(f"The input directory does not exist: {input_dir}", file=sys.stderr) + return 2 + if not input_dir.is_dir(): + print(f"{repr(str(input_dir))} is not a directory.", file=sys.stderr) + return 2 + + output_dir = Path(args.output_dir) + if output_dir.exists() and not output_dir.is_dir(): + print(f"{repr(str(output_dir))} is not a directory.", file=sys.stderr) + return 2 + output_dir.mkdir(parents=True, exist_ok=True) + + device_type, device_ids = parse_device(args.device) + if device_ids is None or len(device_ids) == 1: + print( + "Please specify at least two devices for performing parallel inference.", + file=sys.stderr, + ) + sys.exit(2) + + if args.batch_size <= 0: + print("Batch size must be greater than 0.", file=sys.stderr) + sys.exit(2) + + manager = Manager() + task_queue = manager.Queue() + for img_path in input_dir.glob(args.input_glob_pattern): + task_queue.put(str(img_path)) + + processes = [] + for device_id in device_ids: + for _ in range(args.instances_per_device): + device = constr_device(device_type, [device_id]) + p = Process( + target=worker, + args=( + args.pipeline, + device, + task_queue, + args.batch_size, + str(output_dir), + ), + ) + p.start() + processes.append(p) + + for p in processes: + p.join() + + print("All done") + + return 0 + + +if __name__ == "__main__": + sys.exit(main()) +``` + +假设将上述脚本存储为 `mp_infer.py`，以下是一些调用示例： + +```bash +# 通用版面解析 v3 产线 +# 处理 `input_images` 目录中所有文件 +# 使用 GPU 0、1、2、3，每块 GPU 上 1 个产线实例，每个实例一次处理 1 个输入文件 +python mp_infer.py \ + --pipeline PP-StructureV3 \ + --input_dir input_images/ \ + --device 'gpu:0,1,2,3' \ + --output_dir output1 + +# 通用版面解析 v3 产线 +# 处理 `input_images` 目录中所有后缀为 `.jpg` 的文件 +# 使用 GPU 0、2，每块 GPU 上 2 个产线实例，每个实例一次处理 4 个输入文件 +python mp_infer.py \ + --pipeline PP-StructureV3 \ + --input_dir input_images/ \ + --device 'gpu:0,2' \ + --output_dir output2 \ + --instances_per_device 2 \ + --batch_size 4 \ + --input_glob_pattern '*.jpg' +``` diff --git a/docs/pipeline_usage/instructions/pipeline_python_API.en.md b/docs/pipeline_usage/instructions/pipeline_python_API.en.md index f5a7abcb60..799b21c0d5 100644 --- a/docs/pipeline_usage/instructions/pipeline_python_API.en.md +++ b/docs/pipeline_usage/instructions/pipeline_python_API.en.md @@ -36,7 +36,7 @@ In short, there are only three steps: * `pp_option`: `PaddlePredictorOption` type, used to change inference settings (e.g. the operating mode). Please refer to [4-Inference Configuration](#4-inference-configuration) for more details; * `use_hpip`：`bool | None` type, whether to enable the high-performance inference plugin (`None` for using the setting from the configuration file); * `hpi_config`：`dict | None` type, high-performance inference configuration; - * Return Value: `BasePredictor` type. + * Return Value: `BasePipeline` type. ### 2. Perform Inference by Calling the `predict()` Method of the Prediction Model Pipeline Object diff --git a/docs/pipeline_usage/instructions/pipeline_python_API.md b/docs/pipeline_usage/instructions/pipeline_python_API.md index 1be0345dcd..1287cf9530 100644 --- a/docs/pipeline_usage/instructions/pipeline_python_API.md +++ b/docs/pipeline_usage/instructions/pipeline_python_API.md @@ -37,7 +37,7 @@ for res in output: * `pp_option`：`PaddlePredictorOption` 类型，用于改变运行模式等配置项，关于推理配置的详细说明，请参考下文[4-推理配置](#4-推理配置)； * `use_hpip`：`bool | None` 类型，是否启用高性能推理插件（`None` 表示使用配置文件中的配置）； * `hpi_config`：`dict | None` 类型，高性能推理配置； - * 返回值：`BasePredictor`类型。 + * 返回值：`BasePipeline`类型。 ### 2. 调用预测模型产线对象的`predict()`方法进行推理预测 diff --git a/docs/pipeline_usage/pipeline_develop_guide.en.md b/docs/pipeline_usage/pipeline_develop_guide.en.md index e993b39d27..c7a69e7950 100644 --- a/docs/pipeline_usage/pipeline_develop_guide.en.md +++ b/docs/pipeline_usage/pipeline_develop_guide.en.md @@ -117,8 +117,9 @@ The following steps are executed: > ❗ The results obtained from running the Python script are the same as those from the command line method. -If the pre-trained model pipeline meets your expectations, you can proceed directly to [development integration/deployment](#6-development-integration-and-deployment). If not, optimize the pipeline effects according to the following steps. +If you’d like to perform parallel inference, please refer to [Pipeline Parallel Inference](../pipeline_usage/instructions/parallel_inference.en.md). +If the pre-trained model pipeline meets your expectations, you can proceed directly to [development integration/deployment](#6-development-integration-and-deployment). If not, optimize the pipeline effects according to the following steps. ## 3. Model Selection (Optional) diff --git a/docs/pipeline_usage/pipeline_develop_guide.md b/docs/pipeline_usage/pipeline_develop_guide.md index 7517a4c0d6..7572f68bca 100644 --- a/docs/pipeline_usage/pipeline_develop_guide.md +++ b/docs/pipeline_usage/pipeline_develop_guide.md @@ -117,8 +117,10 @@ for res in output: > ❗ Python脚本运行得到的结果与命令行方式相同。 +如果希望进行并行推理，可参考 [产线并行推理](../pipeline_usage/instructions/parallel_inference.md)。 如果预训练模型产线的效果符合您的预期，即可直接进行[开发集成/部署](#6开发集成部署)，如果不符合，再根据后续步骤对产线的效果进行优化。 + ## 3、模型选择（可选） 由于一个产线中可能包含一个或多个单功能模块，在进行模型微调时，您需要根据测试的情况确定微调其中的哪个模块的模型。 @@ -166,7 +168,6 @@ Pipeline: PaddleX 也提供了其他三种部署方式，详细说明如下： - 🚀 高性能推理：在实际生产环境中，许多应用对部署策略的性能指标（尤其是响应速度）有着较严苛的标准，以确保系统的高效运行与用户体验的流畅性。为此，PaddleX 提供高性能推理插件，旨在对模型推理及前后处理进行深度性能优化，实现端到端流程的显著提速，详细的高性能部署流程请参考[PaddleX高性能部署指南](../pipeline_deploy/high_performance_inference.md)。 ☁️ 服务化部署：服务化部署是实际生产环境中常见的一种部署形式。通过将推理功能封装为服务，客户端可以通过网络请求来访问这些服务，以获取推理结果。PaddleX 支持多种产线服务化部署方案，详细的产线服务化部署流程请参考[PaddleX服务化部署指南](../pipeline_deploy/serving.md)。 diff --git a/docs/pipeline_usage/tutorials/cv_pipelines/human_keypoint_detection.en.md b/docs/pipeline_usage/tutorials/cv_pipelines/human_keypoint_detection.en.md index c6a2249e21..a0f5ac65f4 100644 --- a/docs/pipeline_usage/tutorials/cv_pipelines/human_keypoint_detection.en.md +++ b/docs/pipeline_usage/tutorials/cv_pipelines/human_keypoint_detection.en.md @@ -152,7 +152,7 @@ paddlex --pipeline human_keypoint_detection \ --device gpu:0 ``` -The relevant parameter descriptions and results explanations can be referred to in the parameter explanations and results explanations of [2.2.2 Integration via Python Script](#222-integration-via-python-script). +The relevant parameter descriptions and results explanations can be referred to in the parameter explanations and results explanations of [2.2.2 Integration via Python Script](#222-integration-via-python-script). Supports specifying multiple devices simultaneously for parallel inference. For details, please refer to [Pipeline Parallel Inference](../../instructions/parallel_inference.en.md#specifying-multiple-inference-devices). The visualization results are saved to `save_path`, as shown below: @@ -202,7 +202,7 @@ In the above Python script, the following steps are executed: device -The device used for pipeline inference. It supports specifying the specific card number of GPU, such as "gpu:0", other hardware card numbers, such as "npu:0", or CPU, such as "cpu". +The device used for pipeline inference. It supports specifying the specific card number of GPU, such as "gpu:0", other hardware card numbers, such as "npu:0", or CPU, such as "cpu". Supports specifying multiple devices simultaneously for parallel inference. For details, please refer to Pipeline Parallel Inference. str gpu:0 diff --git a/docs/pipeline_usage/tutorials/cv_pipelines/human_keypoint_detection.md b/docs/pipeline_usage/tutorials/cv_pipelines/human_keypoint_detection.md index 4539237954..806e7bedb0 100644 --- a/docs/pipeline_usage/tutorials/cv_pipelines/human_keypoint_detection.md +++ b/docs/pipeline_usage/tutorials/cv_pipelines/human_keypoint_detection.md @@ -148,7 +148,7 @@ paddlex --pipeline human_keypoint_detection \ --save_path ./output/ \ --device gpu:0 ``` -相关参数和运行结果说明可以参考[2.2.2 Python脚本方式集成](#222-python脚本方式集成)中的参数说明和结果解释。 +相关参数和运行结果说明可以参考[2.2.2 Python脚本方式集成](#222-python脚本方式集成)中的参数说明和结果解释。支持同时指定多个设备以进行并行推理，详情请参考 [产线并行推理](../../instructions/parallel_inference.md#指定多个推理设备)。 可视化结果保存至`save_path`，如下所示： @@ -197,7 +197,7 @@ for res in output: device -产线推理设备。支持指定GPU具体卡号，如“gpu:0”，其他硬件具体卡号，如“npu:0”，CPU如“cpu”。 +产线推理设备。支持指定GPU具体卡号，如“gpu:0”，其他硬件具体卡号，如“npu:0”，CPU如“cpu”。支持同时指定多个设备以进行并行推理，详情请参考 产线并行推理。 str gpu:0 diff --git a/docs/pipeline_usage/tutorials/cv_pipelines/image_anomaly_detection.en.md b/docs/pipeline_usage/tutorials/cv_pipelines/image_anomaly_detection.en.md index 3b6c1e3acd..cf11de23d5 100644 --- a/docs/pipeline_usage/tutorials/cv_pipelines/image_anomaly_detection.en.md +++ b/docs/pipeline_usage/tutorials/cv_pipelines/image_anomaly_detection.en.md @@ -86,7 +86,7 @@ Note: Due to network issues, the above URL could not be successfully parsed. If paddlex --pipeline anomaly_detection --input uad_grid.png --device gpu:0 --save_path ./output ``` -The relevant parameter descriptions can be found in the [2.1.2 Python Script Integration](#212-python脚本方式集成) section. +The relevant parameter descriptions can be found in the [2.1.2 Python Script Integration](#212-python脚本方式集成) section. Supports specifying multiple devices simultaneously for parallel inference. For details, please refer to [Pipeline Parallel Inference](../../instructions/parallel_inference.en.md#specifying-multiple-inference-devices). After running, the results will be printed to the terminal as follows: @@ -141,7 +141,7 @@ In the above Python script, the following steps are executed: device -Pipeline inference device. Supports specifying the specific GPU card number, such as "gpu:0", other hardware specific card numbers, such as "npu:0", CPU such as "cpu". +Pipeline inference device. Supports specifying the specific GPU card number, such as "gpu:0", other hardware specific card numbers, such as "npu:0", CPU such as "cpu". Supports specifying multiple devices simultaneously for parallel inference. For details, please refer to Pipeline Parallel Inference. str gpu:0 diff --git a/docs/pipeline_usage/tutorials/cv_pipelines/image_anomaly_detection.md b/docs/pipeline_usage/tutorials/cv_pipelines/image_anomaly_detection.md index 1fe5d36b39..8a005c522c 100644 --- a/docs/pipeline_usage/tutorials/cv_pipelines/image_anomaly_detection.md +++ b/docs/pipeline_usage/tutorials/cv_pipelines/image_anomaly_detection.md @@ -88,7 +88,7 @@ PaddleX 所提供的模型产线均可以快速体验效果，您可以在本地 paddlex --pipeline anomaly_detection --input uad_grid.png --device gpu:0 --save_path ./output ``` -相关的参数说明可以参考[2.1.2 Python脚本方式集成](#212-python脚本方式集成)中的参数说明。 +相关的参数说明可以参考[2.1.2 Python脚本方式集成](#212-python脚本方式集成)中的参数说明。支持同时指定多个设备以进行并行推理，详情请参考 [产线并行推理](../../instructions/parallel_inference.md#指定多个推理设备)。 运行后，会将结果打印到终端上，结果如下： @@ -146,7 +146,7 @@ for res in output: device -产线推理设备。支持指定GPU具体卡号，如“gpu:0”，其他硬件具体卡号，如“npu:0”，CPU如“cpu”。 +产线推理设备。支持指定GPU具体卡号，如“gpu:0”，其他硬件具体卡号，如“npu:0”，CPU如“cpu”。支持同时指定多个设备以进行并行推理，详情请参考 产线并行推理。 str gpu:0 diff --git a/docs/pipeline_usage/tutorials/cv_pipelines/image_classification.en.md b/docs/pipeline_usage/tutorials/cv_pipelines/image_classification.en.md index 265af00c58..eca8e7dd77 100644 --- a/docs/pipeline_usage/tutorials/cv_pipelines/image_classification.en.md +++ b/docs/pipeline_usage/tutorials/cv_pipelines/image_classification.en.md @@ -751,7 +751,7 @@ You can quickly experience the image classification pipeline with a single comma paddlex --pipeline image_classification --input general_image_classification_001.jpg --device gpu:0 --save_path ./output/ ``` -The relevant parameter descriptions can be found in the parameter explanation section of [2.2.2 Python Script Integration](#222-integration-via-python-script). +The relevant parameter descriptions can be found in the parameter explanation section of [2.2.2 Python Script Integration](#222-integration-via-python-script). Supports specifying multiple devices simultaneously for parallel inference. For details, please refer to [Pipeline Parallel Inference](../../instructions/parallel_inference.en.md#specifying-multiple-inference-devices). ```bash {'res': {'input_path': 'general_image_classification_001.jpg', 'page_index': None, 'class_ids': array([296, 170, 356, 258, 248], dtype=int32), 'scores': array([0.62736, 0.03752, 0.03256, 0.0323 , 0.03194], dtype=float32), 'label_names': ['ice bear, polar bear, Ursus Maritimus, Thalarctos maritimus', 'Irish wolfhound', 'weasel', 'Samoyed, Samoyede', 'Eskimo dog, husky']}} @@ -808,7 +808,7 @@ In the above Python script, the following steps are executed: device -The device used for pipeline inference. It supports specifying the specific card number of GPUs, such as "gpu:0", other hardware card numbers, such as "npu:0", and CPUs, such as "cpu". +The device used for pipeline inference. It supports specifying the specific card number of GPUs, such as "gpu:0", other hardware card numbers, such as "npu:0", and CPUs, such as "cpu". Supports specifying multiple devices simultaneously for parallel inference. For details, please refer to Pipeline Parallel Inference. str gpu:0 diff --git a/docs/pipeline_usage/tutorials/cv_pipelines/image_classification.md b/docs/pipeline_usage/tutorials/cv_pipelines/image_classification.md index 8f9ae89a45..c85808fbd0 100644 --- a/docs/pipeline_usage/tutorials/cv_pipelines/image_classification.md +++ b/docs/pipeline_usage/tutorials/cv_pipelines/image_classification.md @@ -748,7 +748,7 @@ PaddleX 所提供的模型产线均可以快速体验效果，你可以在星河 ```bash paddlex --pipeline image_classification --input general_image_classification_001.jpg --device gpu:0 --save_path ./output/ ``` -相关的参数说明可以参考[2.2.2 Python脚本方式集成](#222-python脚本方式集成)中的参数说明。 +相关的参数说明可以参考[2.2.2 Python脚本方式集成](#222-python脚本方式集成)中的参数说明。支持同时指定多个设备以进行并行推理，详情请参考 [产线并行推理](../../instructions/parallel_inference.md#指定多个推理设备)。 运行后，会将结果打印到终端上，结果如下： @@ -805,7 +805,7 @@ for res in output: device -产线推理设备。支持指定GPU具体卡号，如“gpu:0”，其他硬件具体卡号，如“npu:0”，CPU如“cpu”。 +产线推理设备。支持指定GPU具体卡号，如“gpu:0”，其他硬件具体卡号，如“npu:0”，CPU如“cpu”。支持同时指定多个设备以进行并行推理，详情请参考 产线并行推理。 str gpu:0 diff --git a/docs/pipeline_usage/tutorials/cv_pipelines/image_multi_label_classification.en.md b/docs/pipeline_usage/tutorials/cv_pipelines/image_multi_label_classification.en.md index eddd0460a7..12622d06f0 100644 --- a/docs/pipeline_usage/tutorials/cv_pipelines/image_multi_label_classification.en.md +++ b/docs/pipeline_usage/tutorials/cv_pipelines/image_multi_label_classification.en.md @@ -116,7 +116,7 @@ You can quickly experience the image multi-label classification pipeline effect paddlex --pipeline image_multilabel_classification --input general_image_classification_001.jpg --device gpu:0 ``` -The relevant parameter descriptions can be referred to in the parameter explanations in [2.2.2 Python Script Integration](). +The relevant parameter descriptions can be referred to in the parameter explanations in [2.2.2 Python Script Integration](). Supports specifying multiple devices simultaneously for parallel inference. For details, please refer to [Pipeline Parallel Inference](../../instructions/parallel_inference.en.md#specifying-multiple-inference-devices). After running, the result will be printed to the terminal as follows: @@ -176,7 +176,7 @@ In the above Python script, the following steps are performed: device -Pipeline inference device. Supports specifying the specific GPU card number, such as "gpu:0", other hardware specific card numbers, such as "npu:0", CPU such as "cpu". +Pipeline inference device. Supports specifying the specific GPU card number, such as "gpu:0", other hardware specific card numbers, such as "npu:0", CPU such as "cpu". Supports specifying multiple devices simultaneously for parallel inference. For details, please refer to Pipeline Parallel Inference. str gpu:0 diff --git a/docs/pipeline_usage/tutorials/cv_pipelines/image_multi_label_classification.md b/docs/pipeline_usage/tutorials/cv_pipelines/image_multi_label_classification.md index d8a28fa940..10f8397eb2 100644 --- a/docs/pipeline_usage/tutorials/cv_pipelines/image_multi_label_classification.md +++ b/docs/pipeline_usage/tutorials/cv_pipelines/image_multi_label_classification.md @@ -119,7 +119,7 @@ PaddleX 所提供的模型产线均可以快速体验效果，你可以在星河 ```bash paddlex --pipeline image_multilabel_classification --input general_image_classification_001.jpg --device gpu:0 ``` -相关的参数说明可以参考[2.2.2 Python脚本方式集成](#222-python脚本方式集成)中的参数说明。 +相关的参数说明可以参考[2.2.2 Python脚本方式集成](#222-python脚本方式集成)中的参数说明。支持同时指定多个设备以进行并行推理，详情请参考 [产线并行推理](../../instructions/parallel_inference.md#指定多个推理设备)。 运行后，会将结果打印到终端上，结果如下： @@ -177,7 +177,7 @@ for res in output: device -产线推理设备。支持指定GPU具体卡号，如“gpu:0”，其他硬件具体卡号，如“npu:0”，CPU如“cpu”。 +产线推理设备。支持指定GPU具体卡号，如“gpu:0”，其他硬件具体卡号，如“npu:0”，CPU如“cpu”。支持同时指定多个设备以进行并行推理，详情请参考 产线并行推理。 str gpu:0 diff --git a/docs/pipeline_usage/tutorials/cv_pipelines/instance_segmentation.en.md b/docs/pipeline_usage/tutorials/cv_pipelines/instance_segmentation.en.md index d06cb93d00..96bc13e7b8 100644 --- a/docs/pipeline_usage/tutorials/cv_pipelines/instance_segmentation.en.md +++ b/docs/pipeline_usage/tutorials/cv_pipelines/instance_segmentation.en.md @@ -229,7 +229,7 @@ paddlex --pipeline instance_segmentation \ --device gpu:0 ``` -The relevant parameter descriptions can be referred to in the parameter explanations in [2.2.2 Python Script Integration](). +The relevant parameter descriptions can be referred to in the parameter explanations in [2.2.2 Python Script Integration](). Supports specifying multiple devices simultaneously for parallel inference. For details, please refer to [Pipeline Parallel Inference](../../instructions/parallel_inference.en.md#specifying-multiple-inference-devices). After running, the result will be printed to the terminal as follows: @@ -285,7 +285,7 @@ In the above Python script, the following steps are performed: device -Pipeline inference device. Supports specifying the specific GPU card number, such as "gpu:0", other hardware specific card numbers, such as "npu:0", CPU such as "cpu". +Pipeline inference device. Supports specifying the specific GPU card number, such as "gpu:0", other hardware specific card numbers, such as "npu:0", CPU such as "cpu". Supports specifying multiple devices simultaneously for parallel inference. For details, please refer to Pipeline Parallel Inference. str None diff --git a/docs/pipeline_usage/tutorials/cv_pipelines/instance_segmentation.md b/docs/pipeline_usage/tutorials/cv_pipelines/instance_segmentation.md index 99a0ced4d1..897f70de8b 100644 --- a/docs/pipeline_usage/tutorials/cv_pipelines/instance_segmentation.md +++ b/docs/pipeline_usage/tutorials/cv_pipelines/instance_segmentation.md @@ -234,7 +234,7 @@ paddlex --pipeline instance_segmentation \ --save_path ./output \ --device gpu:0 ``` -相关的参数说明可以参考[2.2.2 Python脚本方式集成](#222-python脚本方式集成)中的参数说明。 +相关的参数说明可以参考[2.2.2 Python脚本方式集成](#222-python脚本方式集成)中的参数说明。支持同时指定多个设备以进行并行推理，详情请参考 [产线并行推理](../../instructions/parallel_inference.md#指定多个推理设备)。 运行后，会将结果打印到终端上，结果如下： ```bash @@ -287,7 +287,7 @@ for res in output: device -产线推理设备。支持指定GPU具体卡号，如“gpu:0”，其他硬件具体卡号，如“npu:0”，CPU如“cpu”。 +产线推理设备。支持指定GPU具体卡号，如“gpu:0”，其他硬件具体卡号，如“npu:0”，CPU如“cpu”。支持同时指定多个设备以进行并行推理，详情请参考 产线并行推理。 str None diff --git a/docs/pipeline_usage/tutorials/cv_pipelines/object_detection.en.md b/docs/pipeline_usage/tutorials/cv_pipelines/object_detection.en.md index d62863fc0d..69836aaf63 100644 --- a/docs/pipeline_usage/tutorials/cv_pipelines/object_detection.en.md +++ b/docs/pipeline_usage/tutorials/cv_pipelines/object_detection.en.md @@ -416,7 +416,7 @@ paddlex --pipeline object_detection \ --device gpu:0 ``` -For the description of parameters and interpretation of results, please refer to the parameter explanation and result interpretation in [2.2.2 Integration via Python Script](#222-integration-via-python-script). +For the description of parameters and interpretation of results, please refer to the parameter explanation and result interpretation in [2.2.2 Integration via Python Script](#222-integration-via-python-script). Supports specifying multiple devices simultaneously for parallel inference. For details, please refer to [Pipeline Parallel Inference](../../instructions/parallel_inference.en.md#specifying-multiple-inference-devices). The visualization results are saved to `save_path`, as shown below: @@ -465,7 +465,7 @@ In the above Python script, the following steps are executed: device -The device for pipeline inference. It supports specifying the specific card number of GPU, such as "gpu:0", other hardware card numbers, such as "npu:0", or CPU as "cpu". +The device for pipeline inference. It supports specifying the specific card number of GPU, such as "gpu:0", other hardware card numbers, such as "npu:0", or CPU as "cpu". Supports specifying multiple devices simultaneously for parallel inference. For details, please refer to Pipeline Parallel Inference. str gpu:0 diff --git a/docs/pipeline_usage/tutorials/cv_pipelines/object_detection.md b/docs/pipeline_usage/tutorials/cv_pipelines/object_detection.md index b8ec073c1d..9889b57f55 100644 --- a/docs/pipeline_usage/tutorials/cv_pipelines/object_detection.md +++ b/docs/pipeline_usage/tutorials/cv_pipelines/object_detection.md @@ -435,7 +435,7 @@ paddlex --pipeline object_detection \ --save_path ./output/ \ --device gpu:0 ``` -相关参数和运行结果说明可以参考[2.2.2 Python脚本方式集成](#222-python脚本方式集成)中的参数说明和结果解释。 +相关参数和运行结果说明可以参考[2.2.2 Python脚本方式集成](#222-python脚本方式集成)中的参数说明和结果解释。支持同时指定多个设备以进行并行推理，详情请参考 [产线并行推理](../../instructions/parallel_inference.md#指定多个推理设备)。 可视化结果保存至`save_path`，如下所示： @@ -483,7 +483,7 @@ for res in output: None device -产线推理设备。支持指定GPU具体卡号，如“gpu:0”，其他硬件具体卡号，如“npu:0”，CPU如“cpu”。 +产线推理设备。支持指定GPU具体卡号，如“gpu:0”，其他硬件具体卡号，如“npu:0”，CPU如“cpu”。支持同时指定多个设备以进行并行推理，详情请参考 产线并行推理。 str gpu:0 diff --git a/docs/pipeline_usage/tutorials/cv_pipelines/pedestrian_attribute_recognition.en.md b/docs/pipeline_usage/tutorials/cv_pipelines/pedestrian_attribute_recognition.en.md index ecea5a3471..f81966b3db 100644 --- a/docs/pipeline_usage/tutorials/cv_pipelines/pedestrian_attribute_recognition.en.md +++ b/docs/pipeline_usage/tutorials/cv_pipelines/pedestrian_attribute_recognition.en.md @@ -134,7 +134,7 @@ You can quickly experience the pedestrian attribute recognition pipeline with a paddlex --pipeline pedestrian_attribute_recognition --input pedestrian_attribute_002.jpg --device gpu:0 --save_path ./output/ ``` -The relevant parameter descriptions can be found in the parameter explanation section of [2.2.2 Python Script Integration](#222-python脚本方式集成). +The relevant parameter descriptions can be found in the parameter explanation section of [2.2.2 Python Script Integration](#222-python脚本方式集成). Supports specifying multiple devices simultaneously for parallel inference. For details, please refer to [Pipeline Parallel Inference](../../instructions/parallel_inference.en.md#specifying-multiple-inference-devices). After running, the result will be printed to the terminal, as shown below: @@ -194,7 +194,7 @@ In the above Python script, the following steps are executed: device -The device used for pipeline inference. It supports specifying the specific card number of GPUs, such as "gpu:0", other hardware card numbers, such as "npu:0", and CPUs, such as "cpu". +The device used for pipeline inference. It supports specifying the specific card number of GPUs, such as "gpu:0", other hardware card numbers, such as "npu:0", and CPUs, such as "cpu". Supports specifying multiple devices simultaneously for parallel inference. For details, please refer to Pipeline Parallel Inference. str gpu:0 diff --git a/docs/pipeline_usage/tutorials/cv_pipelines/pedestrian_attribute_recognition.md b/docs/pipeline_usage/tutorials/cv_pipelines/pedestrian_attribute_recognition.md index 3838b19047..0e849488a6 100644 --- a/docs/pipeline_usage/tutorials/cv_pipelines/pedestrian_attribute_recognition.md +++ b/docs/pipeline_usage/tutorials/cv_pipelines/pedestrian_attribute_recognition.md @@ -134,7 +134,7 @@ PaddleX 所提供的模型产线均可以快速体验效果，你可以在星河 ```bash paddlex --pipeline pedestrian_attribute_recognition --input pedestrian_attribute_002.jpg --device gpu:0 --save_path ./output/ ``` -相关的参数说明可以参考[2.2.2 Python脚本方式集成](#222-python脚本方式集成)中的参数说明。 +相关的参数说明可以参考[2.2.2 Python脚本方式集成](#222-python脚本方式集成)中的参数说明。支持同时指定多个设备以进行并行推理，详情请参考 [产线并行推理](../../instructions/parallel_inference.md#指定多个推理设备)。 运行后，会将结果打印到终端上，结果如下： @@ -192,7 +192,7 @@ for res in output: device -产线推理设备。支持指定GPU具体卡号，如“gpu:0”，其他硬件具体卡号，如“npu:0”，CPU如“cpu”。 +产线推理设备。支持指定GPU具体卡号，如“gpu:0”，其他硬件具体卡号，如“npu:0”，CPU如“cpu”。支持同时指定多个设备以进行并行推理，详情请参考 产线并行推理。 str gpu:0 diff --git a/docs/pipeline_usage/tutorials/cv_pipelines/rotated_object_detection.en.md b/docs/pipeline_usage/tutorials/cv_pipelines/rotated_object_detection.en.md index 362c39d16b..25902f1a2c 100644 --- a/docs/pipeline_usage/tutorials/cv_pipelines/rotated_object_detection.en.md +++ b/docs/pipeline_usage/tutorials/cv_pipelines/rotated_object_detection.en.md @@ -95,7 +95,7 @@ paddlex --pipeline rotated_object_detection \ --device gpu:0 \ ``` -The relevant parameter descriptions can be referred to in the parameter explanations of [2.2.2 Integration via Python Script](#222-integration-via-python-script). +The relevant parameter descriptions can be referred to in the parameter explanations of [2.2.2 Integration via Python Script](#222-integration-via-python-script). Supports specifying multiple devices simultaneously for parallel inference. For details, please refer to [Pipeline Parallel Inference](../../instructions/parallel_inference.en.md#specifying-multiple-inference-devices). After running, the results will be printed to the terminal, as follows: @@ -150,7 +150,7 @@ In the above Python script, the following steps were executed: device -The device used for pipeline inference. It supports specifying the specific card number of the GPU, such as "gpu:0", other hardware card numbers, such as "npu:0", or CPU, such as "cpu". +The device used for pipeline inference. It supports specifying the specific card number of the GPU, such as "gpu:0", other hardware card numbers, such as "npu:0", or CPU, such as "cpu". Supports specifying multiple devices simultaneously for parallel inference. For details, please refer to Pipeline Parallel Inference. str None diff --git a/docs/pipeline_usage/tutorials/cv_pipelines/rotated_object_detection.md b/docs/pipeline_usage/tutorials/cv_pipelines/rotated_object_detection.md index 65d07af48f..730d2d11a1 100644 --- a/docs/pipeline_usage/tutorials/cv_pipelines/rotated_object_detection.md +++ b/docs/pipeline_usage/tutorials/cv_pipelines/rotated_object_detection.md @@ -95,7 +95,7 @@ paddlex --pipeline rotated_object_detection \ --save_path ./output \ --device gpu:0 \ ``` -相关的参数说明可以参考[2.2.2 Python脚本方式集成](#222-python脚本方式集成)中的参数说明。 +相关的参数说明可以参考[2.2.2 Python脚本方式集成](#222-python脚本方式集成)中的参数说明。支持同时指定多个设备以进行并行推理，详情请参考 [产线并行推理](../../instructions/parallel_inference.md#指定多个推理设备)。 运行后，会将结果打印到终端上，结果如下： ```bash @@ -148,7 +148,7 @@ for res in output: device -产线推理设备。支持指定GPU具体卡号，如“gpu:0”，其他硬件具体卡号，如“npu:0”，CPU如“cpu”。 +产线推理设备。支持指定GPU具体卡号，如“gpu:0”，其他硬件具体卡号，如“npu:0”，CPU如“cpu”。支持同时指定多个设备以进行并行推理，详情请参考 产线并行推理。 str None diff --git a/docs/pipeline_usage/tutorials/cv_pipelines/semantic_segmentation.en.md b/docs/pipeline_usage/tutorials/cv_pipelines/semantic_segmentation.en.md index bc68b90435..81b250b139 100644 --- a/docs/pipeline_usage/tutorials/cv_pipelines/semantic_segmentation.en.md +++ b/docs/pipeline_usage/tutorials/cv_pipelines/semantic_segmentation.en.md @@ -262,7 +262,7 @@ paddlex --pipeline semantic_segmentation \ --device gpu:0 \ ``` -The relevant parameter descriptions can be referred to in the parameter explanations in [2.2.2 Python Script Integration](). +The relevant parameter descriptions can be referred to in the parameter explanations in [2.2.2 Python Script Integration](). Supports specifying multiple devices simultaneously for parallel inference. For details, please refer to [Pipeline Parallel Inference](../../instructions/parallel_inference.en.md#specifying-multiple-inference-devices). After running, the result will be printed to the terminal, as follows: @@ -317,7 +317,7 @@ In the above Python script, the following steps are executed: device -Pipeline inference device. Supports specifying the specific GPU card number, such as "gpu:0", other hardware specific card numbers, such as "npu:0", CPU such as "cpu". +Pipeline inference device. Supports specifying the specific GPU card number, such as "gpu:0", other hardware specific card numbers, such as "npu:0", CPU such as "cpu". Supports specifying multiple devices simultaneously for parallel inference. For details, please refer to Pipeline Parallel Inference. str None diff --git a/docs/pipeline_usage/tutorials/cv_pipelines/semantic_segmentation.md b/docs/pipeline_usage/tutorials/cv_pipelines/semantic_segmentation.md index 350322176c..84ea719a01 100644 --- a/docs/pipeline_usage/tutorials/cv_pipelines/semantic_segmentation.md +++ b/docs/pipeline_usage/tutorials/cv_pipelines/semantic_segmentation.md @@ -269,7 +269,7 @@ paddlex --pipeline semantic_segmentation \ --save_path ./output \ --device gpu:0 \ ``` -相关的参数说明可以参考[2.2.2 Python脚本方式集成](#222-python脚本方式集成)中的参数说明。 +相关的参数说明可以参考[2.2.2 Python脚本方式集成](#222-python脚本方式集成)中的参数说明。支持同时指定多个设备以进行并行推理，详情请参考 [产线并行推理](../../instructions/parallel_inference.md#指定多个推理设备)。 运行后，会将结果打印到终端上，结果如下： ```bash @@ -321,7 +321,7 @@ for res in output: device -产线推理设备。支持指定GPU具体卡号，如“gpu:0”，其他硬件具体卡号，如“npu:0”，CPU如“cpu”。 +产线推理设备。支持指定GPU具体卡号，如“gpu:0”，其他硬件具体卡号，如“npu:0”，CPU如“cpu”。支持同时指定多个设备以进行并行推理，详情请参考 产线并行推理。 str None diff --git a/docs/pipeline_usage/tutorials/cv_pipelines/small_object_detection.en.md b/docs/pipeline_usage/tutorials/cv_pipelines/small_object_detection.en.md index 6309df2a4a..999f128c82 100644 --- a/docs/pipeline_usage/tutorials/cv_pipelines/small_object_detection.en.md +++ b/docs/pipeline_usage/tutorials/cv_pipelines/small_object_detection.en.md @@ -113,7 +113,7 @@ paddlex --pipeline small_object_detection \ --device gpu:0 ``` -The relevant parameter descriptions can be referred to in the parameter explanations in [2.2.2 Python Script Integration](#222-python-script-integration). +The relevant parameter descriptions can be referred to in the parameter explanations in [2.2.2 Python Script Integration](#222-python-script-integration). Supports specifying multiple devices simultaneously for parallel inference. For details, please refer to [Pipeline Parallel Inference](../../instructions/parallel_inference.en.md#specifying-multiple-inference-devices). After running, the result will be printed to the terminal as follows: @@ -168,7 +168,7 @@ In the above Python script, the following steps are performed: device -Pipeline inference device. Supports specifying the specific GPU card number, such as "gpu:0", other hardware specific card numbers, such as "npu:0", CPU such as "cpu". +Pipeline inference device. Supports specifying the specific GPU card number, such as "gpu:0", other hardware specific card numbers, such as "npu:0", CPU such as "cpu". Supports specifying multiple devices simultaneously for parallel inference. For details, please refer to Pipeline Parallel Inference. str None diff --git a/docs/pipeline_usage/tutorials/cv_pipelines/small_object_detection.md b/docs/pipeline_usage/tutorials/cv_pipelines/small_object_detection.md index c73a2ea08b..41359e74ce 100644 --- a/docs/pipeline_usage/tutorials/cv_pipelines/small_object_detection.md +++ b/docs/pipeline_usage/tutorials/cv_pipelines/small_object_detection.md @@ -117,7 +117,7 @@ paddlex --pipeline small_object_detection \ --save_path ./output \ --device gpu:0 ``` -相关的参数说明可以参考[2.2.2 Python脚本方式集成](#222-python脚本方式集成)中的参数说明。 +相关的参数说明可以参考[2.2.2 Python脚本方式集成](#222-python脚本方式集成)中的参数说明。支持同时指定多个设备以进行并行推理，详情请参考 [产线并行推理](../../instructions/parallel_inference.md#指定多个推理设备)。 运行后，会将结果打印到终端上，结果如下： ```bash @@ -169,7 +169,7 @@ for res in output: device -产线推理设备。支持指定GPU具体卡号，如“gpu:0”，其他硬件具体卡号，如“npu:0”，CPU如“cpu”。 +产线推理设备。支持指定GPU具体卡号，如“gpu:0”，其他硬件具体卡号，如“npu:0”，CPU如“cpu”。支持同时指定多个设备以进行并行推理，详情请参考 产线并行推理。 str None diff --git a/docs/pipeline_usage/tutorials/cv_pipelines/vehicle_attribute_recognition.en.md b/docs/pipeline_usage/tutorials/cv_pipelines/vehicle_attribute_recognition.en.md index 7f768a9b33..05f600d4ef 100644 --- a/docs/pipeline_usage/tutorials/cv_pipelines/vehicle_attribute_recognition.en.md +++ b/docs/pipeline_usage/tutorials/cv_pipelines/vehicle_attribute_recognition.en.md @@ -136,7 +136,7 @@ Parameter Description: {'res': {'input_path': 'vehicle_attribute_002.jpg', 'boxes': [{'labels': ['red(红色)', 'sedan(轿车)'], 'cls_scores': array([0.96375, 0.94025]), 'det_score': 0.9774094820022583, 'coordinate': [196.32553, 302.3847, 639.3131, 655.57904]}, {'labels': ['suv(SUV)', 'brown(棕色)'], 'cls_scores': array([0.99968, 0.99317]), 'det_score': 0.9705657958984375, 'coordinate': [769.4419, 278.8417, 1401.0217, 641.3569]}]}} ``` -For the explanation of the running result parameters, you can refer to the result interpretation in [Section 2.2.2 Integration via Python Script](#222-integration-via-python-script). +For the explanation of the running result parameters, you can refer to the result interpretation in [Section 2.2.2 Integration via Python Script](#222-integration-via-python-script). Supports specifying multiple devices simultaneously for parallel inference. For details, please refer to [Pipeline Parallel Inference](../../instructions/parallel_inference.en.md#specifying-multiple-inference-devices). The visualization results are saved under `save_path`, and the visualization result is as follows: @@ -176,7 +176,7 @@ In the above Python script, the following steps are executed: device -The device used for pipeline inference. It supports specifying the specific card number of GPUs, such as "gpu:0", other hardware card numbers, such as "npu:0", and CPUs, such as "cpu". +The device used for pipeline inference. It supports specifying the specific card number of GPUs, such as "gpu:0", other hardware card numbers, such as "npu:0", and CPUs, such as "cpu". Supports specifying multiple devices simultaneously for parallel inference. For details, please refer to Pipeline Parallel Inference. str gpu:0 diff --git a/docs/pipeline_usage/tutorials/cv_pipelines/vehicle_attribute_recognition.md b/docs/pipeline_usage/tutorials/cv_pipelines/vehicle_attribute_recognition.md index 36aa5fb66c..3c08c8a165 100644 --- a/docs/pipeline_usage/tutorials/cv_pipelines/vehicle_attribute_recognition.md +++ b/docs/pipeline_usage/tutorials/cv_pipelines/vehicle_attribute_recognition.md @@ -132,7 +132,7 @@ PaddleX 所提供的模型产线均可以快速体验效果，你可以在星河 ```bash paddlex --pipeline vehicle_attribute_recognition --input vehicle_attribute_002.jpg --device gpu:0 --save_path ./output/ ``` -相关的参数说明可以参考[2.2.2 Python脚本方式集成](#222-python脚本方式集成)中的参数说明。 +相关的参数说明可以参考[2.2.2 Python脚本方式集成](#222-python脚本方式集成)中的参数说明。支持同时指定多个设备以进行并行推理，详情请参考 [产线并行推理](../../instructions/parallel_inference.md#指定多个推理设备)。 运行后，会将结果打印到终端上，结果如下： @@ -190,7 +190,7 @@ for res in output: device -产线推理设备。支持指定GPU具体卡号，如“gpu:0”，其他硬件具体卡号，如“npu:0”，CPU如“cpu”。 +产线推理设备。支持指定GPU具体卡号，如“gpu:0”，其他硬件具体卡号，如“npu:0”，CPU如“cpu”。支持同时指定多个设备以进行并行推理，详情请参考 产线并行推理。 str gpu:0 diff --git a/docs/pipeline_usage/tutorials/information_extraction_pipelines/document_scene_information_extraction_v3.en.md b/docs/pipeline_usage/tutorials/information_extraction_pipelines/document_scene_information_extraction_v3.en.md index 3beb75304f..73d22dc36b 100644 --- a/docs/pipeline_usage/tutorials/information_extraction_pipelines/document_scene_information_extraction_v3.en.md +++ b/docs/pipeline_usage/tutorials/information_extraction_pipelines/document_scene_information_extraction_v3.en.md @@ -1582,7 +1582,7 @@ To remove the page limit, please add the following configuration to the pipeline vectorInfo object | null -Serialized result of the vector database. Provided by the buildVectorStore operation. +Serialized result of the vector database. Provided by the buildVectorStore operation. Please note that the deserialization process involves performing an unpickle operation. To prevent malicious attacks, be sure to use data from trusted sources. No diff --git a/docs/pipeline_usage/tutorials/information_extraction_pipelines/document_scene_information_extraction_v3.md b/docs/pipeline_usage/tutorials/information_extraction_pipelines/document_scene_information_extraction_v3.md index 42fbfb0f42..940bd547f6 100644 --- a/docs/pipeline_usage/tutorials/information_extraction_pipelines/document_scene_information_extraction_v3.md +++ b/docs/pipeline_usage/tutorials/information_extraction_pipelines/document_scene_information_extraction_v3.md @@ -1587,7 +1587,7 @@ for res in visual_predict_res: vectorInfo object | null -向量数据库序列化结果。由buildVectorStore操作提供。 +向量数据库序列化结果。由buildVectorStore操作提供。请注意，反序列化过程需要进行 unpickle 操作，为了防止恶意攻击，请确保使用可信任来源的数据。 否 diff --git a/docs/pipeline_usage/tutorials/information_extraction_pipelines/document_scene_information_extraction_v4.en.md b/docs/pipeline_usage/tutorials/information_extraction_pipelines/document_scene_information_extraction_v4.en.md index 88312303c3..6fb1d5e577 100644 --- a/docs/pipeline_usage/tutorials/information_extraction_pipelines/document_scene_information_extraction_v4.en.md +++ b/docs/pipeline_usage/tutorials/information_extraction_pipelines/document_scene_information_extraction_v4.en.md @@ -1775,7 +1775,7 @@ To remove the page limit, please add the following configuration to the pipeline vectorInfo object | null -Serialized result of the vector database. Provided by the buildVectorStore operation. +Serialized result of the vector database. Provided by the buildVectorStore operation. Please note that the deserialization process involves performing an unpickle operation. To prevent malicious attacks, be sure to use data from trusted sources. No diff --git a/docs/pipeline_usage/tutorials/information_extraction_pipelines/document_scene_information_extraction_v4.md b/docs/pipeline_usage/tutorials/information_extraction_pipelines/document_scene_information_extraction_v4.md index 6b0de8123e..c0f1c9f4d9 100644 --- a/docs/pipeline_usage/tutorials/information_extraction_pipelines/document_scene_information_extraction_v4.md +++ b/docs/pipeline_usage/tutorials/information_extraction_pipelines/document_scene_information_extraction_v4.md @@ -1979,7 +1979,7 @@ for res in visual_predict_res: vectorInfo object | null -向量数据库序列化结果。由buildVectorStore操作提供。 +向量数据库序列化结果。由buildVectorStore操作提供。请注意，反序列化过程需要进行 unpickle 操作，为了防止恶意攻击，请确保使用可信任来源的数据。 否 diff --git a/docs/pipeline_usage/tutorials/ocr_pipelines/OCR.en.md b/docs/pipeline_usage/tutorials/ocr_pipelines/OCR.en.md index d11541a238..33b4c162be 100644 --- a/docs/pipeline_usage/tutorials/ocr_pipelines/OCR.en.md +++ b/docs/pipeline_usage/tutorials/ocr_pipelines/OCR.en.md @@ -471,7 +471,7 @@ paddlex --pipeline OCR \ --device gpu:0 ``` -For details on the relevant parameter descriptions, please refer to the parameter descriptions in [2.2.2 Python Script Integration](#222-python-script-integration). +For details on the relevant parameter descriptions, please refer to the parameter descriptions in [2.2.2 Python Script Integration](#222-python-script-integration). Supports specifying multiple devices simultaneously for parallel inference. For details, please refer to [Pipeline Parallel Inference](../../instructions/parallel_inference.en.md#specifying-multiple-inference-devices). After running, the results will be printed to the terminal as follows: @@ -539,7 +539,7 @@ In the above Python script, the following steps are executed: device -The device used for pipeline inference. It supports specifying specific GPU card numbers, such as "gpu:0", other hardware card numbers, such as "npu:0", or CPU, such as "cpu". +The device used for pipeline inference. It supports specifying specific GPU card numbers, such as "gpu:0", other hardware card numbers, such as "npu:0", or CPU, such as "cpu". Supports specifying multiple devices simultaneously for parallel inference. For details, please refer to Pipeline Parallel Inference. str gpu:0 diff --git a/docs/pipeline_usage/tutorials/ocr_pipelines/OCR.md b/docs/pipeline_usage/tutorials/ocr_pipelines/OCR.md index d300bc733c..a4497ae614 100644 --- a/docs/pipeline_usage/tutorials/ocr_pipelines/OCR.md +++ b/docs/pipeline_usage/tutorials/ocr_pipelines/OCR.md @@ -482,7 +482,7 @@ paddlex --pipeline OCR \ --save_path ./output \ --device gpu:0 ``` -相关的参数说明可以参考[2.2.2 Python脚本方式集成](#222-python脚本方式集成)中的参数说明。 +相关的参数说明可以参考[2.2.2 Python脚本方式集成](#222-python脚本方式集成)中的参数说明。支持同时指定多个设备以进行并行推理，详情请参考 [产线并行推理](../../instructions/parallel_inference.md#指定多个推理设备)。 运行后，会将结果打印到终端上，结果如下： ```bash @@ -549,7 +549,7 @@ for res in output: device -产线推理设备。支持指定GPU具体卡号，如“gpu:0”，其他硬件具体卡号，如“npu:0”，CPU如“cpu”。 +产线推理设备。支持指定GPU具体卡号，如“gpu:0”，其他硬件具体卡号，如“npu:0”，CPU如“cpu”。支持同时指定多个设备以进行并行推理，详情请参考 产线并行推理。 str gpu:0 diff --git a/docs/pipeline_usage/tutorials/ocr_pipelines/PP-StructureV3.en.md b/docs/pipeline_usage/tutorials/ocr_pipelines/PP-StructureV3.en.md index c6b7ce3613..6f78887b7f 100644 --- a/docs/pipeline_usage/tutorials/ocr_pipelines/PP-StructureV3.en.md +++ b/docs/pipeline_usage/tutorials/ocr_pipelines/PP-StructureV3.en.md @@ -648,7 +648,7 @@ paddlex --pipeline PP-StructureV3 \ --device gpu:0 ``` -The parameter description can be found in [2.2.2 Python Script Integration](#222-python-script-integration). +The parameter description can be found in [2.2.2 Python Script Integration](#222-python-script-integration). Supports specifying multiple devices simultaneously for parallel inference. For details, please refer to [Pipeline Parallel Inference](../../instructions/parallel_inference.en.md#specifying-multiple-inference-devices). After running, the result will be printed to the terminal, as follows: @@ -784,7 +784,7 @@ In the above Python script, the following steps are executed: device -The inference device for the pipeline. It supports specifying the specific GPU card number, such as "gpu:0", other hardware card numbers, such as "npu:0", or CPU, such as "cpu". +The inference device for the pipeline. It supports specifying the specific GPU card number, such as "gpu:0", other hardware card numbers, such as "npu:0", or CPU, such as "cpu". Supports specifying multiple devices simultaneously for parallel inference. For details, please refer to Pipeline Parallel Inference. str gpu:0 @@ -1523,6 +1523,18 @@ To remove the page limit, please add the following configuration to the pipeline No +useChartRecognition +boolean | null +Please refer to the description of the use_chart_recognition parameter of the pipeline object's predict method. +No + + +useRegionDetection +boolean | null +Please refer to the description of the use_region_detection parameter of the pipeline object's predict method. +No + + layoutThreshold number | null Please refer to the description of the layout_threshold parameter of the pipeline object's predict method. @@ -1636,6 +1648,12 @@ To remove the page limit, please add the following configuration to the pipeline Please refer to the description of the use_e2e_wireless_table_rec_model parameter of the pipeline object's predict method. No + +prettyMarkdown +boolean +Please refer to the description of the pretty_markdown parameter of the pipeline object's predict method. +No +