Skip to content

Commit 5bf826f

Browse files
authored
[Docs] Plugin EP packaging guidance (#26951)
### Description <!-- Describe your changes. --> Add initial plugin EP packaging guidance documentation. The new page is not connected up to the rest of the site yet. That will be done in a future PR. ### Motivation and Context <!-- - Why is this change required? What problem does it solve? - If it fixes an open issue, please link to the issue here. --> Provide guidance for implementers.
1 parent 6011643 commit 5bf826f

File tree

2 files changed

+127
-1
lines changed

2 files changed

+127
-1
lines changed

docs/execution-providers/plugin-ep-libraries.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -45,7 +45,7 @@ The following table lists the **required** variables and functions that an imple
4545

4646
<tr>
4747
<td>GetName</td>
48-
<td>Get the execution provider name.</td>
48+
<td>Get the execution provider name.<br/><br/>The recommended convention for a plugin execution provider name is to have the name end with the suffix "PluginExecutionProvider". E.g., "ContosoAiPluginExecutionProvider".</td>
4949
<td><a href="https://github.com/microsoft/onnxruntime/blob/16ae99ede405d3d6c59d7cce80c53f5f7055aeed/onnxruntime/test/autoep/library/ep.cc#L181">ExampleEp::GetNameImpl()</a></td>
5050
</tr>
5151

Lines changed: 126 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,126 @@
1+
# ONNX Runtime Plugin Execution Provider Packaging Guidance
2+
3+
## Overview
4+
5+
This document aims to provide guidance for ONNX Runtime (ORT) plugin Execution Provider (EP) implementers to consider with regards to packaging for a plugin EP.
6+
7+
## General Guidance
8+
9+
### Usage
10+
11+
Note: Generally, when referring to the ORT API, we will refer to the C API functions. Equivalents should exist for other language bindings that support plugin EP usage.
12+
13+
#### Manual EP Library Registration
14+
15+
Users are expected to call [`OrtApi::RegisterExecutionProviderLibrary()`](https://onnxruntime.ai/docs/api/c/struct_ort_api.html#a7c8ea74a2ee54d03052f3d7cd1e1335d) to register the plugin EP library. Then, they may either choose to use the auto EP selection mechanism or manually call [`OrtApi::SessionOptionsAppendExecutionProvider_V2()`](https://onnxruntime.ai/docs/api/c/struct_ort_api.html#a285a5da8c9a63eff55dc48e4cf3b56f6) to explicitly use the plugin EP.
16+
17+
### Structure
18+
19+
#### Contents
20+
21+
A plugin EP package should contain the plugin EP shared library file and any other files that need to be distributed with it.
22+
23+
A plugin EP package should NOT contain the ORT shared library or other core ORT libraries (e.g., onnxruntime.dll or libonnxruntime.so). Users should obtain the ORT library separately, most likely via installing the separate ONNX Runtime package.
24+
25+
A plugin EP package should have no need to depend on the separate ONNX Runtime package, so it is NOT recommended to do so.
26+
27+
#### Additional Information to Provide
28+
29+
##### Library Path
30+
31+
There should be a way to get the package's plugin EP library path. The user will need the plugin EP library path to call `OrtApi::RegisterExecutionProviderLibrary()`.
32+
33+
For example, the package may provide a helper function that returns the path to the plugin EP library. The recommended name for this helper function is "get library path".
34+
35+
##### EP name(s)
36+
There should be a way to get the plugin EP name(s) provided by the package. The user may require the plugin EP name(s) to select the appropriate `OrtEpDevice` instances to provide to `OrtApi::SessionOptionsAppendExecutionProvider_V2()`.
37+
38+
For example, the plugin EP name(s) may be well-documented or made available with a helper function provided by the package. The recommended name for this helper function is "get EP name", or "get EP names" if there are multiple names.
39+
40+
#### Package Naming
41+
42+
The name of the package should indicate that the package contains a plugin EP and be distinguishable from other ORT packages.
43+
44+
For example, this may be done by using a special prefix or suffix.
45+
46+
## Package-specific Guidance
47+
48+
### PyPI
49+
50+
#### Package Naming
51+
52+
The prefix "onnxruntime-ep" can be used to identify a plugin EP.
53+
54+
The suggested package naming convention is "onnxruntime-ep-\<EP identifier\>".
55+
56+
For example, "onnxruntime-ep-contoso-ai".
57+
58+
#### Helper Functions
59+
60+
As mentioned in the general guidance section, the package should provide helper function `get_library_path()` to get the EP library path. The package may provide helper function `get_ep_name()` or `get_ep_names()` to get the EP name(s).
61+
62+
#### Usage example
63+
64+
```python
65+
import onnxruntime as ort
66+
import onnxruntime_ep_contoso_ai as contoso_ep
67+
68+
# Path to the plugin EP library
69+
ep_lib_path = contoso_ep.get_library_path()
70+
# Registration name can be anything the application chooses
71+
ep_registration_name = "contoso_ep_registration"
72+
73+
# Register plugin EP library with ONNX Runtime
74+
ort.register_execution_provider_library(ep_registration_name, ep_lib_path)
75+
76+
# Create ORT session with explicit OrtEpDevice(s)
77+
78+
# Get EP name(s) from the plugin EP library
79+
ep_names = contoso_ep.get_ep_names()
80+
# For this example we'll use the first one
81+
ep_name = ep_names[0]
82+
83+
# Select an OrtEpDevice
84+
# For this example, we'll use any OrtEpDevices matching our EP name
85+
all_ep_devices = ort.get_ep_devices()
86+
selected_ep_devices = [ep_device for ep_device in all_ep_devices if ep_device.ep_name == ep_name]
87+
88+
assert len(selected_ep_devices) > 0
89+
90+
sess_options = ort.SessionOptions()
91+
92+
# EP-specific options
93+
ep_options = {}
94+
95+
# Equivalent to the C API's SessionOptionsAppendExecutionProvider_V2 that appends the plugin EP to the session options
96+
sess_options.add_provider_for_devices(selected_ep_devices, ep_options)
97+
98+
assert sess_options.has_providers() == True
99+
100+
# Create ORT session with the plugin EP
101+
model_path = "/path/to/model.onnx"
102+
sess = ort.InferenceSession(model_path, sess_options=sess_options)
103+
104+
# Use `sess`
105+
# ...
106+
107+
del sess
108+
109+
# Unregister the library using the same registration name specified earlier
110+
# Must only unregister a library after all sessions that use the library have been released
111+
ort.unregister_execution_provider_library(ep_registration_name)
112+
```
113+
114+
### NuGet
115+
116+
#### Package Naming
117+
118+
NuGet packages may use a reserved ID prefix.
119+
120+
The suggested package naming convention is "\<Vendor prefix\>.ML.OnnxRuntime.\<EP identifier\>.EP".
121+
122+
For example, "Contoso.ML.OnnxRuntime.ContosoAI.EP".
123+
124+
#### Helper Functions
125+
126+
As mentioned in the general guidance section, the package should provide helper function `GetLibraryPath()` to get the EP library path. The package may provide helper function `GetEpName()` or `GetEpNames()` to get the EP name(s).

0 commit comments

Comments
 (0)