Skip to content

Commit 6334632

Browse files
Add section about auto ep
1 parent 6a82910 commit 6334632

File tree

1 file changed

+40
-1
lines changed

1 file changed

+40
-1
lines changed

docs/execution-providers/plugin-ep-libraries.md

Lines changed: 40 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -335,7 +335,7 @@ For example, if a single factory instance supports both CPU and NPU, then the ca
335335
<p align="center"><img width="100%" src="../../images/plugin_ep_sd_lib_reg.png" alt="Sequence diagram showing registration and unregistration of a plugin EP library"/></p>
336336
337337
### Session creation with explicit OrtEpDevice(s)
338-
The application code below use the API function [SessionOptionsAppendExecutionProvider_V2](https://onnxruntime.ai/docs/api/c/struct_ort_api.html#a285a5da8c9a63eff55dc48e4cf3b56f6) to add an EP to an ONNX Runtime session.
338+
The application code below uses the API function [SessionOptionsAppendExecutionProvider_V2](https://onnxruntime.ai/docs/api/c/struct_ort_api.html#a285a5da8c9a63eff55dc48e4cf3b56f6) to add an EP from a library to an ONNX Runtime session.
339339
340340
The application first calls [GetEpDevices](https://onnxruntime.ai/docs/api/c/struct_ort_api.html#a52107386ff1be870f55a0140e6add8dd) to get a list of `OrtEpDevices`
341341
available to the application. Each `OrtEpDevice` represents a hardware device supported by an `OrtEpFactory`.
@@ -398,6 +398,45 @@ As shown in the following sequence diagram, ONNX Runtime calls `OrtEpFactory::Cr
398398
<br/>
399399
<p align="center"><img width="100%" src="../../images/plugin_ep_sd_appendv2.png" alt="Sequence diagram showing session creation with explicit ep devices"/></p>
400400

401+
### Session creation with automatic EP selection
402+
The application code below uses the API function [SessionOptionsSetEpSelectionPolicy](https://onnxruntime.ai/docs/api/c/struct_ort_api.html#a2ae116df2c6293e4094a6742a6c46f7e) to have ONNX Runtime automatically select an EP based on the user's policy (e.g., PREFER_NPU).
403+
If the plugin EP library registered with ONNX Runtime has a factory that supports NPU, then ONNX Runtime may select an EP from that factory to run the model.
404+
405+
```cpp
406+
// NOTE: this snippet does not properly handle errors.
407+
408+
// NOTE: Assume plugin EP library has been registered and supports NPU
409+
OrtStatus* status = ort_api->RegisterExecutionProviderLibrary(/*...*/);
410+
411+
OrtSessionOptions* session_options = nullptr;
412+
status = ort_api->CreateSessionOptions(&session_options);
413+
status = ort_api->SessionOptionsSetEpSelectionPolicy(
414+
session_options,
415+
OrtExecutionProviderDevicePolicy::PREFER_NPU,
416+
);
417+
418+
419+
OrtSession* session = nullptr;
420+
status = ort_api->CreateSession(
421+
ort_env,
422+
ORT_TSTR("model.onnx"),
423+
session_options,
424+
&session
425+
);
426+
427+
// Run model ...
428+
429+
// Release resources
430+
ort_api->ReleaseStatus(status);
431+
ort_api->ReleaseSession(session);
432+
ort_api->ReleaseSessionOptions(session_options);
433+
434+
status = ort_api->UnregisterExecutionProviderLibrary(/*...*/);
435+
```
436+
437+
<br/>
438+
<p align="center"><img width="100%" src="../../images/plugin_ep_sd_autoep.png" alt="Sequence diagram showing session creation with automatic EP selection"/></p>
439+
401440
## API reference
402441
API header files:
403442
- [onnxruntime_ep_c_api.h](https://github.com/microsoft/onnxruntime/blob/main/include/onnxruntime/core/session/onnxruntime_ep_c_api.h)

0 commit comments

Comments
 (0)