You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/execution-providers/plugin-ep-libraries.md
+40-1Lines changed: 40 additions & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -335,7 +335,7 @@ For example, if a single factory instance supports both CPU and NPU, then the ca
335
335
<p align="center"><img width="100%" src="../../images/plugin_ep_sd_lib_reg.png" alt="Sequence diagram showing registration and unregistration of a plugin EP library"/></p>
336
336
337
337
### Session creation with explicit OrtEpDevice(s)
338
-
The application code below use the API function [SessionOptionsAppendExecutionProvider_V2](https://onnxruntime.ai/docs/api/c/struct_ort_api.html#a285a5da8c9a63eff55dc48e4cf3b56f6) to add an EP to an ONNX Runtime session.
338
+
The application code below uses the API function [SessionOptionsAppendExecutionProvider_V2](https://onnxruntime.ai/docs/api/c/struct_ort_api.html#a285a5da8c9a63eff55dc48e4cf3b56f6) to add an EP from a library to an ONNX Runtime session.
339
339
340
340
The application first calls [GetEpDevices](https://onnxruntime.ai/docs/api/c/struct_ort_api.html#a52107386ff1be870f55a0140e6add8dd) to get a list of `OrtEpDevices`
341
341
available to the application. Each `OrtEpDevice` represents a hardware device supported by an `OrtEpFactory`.
@@ -398,6 +398,45 @@ As shown in the following sequence diagram, ONNX Runtime calls `OrtEpFactory::Cr
398
398
<br/>
399
399
<palign="center"><imgwidth="100%"src="../../images/plugin_ep_sd_appendv2.png"alt="Sequence diagram showing session creation with explicit ep devices"/></p>
400
400
401
+
### Session creation with automatic EP selection
402
+
The application code below uses the API function [SessionOptionsSetEpSelectionPolicy](https://onnxruntime.ai/docs/api/c/struct_ort_api.html#a2ae116df2c6293e4094a6742a6c46f7e) to have ONNX Runtime automatically select an EP based on the user's policy (e.g., PREFER_NPU).
403
+
If the plugin EP library registered with ONNX Runtime has a factory that supports NPU, then ONNX Runtime may select an EP from that factory to run the model.
404
+
405
+
```cpp
406
+
// NOTE: this snippet does not properly handle errors.
407
+
408
+
// NOTE: Assume plugin EP library has been registered and supports NPU
409
+
OrtStatus* status = ort_api->RegisterExecutionProviderLibrary(/*...*/);
410
+
411
+
OrtSessionOptions* session_options = nullptr;
412
+
status = ort_api->CreateSessionOptions(&session_options);
413
+
status = ort_api->SessionOptionsSetEpSelectionPolicy(
414
+
session_options,
415
+
OrtExecutionProviderDevicePolicy::PREFER_NPU,
416
+
);
417
+
418
+
419
+
OrtSession* session = nullptr;
420
+
status = ort_api->CreateSession(
421
+
ort_env,
422
+
ORT_TSTR("model.onnx"),
423
+
session_options,
424
+
&session
425
+
);
426
+
427
+
// Run model ...
428
+
429
+
// Release resources
430
+
ort_api->ReleaseStatus(status);
431
+
ort_api->ReleaseSession(session);
432
+
ort_api->ReleaseSessionOptions(session_options);
433
+
434
+
status = ort_api->UnregisterExecutionProviderLibrary(/*...*/);
435
+
```
436
+
437
+
<br/>
438
+
<p align="center"><img width="100%" src="../../images/plugin_ep_sd_autoep.png" alt="Sequence diagram showing session creation with automatic EP selection"/></p>
0 commit comments