You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Compile <code>OrtGraph</code> instances assigned to the <code>OrtEp</code>. Implementation must set a <code>OrtNodeComputeInfo</code> instance for each <code>OrtGraph</code> in order to define its computation function.<br/><br/>
62
62
If the session is configured to generate a pre-compiled model, the execution provider must return <code>count</code> number of EPContext nodes.
@@ -65,7 +65,7 @@ If the session is configured to generate a pre-compiled model, the execution pro
Given an op with domain <code>domain</code> and type <code>op_type</code>, determine whether an associated node's data layout should be converted to a <code>target_data_layout</code>. If the EP prefers a non-default data layout, this function will be called during layout transformation with <code>target_data_layout</code> set to the EP's preferred data layout<br/><br/>
99
99
Implementation of this function is optional. If an EP prefers a non-default data layout, it may implement this to customize the specific op data layout preferences at a finer granularity.
@@ -102,7 +102,7 @@ Implementation of this function is optional. If an EP prefers a non-default data
Set dynamic options on this EP. Dynamic options can be set by the application at any time after session creation with <ahref="https://onnxruntime.ai/docs/api/c/struct_ort_api.html#ab1117a51683e4fbb42687c9db6e8d5fb"><code>OrtApi::SetEpDynamicOptions()</code></a>.<br/><br/>
108
108
Implementation of this function is optional. An EP should only impliment this function if it needs to handle any dynamic options.
@@ -111,7 +111,7 @@ Implementation of this function is optional. An EP should only impliment this fu
Called by ORT to notify the EP of the start of a run.<br/><br/>
117
117
Implementation of this function is optional. An EP should only impliment this function if it needs to handle application-provided options at the start of a run.
@@ -120,7 +120,7 @@ Implementation of this function is optional. An EP should only impliment this fu
Called by ORT to notify the EP of the end of a run.<br/><br/>
126
126
Implementation of this function is optional. An EP should only impliment this function if it needs to handle application-provided options at the end of a run.
@@ -129,7 +129,7 @@ Implementation of this function is optional. An EP should only impliment this fu
Create an <code>OrtAllocator</code> for the given <code>OrtMemoryInfo</code> for an <code>OrtSession</code>.<br/><br/>The <code>OrtMemoryInfo</code> instance will match one of the values set in the <code>OrtEpDevice</code> using <code>EpDevice_AddAllocatorInfo</code>. Any allocator specific options should be read from the session options.<br/><br/>
135
135
Implementation of this function is optional. If not provided, ORT will use `OrtEpFactory::CreateAllocator()`.
@@ -138,7 +138,7 @@ Implementation of this function is optional. If not provided, ORT will use `OrtE
Create a synchronization stream for the given memory device for an <code>OrtSession</code>.<br/><br/>This is used to create a synchronization stream for the execution provider and is used to synchronize operations on the device during model execution. Any stream specific options should be read from the session options.<br/><br/>
144
144
Implementation of this function is optional. If not provided, ORT will use `OrtEpFactory::CreateSyncStreamForDevice()`.
@@ -147,7 +147,7 @@ Implementation of this function is optional. If not provided, ORT will use `OrtE
Get a string with details about the EP stack used to produce a compiled model.<br/><br/>
153
153
The compatibility information string can be used with <code>OrtEpFactory::ValidateCompiledModelCompatibilityInfo</code> to determine if a compiled model is compatible with the EP.
@@ -170,43 +170,43 @@ The following table lists the **required** variables and functions that an imple
<td>Get the vendor ID of the vendor that owns the EP that the factory creates. This is typically the <ahref="https://pcisig.com/membership/member-companies">PCI vendor ID</a>.</td>
<td>Get the version of the EP that the factory creates. The version string should adhere to the <ahref="https://github.com/semver/semver/blob/v2.0.0/semver.md">Semantic Versioning 2.0 specification</a>.</td>
<td>Creates an <code>OrtEp</code> instance for use in an ONNX Runtime session. ORT calls <code>OrtEpFactory::ReleaseEp()</code> to release the instance.</td>
<td>Validate the compatibility of a compiled model with the EP.<br/><br/>
228
228
This function validates if a model produced with the supllied compatibility information string is supported by the underlying EP.
229
229
The implementation should check if a compiled model is compatible with the EP and return the appropriate <code>OrtCompiledModelCompatibility</code> value.</td>
<td>Create an <code>OrtAllocator</code> that can be shared across sessions for the given <code>OrtMemoryInfo</code>.<br/><br/>
236
236
The factory that creates the EP is responsible for providing the allocators required by the EP. The <code>OrtMemoryInfo</code> instance will match one of the values set in the <code>OrtEpDevice</code> using <code>EpDevice_AddAllocatorInfo</code>.</td>
Struct of functions that an EP implements if it needs to support Streams.
@@ -579,7 +579,7 @@ Struct of functions that an EP implements if it needs to support Streams.
579
579
580
580
<tr>
581
581
<td>
582
-
<a href="#ort-ep-factory">OrtEpFactory</a>
582
+
OrtEpFactory
583
583
</td>
584
584
<td>
585
585
A plugin EP library provides ORT with one or more instances of <code>OrtEpFactory</code>. An <code>OrtEpFactory</code> implements functions that are used by ORT to query device support, create allocators, create data transfer objects, and create instances of an EP (i.e., an <code>OrtEp</code> instance).<br/>
@@ -592,7 +592,7 @@ Alternatively, if an EP library author needs ONNX Runtime to partition the graph
592
592
593
593
<tr>
594
594
<td>
595
-
<a href="#ort-ep">OrtEp</a>
595
+
OrtEp
596
596
</td>
597
597
<td>
598
598
An instance of an Ep that can execute model nodes on one or more hardware devices (<code>OrtHardwareDevice</code>). An <code>OrtEp</code> implements functions that are used by ORT to query graph node support, compile supported nodes, query preferred data layout, set run options, etc. An <code>OrtEpFactory</code> creates an <code>OrtEp</code> instance via the <code>OrtEpFactory::CreateEp()</code> function.
@@ -670,7 +670,7 @@ Opaque type that represents a node in a graph.
0 commit comments