Skip to content

Commit 0b4b9b7

Browse files
OskarFiedotOskarFiedot
authored
[DLStreamer] Add support for OpenVINO custom operations (open-edge-platform#1063)
1 parent 863f93e commit 0b4b9b7

File tree

15 files changed

+209
-44
lines changed

15 files changed

+209
-44
lines changed

‎libraries/dl-streamer/docs/source/dev_guide/custom_processing.md‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,9 @@ application:
3030
elements.
3131
6. Create a custom post-processing library.
3232

33+
For models with custom operations not natively supported by OpenVINO™, see
34+
[OpenVINO Custom Operations](./openvino_custom_operations.md).
35+
3336
See the sections below for more details on these options.
3437

3538
## 1. Consume tensor data and parse/convert it on application side

‎libraries/dl-streamer/docs/source/dev_guide/dev_guide_index.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,7 @@
1010
- [1. Model file format used by OpenVINO™ Toolkit](./model_preparation.md#1-model-file-format-used-by-openvino-toolkit)
1111
- [2. Model pre- and post-processing](./model_preparation.md#2-model-pre--and-post-processing)
1212
- [3. Specify model files in GStreamer elements](./model_preparation.md#3-specify-model-files-in-gstreamer-elements)
13+
- [OpenVINO Custom Operations](./openvino_custom_operations.md)
1314
- [Model Info Section](./model_info_xml.md)
1415
- [Python Bindings](./python_bindings.md)
1516
- [1. GStreamer Python bindings](./python_bindings.md#1-gstreamer-python-bindings)
@@ -75,6 +76,7 @@
7576
advanced_install/advanced_install_guide_index
7677
metadata
7778
model_preparation
79+
openvino_custom_operations
7880
model_info_xml
7981
python_bindings
8082
custom_plugin_installation

‎libraries/dl-streamer/docs/source/dev_guide/model_preparation.md‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -106,6 +106,9 @@ tensors to a GStreamer representation attached to a frame as metadata.
106106
If there is no suitable pre- and/or post-processing implementation in the Deep Learning Streamer,
107107
[Custom Processing](./custom_processing.md) can be used.
108108

109+
For models with custom operations not natively supported by OpenVINO™, see
110+
[OpenVINO Custom Operations](./openvino_custom_operations.md).
111+
109112
## 3. Specify model files in GStreamer elements
110113

111114
The path to the .xml model file must be specified by the mandatory `model`

‎libraries/dl-streamer/docs/source/dev_guide/openvino_custom_operations.md‎

Lines changed: 99 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,99 @@
1+
# OpenVINO Custom Operations Support
2+
3+
## Overview
4+
5+
DL Streamer provides support for OpenVINO™ custom operations through the `ov-extension-lib` parameter. This feature enables the use of models with custom operations that are not natively supported by OpenVINO™ Runtime, by loading extension libraries that define these custom operations.
6+
7+
Custom operations may be required in two scenarios:
8+
9+
1. **New or rarely used operations** - Operations from frameworks (TensorFlow, PyTorch, ONNX, etc.) that are not yet supported in OpenVINO™
10+
2. **User-defined operations** - Custom operations created specifically for a model using framework extension capabilities
11+
12+
The `ov-extension-lib` parameter is available in the following DLStreamer elements:
13+
14+
- `gvadetect` - Object detection
15+
- `gvaclassify` - Object classification
16+
- `gvainference` - Generic inference
17+
18+
## Prerequisites
19+
20+
Before using custom operations, you need:
21+
22+
1. **OpenVINO™ Extension Library** - A compiled `.so` file (on Linux) containing the implementation of custom operations
23+
2. **Model with Custom Operations** - An OpenVINO™ IR model that uses the custom operations defined in the extension library
24+
25+
For information on creating OpenVINO™ extension libraries, refer to the [OpenVINO™ Extensibility documentation](https://docs.openvino.ai/2025/documentation/openvino-extensibility.html).
26+
27+
## Usage
28+
29+
### Basic Usage
30+
31+
To use a model with custom operations, specify the path to the extension library using the `ov-extension-lib` parameter:
32+
33+
```bash
34+
gst-launch-1.0 filesrc location=input.mp4 ! decodebin3 ! \
35+
gvadetect model=model_with_custom_ops.xml \
36+
ov-extension-lib=/path/to/custom_operations.so \
37+
device=CPU ! \
38+
queue ! gvawatermark ! videoconvert ! autovideosink sync=false
39+
```
40+
41+
### Parameter Details
42+
43+
**Property Name:** `ov-extension-lib`
44+
45+
**Type:** String (file path)
46+
47+
**Default:** `null` (no extension library loaded)
48+
49+
**Description:** Absolute or relative path to the `.so` file defining custom OpenVINO operations.
50+
51+
**Flags:** readable, writable
52+
53+
### Usage Examples
54+
55+
#### Example 1: Object Detection with Custom Operations
56+
57+
```bash
58+
gst-launch-1.0 filesrc location=video.mp4 ! decodebin3 ! \
59+
gvadetect model=custom_detector.xml \
60+
ov-extension-lib=/opt/intel/openvino/custom_extensions/my_ops.so \
61+
device=CPU \
62+
batch-size=1 ! \
63+
queue ! gvawatermark ! videoconvert ! autovideosink sync=false
64+
```
65+
66+
#### Example 2: Classification with Custom Operations
67+
68+
```bash
69+
gst-launch-1.0 filesrc location=video.mp4 ! decodebin3 ! \
70+
gvadetect model=person-detection.xml device=CPU ! \
71+
gvaclassify model=custom_attributes.xml \
72+
ov-extension-lib=/home/user/extensions/attribute_ops.so \
73+
device=CPU \
74+
object-class=person ! \
75+
queue ! gvawatermark ! videoconvert ! autovideosink sync=false
76+
```
77+
78+
#### Example 3: Generic Inference with Custom Operations
79+
80+
```bash
81+
gst-launch-1.0 filesrc location=video.mp4 ! decodebin3 ! \
82+
gvainference model=custom_model.xml \
83+
ov-extension-lib=./extensions/libcustom.so \
84+
device=GPU ! \
85+
fakesink
86+
```
87+
88+
## Related Documentation
89+
90+
- [Model Preparation Guide](model_preparation.md)
91+
- [GStreamer Elements Reference](../elements/elements.md)
92+
- [OpenVINO™ Documentation](https://docs.openvino.ai/)
93+
94+
## See Also
95+
96+
- [gvadetect element documentation](../elements/gvadetect.md)
97+
- [gvaclassify element documentation](../elements/gvaclassify.md)
98+
- [gvainference element documentation](../elements/gvainference.md)
99+
- [Custom Processing Guide](custom_processing.md)

‎libraries/dl-streamer/docs/source/elements/gvaclassify.md‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -119,6 +119,9 @@ no-block : (Experimental) Option to help maintain frames per second o
119119
object-class : Filter for Region of Interest class label on this element input
120120
flags: readable, writable
121121
String. Default: null
122+
ov-extension-lib : Path to the .so file defining custom OpenVINO operations.
123+
flags: readable, writable
124+
String. Default: null
122125
parent : The parent of the object
123126
flags: readable, writable
124127
Object of type "GstObject"
@@ -150,7 +153,7 @@ reshape-width : Width to which the network will be reshaped.
150153
scale-method : Scale method to use in pre-preprocessing before inference. Only default and scale-method=fast (VAAPI based) supported in this element
151154
flags: readable, writable
152155
String. Default: null
153-
scheduling-policy : Scheduling policy across streams sharing same model instance: throughput (select first incoming frame), latency (select frames with earliest presentation time)
156+
scheduling-policy : Scheduling policy across streams sharing same model instance: throughput (select first incoming frame), latency (select frames with earliest presentation time out of the streams sharing same model-instance-id; recommended batch-size less than or equal to the number of streams)
154157
flags: readable, writable
155158
String. Default: null
156159
```

‎libraries/dl-streamer/docs/source/elements/gvadetect.md‎

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -120,6 +120,9 @@ Element Properties:
120120
object-class : Filter for Region of Interest class label on this element input
121121
flags: readable, writable
122122
String. Default: null
123+
ov-extension-lib : Path to the .so file defining custom OpenVINO operations.
124+
flags: readable, writable
125+
String. Default: null
123126
parent : The parent of the object
124127
flags: readable, writable
125128
Object of type "GstObject"
@@ -144,9 +147,9 @@ Element Properties:
144147
scale-method : Scale method to use in pre-preprocessing before inference. Only default and scale-method=fast (VAAPI based) supported in this element
145148
flags: readable, writable
146149
String. Default: null
147-
scheduling-policy : Scheduling policy across streams sharing same model instance: throughput (select first incoming frame), latency (select frames with earliest presentation time)
150+
scheduling-policy : Scheduling policy across streams sharing same model instance: throughput (select first incoming frame), latency (select frames with earliest presentation time out of the streams sharing same model-instance-id; recommended batch-size less than or equal to the number of streams)
148151
flags: readable, writable
149-
String. Default: null Write only
152+
String. Default: null
150153
threshold : Threshold for detection results. Only regions of interest with confidence values above the threshold will be added to the frame
151154
flags: readable, writable
152155
Float. Range: 0 - 1 Default: 0.5

‎libraries/dl-streamer/docs/source/elements/gvainference.md‎

Lines changed: 36 additions & 28 deletions
Original file line numberDiff line numberDiff line change
@@ -2,9 +2,9 @@
22

33
Runs deep learning inference using any model with RGB or BGR input.
44

5-
```sh
5+
```none
66
Pad Templates:
7-
SRC template: 'src'
7+
SINK template: 'sink'
88
Availability: Always
99
Capabilities:
1010
video/x-raw
@@ -13,7 +13,7 @@ Pad Templates:
1313
height: [ 1, 2147483647 ]
1414
framerate: [ 0/1, 2147483647/1 ]
1515
video/x-raw(memory:DMABuf)
16-
format: { (string)RGBA, (string)I420 }
16+
format: { (string)DMA_DRM }
1717
width: [ 1, 2147483647 ]
1818
height: [ 1, 2147483647 ]
1919
framerate: [ 0/1, 2147483647/1 ]
@@ -28,7 +28,7 @@ Pad Templates:
2828
height: [ 1, 2147483647 ]
2929
framerate: [ 0/1, 2147483647/1 ]
3030
31-
SINK template: 'sink'
31+
SRC template: 'src'
3232
Availability: Always
3333
Capabilities:
3434
video/x-raw
@@ -37,7 +37,7 @@ Pad Templates:
3737
height: [ 1, 2147483647 ]
3838
framerate: [ 0/1, 2147483647/1 ]
3939
video/x-raw(memory:DMABuf)
40-
format: { (string)RGBA, (string)I420 }
40+
format: { (string)DMA_DRM }
4141
width: [ 1, 2147483647 ]
4242
height: [ 1, 2147483647 ]
4343
framerate: [ 0/1, 2147483647/1 ]
@@ -56,8 +56,10 @@ Element has no clocking capabilities.
5656
Element has no URI handling capabilities.
5757
5858
Pads:
59-
SRC: 'src'
6059
SINK: 'sink'
60+
Pad Template: 'sink'
61+
SRC: 'src'
62+
Pad Template: 'src'
6163
6264
Element Properties:
6365
batch-size : Number of frames batched together for a single inference. If the batch-size is 0, then it will be set by default to be optimal for the device. Not all models support batching. Use model optimizer to ensure that the model has batching support.
@@ -66,12 +68,15 @@ Element Properties:
6668
cpu-throughput-streams: Deprecated. Use ie-config=CPU_THROUGHPUT_STREAMS=<number-streams> instead
6769
flags: readable, writable, deprecated
6870
Unsigned Integer. Range: 0 - 4294967295 Default: 0
71+
custom-postproc-lib : Path to the .so file defining custom model output converter. The library must implement the Convert function: void Convert(GstTensorMeta *outputTensors, const GstStructure *network, const GstStructure *params, GstAnalyticsRelationMeta *relationMeta);
72+
flags: readable, writable
73+
String. Default: null
74+
custom-preproc-lib : Path to the .so file defining custom input image pre-processing
75+
flags: readable, writable
76+
String. Default: null
6977
device : Target device for inference. Please see OpenVINO™ Toolkit documentation for list of supported devices.
7078
flags: readable, writable
7179
String. Default: "CPU"
72-
device-extensions : Comma separated list of KEY=VALUE pairs specifying the Inference Engine extension for a device
73-
flags: readable, writable
74-
String. Default: ""
7580
gpu-throughput-streams: Deprecated. Use ie-config=GPU_THROUGHPUT_STREAMS=<number-streams> instead
7681
flags: readable, writable, deprecated
7782
Unsigned Integer. Range: 0 - 4294967295 Default: 0
@@ -83,51 +88,51 @@ Element Properties:
8388
Unsigned Integer. Range: 1 - 4294967295 Default: 1
8489
inference-region : Identifier responsible for the region on which inference will be performed
8590
flags: readable, writable
86-
Enum "GvaInferenceBinRegion" Default: 0, "full-frame"
91+
Enum "InferenceRegionType3" Default: 0, "full-frame"
8792
(0): full-frame - Perform inference for full frame
8893
(1): roi-list - Perform inference for roi list
8994
labels : Array of object classes. It could be set as the following example: labels=<label1,label2,label3>
9095
flags: readable, writable
91-
String. Default: ""
96+
String. Default: null
9297
labels-file : Path to .txt file containing object classes (one per line)
9398
flags: readable, writable
9499
String. Default: null
95100
model : Path to inference model network file
96101
flags: readable, writable
97-
String. Default: ""
98-
model-instance-id : Identifier for sharing resources between inference elements of the same type. Elements with the instance-id will share model and other properties. If not specified, a unique identifier will be generated.
99-
flags: readable, writable
100-
String. Default: ""
101-
scheduling-policy : Scheduling policy across streams sharing same model instance: throughput (select first incoming frame), latency (select frames with earliest presentation time).
102+
String. Default: null
103+
model-instance-id : Identifier for sharing a loaded model instance between elements of the same type. Elements with the same model-instance-id will share all model and inference engine related properties
102104
flags: readable, writable
103-
String. Default: "throughput"
105+
String. Default: null
104106
model-proc : Path to JSON file with description of input/output layers pre-processing/post-processing
105107
flags: readable, writable
106-
String. Default: ""
108+
String. Default: null
107109
name : The name of the object
108-
flags: readable, writable, 0x2000
109-
String. Default: "gvainferencebin0"
110+
flags: readable, writable
111+
String. Default: "gvainference0"
110112
nireq : Number of inference requests
111113
flags: readable, writable
112114
Unsigned Integer. Range: 0 - 1024 Default: 0
113115
no-block : (Experimental) Option to help maintain frames per second of incoming stream. Skips inference on an incoming frame if all inference requests are currently processing outstanding frames
114-
flags: readable, writable
116+
flags: readable, writable, deprecated
115117
Boolean. Default: false
116118
object-class : Filter for Region of Interest class label on this element input
117119
flags: readable, writable
118-
String. Default: ""
120+
String. Default: null
121+
ov-extension-lib : Path to the .so file defining custom OpenVINO operations.
122+
flags: readable, writable
123+
String. Default: null
119124
parent : The parent of the object
120-
flags: readable, writable, 0x2000
125+
flags: readable, writable
121126
Object of type "GstObject"
122-
pre-process-backend : Select a pre-processing method (color conversion, resize and crop), one of 'ie', 'opencv', 'va', 'va-surface-sharing'. If not set, it will be selected automatically: 'va' for VAMemory and DMABuf, 'ie' for SYSTEM memory.
127+
pre-process-backend : Select a pre-processing method (color conversion, resize and crop), one of 'ie', 'opencv', 'va', 'va-surface-sharing, 'vaapi', 'vaapi-surface-sharing'. If not set, it will be selected automatically: 'va' for VAMemory and DMABuf, 'ie' for SYSTEM memory.
123128
flags: readable, writable
124129
String. Default: ""
125-
qos : Handle Quality-of-Service events
126-
flags: readable, writable
127-
Boolean. Default: false
128130
pre-process-config : Comma separated list of KEY=VALUE parameters for image processing pipeline configuration
129131
flags: readable, writable
130132
String. Default: ""
133+
qos : Handle Quality-of-Service events
134+
flags: readable, writable
135+
Boolean. Default: false
131136
reshape : If true, model input layer will be reshaped to resolution of input frames (no resize operation before inference). Note: this feature has limitations, not all network supports reshaping.
132137
flags: readable, writable
133138
Boolean. Default: false
@@ -139,5 +144,8 @@ Element Properties:
139144
Unsigned Integer. Range: 0 - 4294967295 Default: 0
140145
scale-method : Scale method to use in pre-preprocessing before inference. Only default and scale-method=fast (VAAPI based) supported in this element
141146
flags: readable, writable
142-
String. Default: null Write only
147+
String. Default: null
148+
scheduling-policy : Scheduling policy across streams sharing same model instance: throughput (select first incoming frame), latency (select frames with earliest presentation time out of the streams sharing same model-instance-id; recommended batch-size less than or equal to the number of streams)
149+
flags: readable, writable
150+
String. Default: null
143151
```

0 commit comments

Comments
 (0)