Skip to content

Commit a648271

Browse files
authored
multimodal: docs: Updated Docs to add dlstreamer references (open-edge-platform#1002)
This PR updates documentation to include DLStreamer pipeline server references and modernizes the multimodal weld defect detection system requirements. The changes reflect a shift in the architecture to explicitly document the DLStreamer component's role in video processing and adds detailed pipeline configuration documentation. Signed-off-by: Vellaisamy, Sathyendran <sathyendran.vellaisamy@intel.com>
1 parent 5f06473 commit a648271

3 files changed

Lines changed: 52 additions & 7 deletions

File tree

manufacturing-ai-suite/industrial-edge-insights-multimodal/docs/user-guide/weld-defect-detection/get-started.md

Lines changed: 38 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -57,21 +57,56 @@ RTSP stream and csv data over mqtt using simulator and publishing the anomaly re
5757

5858
Using the `edge-ai-suites/manufacturing-ai-suite/industrial-edge-insights-multimodal/weld-data-simulator/simulation-data/` which is a normalized version of open source data welding dataset from <https://huggingface.co/datasets/amr-lopezjos/Intel_Robotic_Welding_Multimodal_Dataset>.
5959

60+
The simulator reads `.avi` video files from the dataset and streams them as RTSP for vision data using the **mediamtx** server. This enables real-time video ingestion, simulating camera feeds for weld defect detection. The **dlstreamer-pipeline-server** connects to the RTSP stream and processes the video frames using a Geti model for automated defect analysis.
61+
6062
Timeseries data is being ingested into **Telegraf** using the **MQTT** protocol using the **weld-data-simulator** data simulator
6163
Vision data is being ingested into **dlstreamer-pipeline-server** using the **RTSP** protocol using the **weld-data-simulator** data simulator
6264

6365
### **Data Ingestion**
6466

65-
**Telegraf** through its input plugins (**MQTT**) gathers the data and sends this input data to both **InfluxDB** and **Time Series Analytics Microservice**.
66-
**dlstreamer-pipeline-server** gathers the data through RTSP Stream using **mediamxt** as the **RTSP Server**.
67+
Vision Data: **dlstreamer-pipeline-server** gathers the data through RTSP Stream using **mediamtx** as the **RTSP Server**.
6768

69+
Time-series Data: **Telegraf** through its input plugins (**MQTT**) gathers the data and sends this input data to both **InfluxDB** and **Time Series Analytics Microservice**.
6870

6971
### **Data Storage**
7072

7173
**InfluxDB** stores the incoming data coming from **Telegraf**, **Time Series Analytics Microservice** and **Fusion Analytics** .
7274

7375
### **Data Processing**
7476

77+
**DL Streamer Pipeline Server** sends the images with overlaid bounding boxes through webrtc protocol to webrtc browser client. This is done via the MediaMTX server used for signaling. Coturn server is used to facilitate NAT traversal and ensure that the webrtc stream is accessible on a non-native browser client and helps in cases where firewall is enabled.
78+
79+
#### **`DL Streamer Pipeline Server config.json`**
80+
81+
**Pipeline Configuration**:
82+
83+
| Key | Description | Example Value |
84+
|----------------|-----------------------------------------------------------------------------|----------------------------------------|
85+
| `name` | The name of the pipeline configuration. | `"weld_defect_classification"` |
86+
| `source` | The source type for video ingestion. | `"gstreamer"` |
87+
| `queue_maxsize`| Maximum size of the queue for processing frames. | `50` |
88+
| `pipeline` | GStreamer pipeline string defining the video processing flow from RTSP source through classification to output. | `"rtspsrc location=\"rtsp://mediamtx:8554/live.stream\" latency=100 name=source ! rtph264depay ! h264parse ! decodebin ! videoconvert ! gvaclassify inference-region=full-frame name=classification ! gvametaconvert add-empty-results=true name=metaconvert ! queue ! gvafpscounter ! appsink name=destination"` |
89+
| `parameters` | Configuration parameters for pipeline elements, specifically for the classification element properties. | See below for nested structure |
90+
91+
**Parameters Properties**:
92+
93+
| Key | Description | Value |
94+
|------------------------------|-----------------------------------------------------------------------------|--------------------------------|
95+
| `classification-properties` | Properties for the classification element in the pipeline. | Object containing element configuration |
96+
| `element.name` | Name of the GStreamer element to configure. | `"classification"` |
97+
| `element.format` | Format type for element properties. | `"element-properties"` |
98+
99+
**Destination Configuration**:
100+
101+
| Key | Description | Example Value |
102+
|--------------------|-----------------------------------------------------------------------------|----------------------------------------|
103+
| `destination` | Configuration for output destinations of the pipeline. | Object containing metadata and frame settings |
104+
| `metadata.type` | The protocol type for sending metadata information. | `"mqtt"` |
105+
| `metadata.topic` | The MQTT topic where vision classification results are published. | `"vision_weld_defect_classification"` |
106+
| `frame.type` | The protocol type for streaming video frames. | `"webrtc"` |
107+
| `frame.peer-id` | Unique identifier for the WebRTC peer connection. | `"samplestream"` |
108+
---
109+
75110
**Time Series Analytics Microservice** uses the User Defined Function(UDF) deployment package(TICK Scripts, UDFs, Models) which is already built-in to the container image. The UDF deployment package is available
76111
at `edge-ai-suites/manufacturing-ai-suite/industrial-edge-insights-multimodal/config/time-series-analytics-microservice`. Directory details is as below:
77112

@@ -240,5 +275,5 @@ Use the following command to verify that all containers are active and error-fre
240275
## Advanced setup
241276
242277
- [How to build from source and deploy](./how-to-build-from-source.md): Guide to build from source and docker compose deployment
243-
- [How to configure OPC-UA/MQTT alerts](./how-to-configure-alerts.md): Guide for configuring the OPC-UA/MQTT alerts in the Time Series Analytics microservice
278+
- [How to configure MQTT alerts](./how-to-configure-alerts.md): Guide for configuring the MQTT alerts in the Time Series Analytics microservice
244279
- [How to configure custom UDF deployment package](./how-to-configure-custom-udf.md): Guide for deploying a customized UDF deployment package (udfs/models/tick scripts)

manufacturing-ai-suite/industrial-edge-insights-multimodal/docs/user-guide/weld-defect-detection/how-to-build-from-source.md

Lines changed: 13 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,17 @@ before proceeding with the below steps.
77

88
## Steps to Build from Source
99

10-
1. **Clone the source and build the `Time Series Analytics` microservice**:
10+
1. **Clone the source and build the `DLStreamer Pipeline Server` microservice**:
11+
12+
```bash
13+
git clone https://github.com/open-edge-platform/edge-ai-libraries.git
14+
cd edge-ai-libraries/microservices/dlstreamer-pipeline-server/docker
15+
16+
# build
17+
docker compose build
18+
```
19+
20+
2. **Clone the source and build the `Time Series Analytics` microservice**:
1121

1222
```bash
1323
git clone https://github.com/open-edge-platform/edge-ai-libraries.git
@@ -24,7 +34,7 @@ before proceeding with the below steps.
2434
> docker compose build --build-arg COPYLEFT_SOURCES=true
2535
> ```
2636

27-
2. **Clone the source and build the sample app**:
37+
3. **Clone the source and build the sample app**:
2838

2939
```bash
3040
git clone https://github.com/open-edge-platform/edge-ai-suites.git
@@ -41,6 +51,6 @@ before proceeding with the below steps.
4151
> make build_copyleft_sources
4252
> ```
4353

44-
2. **Docker compose deployment and Verification**:
54+
4. **Docker compose deployment and Verification**:
4555

4656
Follow the remaining steps/sections starting from [docker compose deployment](./get-started.md#deploy-with-docker-compose)

manufacturing-ai-suite/industrial-edge-insights-multimodal/docs/user-guide/weld-defect-detection/system-requirements.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@ This page provides detailed hardware, software, and platform requirements to hel
66
## Minimum Requirements
77
| Component | Specification |
88
|---------------------|-------------------------------------------------------------------------------------------------|
9-
| **Processor** | 11th Gen Intel® Core™ or Intel® Xeon® processor with Intel® HD Graphics |
9+
| **Processor** | 13th Gen Intel® Core™ or Intel® Xeon® processor with Intel® HD Graphics |
1010
| **RAM** | Minimum 16 GB |
1111
| **Storage** | Minimum 64 GB (128 GB recommended) |
1212

0 commit comments

Comments
 (0)