Skip to content

Commit c9a595b

Browse files
mohitmeh12jouillet
andauthored
Co-authored-by: Jeremy Ouillette <jeremy.ouillette@intel.com>
1 parent 8e7abac commit c9a595b

17 files changed

Lines changed: 1554 additions & 87 deletions

robotics-ai-suite/docs/robotics/dev_guide/gmsl-guide/index.rst

Lines changed: 1480 additions & 0 deletions
Large diffs are not rendered by default.
Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
.. _gmsl-guide-index:
2+
3+
GMSL Guide
4+
===========================
5+
6+
Prerequisite: Follow the instructions in :doc:`../gsg_robot/index`.
7+
8+
With step-by-step instructions on how to use and configure GMSL cameras, this guide provides a learning path for developers to use GMSL cameras in their robotics applications.
9+
10+
11+
.. toctree::
12+
:maxdepth: 1
13+
14+
gmsl-guide/index
15+

robotics-ai-suite/docs/robotics/dev_guide/tutorials_amr/perception/openvino/pyrealsense2_d457_multicam_object_detection_tutorial.md

Lines changed: 57 additions & 86 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,10 @@
1-
# OpenVINO™ Tutorial on Multi-camera Object Detection using RealSense™ Depth Camera D457
1+
# OpenVINO™ Tutorial on Multi-camera Object Detection using RealSense™ Depth Camera D457 and D3CMCXXX-115-084 Camera
22

33
In this tutorial, the multi-camera use case is demonstrated using an
44
[Axiomtek Robox500 ROS 2 AMR Controller](https://www.axiomtek.com/Default.aspx?MenuId=Products&FunctionId=ProductView&ItemId=27392&C=ROBOX500&upcat=408)
55
and four
6-
[RealSense™ Depth Camera D457](https://www.realsenseai.com/products/d457-gmsl-fakra/).
6+
[RealSense™ Depth Camera D457](https://www.realsenseai.com/products/d457-gmsl-fakra/) or
7+
[D3CMCXXX-115-084](https://www.d3embedded.com/product/isx031-smart-camera-medium-fov-gmsl2-unsealed/).
78
Here, the four cameras are connected to the Industrial Gigabit Multimedia Serial
89
Link™ (GMSL) supported Axiomtek Robox500 ROS 2 AMR Controller through GMSL/FAKRA
910
(female-to-female) cables, which provide high-bandwidth video transmission.
@@ -29,8 +30,8 @@ The setup looks like as described in the table below.
2930
|Camera-3|YOLOv8n:FP16|Object detection|GPU|
3031
|Camera-4|YOLOv8n-seg:FP16|Object detection & segmentation|GPU|
3132

32-
The following steps are required in order to set up the Axiomtek Robox500 ROS 2
33-
AMR Controller to support four Intel® RealSense™ Depth Camera D457.
33+
The following steps are required in order to run the MultiCam Demo
34+
AMR Controller to support four Intel® RealSense™ Depth Camera D457, and D3CMCXXX-115-084
3435

3536
## Source Code
3637

@@ -41,123 +42,82 @@ The source code of this component can be found here:
4142

4243
Complete the [get started guide](../../../../gsg_robot/index.md) before continuing.
4344

44-
## Axiomtek Robox500 ROS 2 AMR Controller Setup
45+
Complete the [GMSL setup guide](../../../gmsl-guide/index.rst) before continuing.
4546

46-
Connect four Intel® RealSense™ Depth Camera D457 to the Axiomtek Robox500 ROS 2
47-
AMR Controller as shown in the below picture. Now, power-on the target.
4847

49-
![Axiomtek_GMSL_Camera](../../../../images/Axiomtek_GMSL_Camera.jpg)
5048

51-
> **Note:** Select the "MIPI" mode of the Intel® RealSense™ Depth Camera D457 by
49+
> **Note:** If using D457 select the "MIPI" mode of the Intel® RealSense™ Depth Camera D457 by
5250
> moving the select switch on the camera to "M", as shown in the picture below.
5351
5452
![MIPI_USB_Switch_in_D457](../../../../images/MIPI_USB_Switch_in_D457.jpeg)
5553

56-
### Install GPU driver
5754

58-
From the setup description explained above, three out of the four instances of
59-
the AI-based applications are run on a GPU. Therefore, the appropriate
60-
GPU drivers need to be installed.
61-
62-
Run the below command to check for the GPU device.
63-
64-
```bash
65-
# Install clinfo
66-
sudo apt install -y clinfo
67-
68-
# Run clinfo command to check GPU Device
69-
clinfo | grep -i "Device Name"
70-
Device Name Intel(R) UHD Graphics
71-
Device Name 13th Gen Intel(R) Core(TM) i7-1370PE
72-
Device Name Intel(R) UHD Graphics
73-
Device Name Intel(R) UHD Graphics
74-
Device Name Intel(R) UHD Graphics
75-
```
76-
77-
If no GPU device is listed, then the GPU driver is not installed.
78-
79-
> **Note:**
80-
> If the above GPU driver is not installed, then follow the steps described in the documentation
81-
> [Enable Intel® Level Zero and OpenCL™ Graphics Compute Runtime](https://eci.intel.com/docs/3.3/development/tutorials/enable-graphics.html#enable-intel-level-zero-and-opencl-graphics-compute-runtime)
82-
> (excluding the **step#2** to install the ``linux-intel-rt``, which is
83-
> not required for this tutorial) to install the same.
84-
85-
Verify that the GPU driver is installed using the previous ``clinfo`` command.
8655

8756
### Install ``librealsense2`` and ``realsense2`` tools
8857

89-
<!--hide_directive::::{tab-set}hide_directive-->
90-
<!--hide_directive:::{tab-item}hide_directive--> **Jazzy**
91-
<!--hide_directive:sync: jazzyhide_directive-->
58+
::::{tab-set}
59+
:::{tab-item} **Jazzy**
60+
:sync: jazzy
9261

9362
```bash
9463
sudo apt install -y ros-jazzy-librealsense2-tools
9564
```
9665

97-
<!--hide_directive:::hide_directive-->
98-
<!--hide_directive:::{tab-item}hide_directive--> **Humble**
99-
<!--hide_directive:sync: humblehide_directive-->
66+
:::
67+
:::{tab-item} **Humble**
68+
:sync: humble
10069

10170
```bash
10271
sudo apt install -y ros-humble-librealsense2-tools
10372
```
10473

105-
<!--hide_directive:::hide_directive-->
106-
<!--hide_directive::::hide_directive-->
107-
108-
### Configure the Intel’s GMSL Serializer-Deserializer ACPI devices and install ``intel-ipu6-dkms`` Dynamic Kernel Module Support package
109-
110-
The following steps describe how to configure the Intel’s GMSL
111-
Serializer-Deserializer ACPI devices and further to install and load the
112-
``intel-ipu6-dkms`` Dynamic Kernel Module Support package.
74+
:::
75+
::::
11376

114-
1. The design approach of the GMSL Add-in-Card present in the Axiomtek Robox500
115-
ROS 2 AMR Controller is called ``Standalone-mode``.
77+
### Load the Intel IPU Driver ###
78+
::::{tab-set}
79+
:::{tab-item} **IPU7**
80+
:sync: ipu7
11681

117-
That is, a single GMSL Serializer and Camera Sensor device is connected per
118-
Deserializer. In order to configure the Intel’s GMSL Serializer-Deserializer
119-
ACPI devices in ``Standalone-mode``, follow the steps described in
120-
[Configure Intel® GMSL SerDes ACPI devices](https://eci.intel.com/docs/3.3/development/tutorials/enable-gmsl.html#configure-intel-gmsl-serdes-acpi-devices).
121-
122-
2. Download and install the ``intel-ipu6-dkms`` Dynamic Kernel Module
123-
Support package.
82+
```bash
83+
sudo modprobe intel-ipu7-isys
84+
```
12485

125-
Follow the steps described in
126-
[Intel® GMSL intel-ipu6 Debian kernel modules (DKMS)](https://eci.intel.com/docs/3.3/development/tutorials/enable-gmsl.html#intel-gmsl-intel-ipu6-debian-kernel-modules-dkms).
86+
:::
87+
:::{tab-item} **IPU6**
88+
:sync: ipu6
12789

128-
3. To load the ``intel-ipu6`` kernel modules after installation and to enable
129-
the Intel® RealSense™ Depth Camera D457, follow the steps described in
130-
[Enable ROS2 Intel® RealSense™ Depth Camera D457 GMSL](https://eci.intel.com/docs/3.3/development/tutorials/enable-gmsl.html#enable-ros2-intel-realsense-depth-camera-d457-gmsl).
90+
```bash
91+
sudo modprobe intel-ipu6-isys
92+
```
13193

132-
> **Note:**
133-
> The steps, such as BIOS settings and d4xx module user parameters, must be
134-
> configured to be relevant to the ``Standalone-mode`` of the Add-in-Card for
135-
> Axiomtek Robox500 ROS 2 AMR Controller.
94+
:::
95+
::::
13696

13797
## Install and run multi-camera object detection tutorial using the Intel® RealSense™ Depth Camera D457
13898

13999
### Install
140100

141101
Install the multi-camera object detection tutorial by using the following command.
142102

143-
<!--hide_directive::::{tab-set}hide_directive-->
144-
<!--hide_directive:::{tab-item}hide_directive--> **Jazzy**
145-
<!--hide_directive:sync: jazzyhide_directive-->
103+
::::{tab-set}
104+
:::{tab-item} **Jazzy**
105+
:sync: jazzy
146106

147107
```bash
148108
sudo apt install -y ros-jazzy-pyrealsense2-ai-demo
149109
```
150110

151-
<!--hide_directive:::hide_directive-->
152-
<!--hide_directive:::{tab-item}hide_directive--> **Humble**
153-
<!--hide_directive:sync: humblehide_directive-->
111+
:::
112+
:::{tab-item} **Humble**
113+
:sync: humble
154114

155115
```bash
156116
sudo apt install -y ros-humble-pyrealsense2-ai-demo
157117
```
158118

159-
<!--hide_directive:::hide_directive-->
160-
<!--hide_directive::::hide_directive-->
119+
:::
120+
::::
161121

162122
> **Note:** The `pyrealsense2-ai-demo` installation will also do the following:
163123
>
@@ -170,24 +130,35 @@ sudo apt install -y ros-humble-pyrealsense2-ai-demo
170130

171131
Run the below commands to start the tutorial.
172132

173-
<!--hide_directive::::{tab-set}hide_directive-->
174-
<!--hide_directive:::{tab-item}hide_directive--> **Jazzy**
175-
<!--hide_directive:sync: jazzyhide_directive-->
133+
::::{tab-set}
134+
:::{tab-item} **Jazzy**
135+
:sync: jazzy
176136

177137
```bash
178138
# Activate the pyrealsense2-ai-demo python environment
179139
. /opt/ros/jazzy/share/pyrealsense2-ai-demo/venv/bin/activate
180140

181141
# Source the ros2 jazzy
182142
source /opt/ros/jazzy/setup.bash
143+
```
144+
145+
**D457:**
183146

147+
```bash
184148
# Run the pyrealsense2-ai-demo tutorial for four camera input streams
185149
python3 /opt/ros/jazzy/bin/pyrealsense2_ai_demo_launcher.py --config=/opt/ros/jazzy/share/pyrealsense2-ai-demo/config/config_ros2_v4l2_rs-color-0_3.js
186150
```
187151

188-
<!--hide_directive:::hide_directive-->
189-
<!--hide_directive:::{tab-item}hide_directive--> **Humble**
190-
<!--hide_directive:sync: humblehide_directive-->
152+
**D3CMCXXX-115-084:**
153+
154+
```bash
155+
# Run the pyrealsense2-ai-demo tutorial for four camera input streams
156+
python3 /opt/ros/jazzy/bin/pyrealsense2_ai_demo_launcher.py --config=/opt/ros/jazzy/share/pyrealsense2-ai-demo/config/config_isx031_4cameras.js
157+
```
158+
159+
:::
160+
:::{tab-item} **Humble**
161+
:sync: humble
191162

192163
```bash
193164
# Activate the pyrealsense2-ai-demo python environment
@@ -200,8 +171,8 @@ source /opt/ros/humble/setup.bash
200171
python3 /opt/ros/humble/bin/pyrealsense2_ai_demo_launcher.py --config=/opt/ros/humble/share/pyrealsense2-ai-demo/config/config_ros2_v4l2_rs-color-0_3.js
201172
```
202173

203-
<!--hide_directive:::hide_directive-->
204-
<!--hide_directive::::hide_directive-->
174+
:::
175+
::::
205176

206177
All the four cameras are started after approximately 15-20 secs, as shown in the below picture.
207178

131 KB
Loading
749 KB
Loading
814 KB
Loading
814 KB
Loading
76.7 KB
Loading
703 KB
Loading
761 KB
Loading

0 commit comments

Comments
 (0)