Skip to content

Commit 7987d4c

Browse files
ymodlinymodlin
authored
Fix readme (realsenseai#309)
1 parent cfcefcf commit 7987d4c

3 files changed

Lines changed: 366 additions & 36 deletions

File tree

‎README.md‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,7 @@ The system shall include:
1515

1616
# Documentation
1717

18-
- NVIDIA® Jetson AGX Orin™ board setup - AGX Orin™ [JetPack 6.x](./README_JP6.md) setup guide
18+
- NVIDIA® Jetson AGX Orin™ board setup - AGX Orin™ [JetPack 6.0](./README_JP6.0.md) setup guide
1919
- NVIDIA® Jetson AGX Xavier™ board setup - AGX Xavier™ [JetPack 5.x.2](./README_JP5.md) setup guide
2020
- NVIDIA® Jetson AGX Xavier™ board setup - AGX Xavier™ [JetPack 4.6.1](./README_JP4.md) setup guide
2121
- Build Tools manual page [Build Manual page](./README_tools.md)
@@ -29,7 +29,8 @@ Please follow the [instruction](https://docs.nvidia.com/sdk-manager/install-with
2929

3030
# JetPack build
3131

32-
- [JetPack 6.x](./README_JP6.md#build-environment-prerequisites)
32+
- [JetPack 6.0](./README_JP6.0.md#build-environment-prerequisites)
33+
- [JetPack 6.2](./README_JP6.2.md#build-environment-prerequisites)
3334
- [JetPack 5.x.2](./README_JP5.md#build-kernel-dtb-and-d457-driver)
3435
- [JetPack 4.6.1](./README_JP4.md#build-kernel-dtb-and-d457-driver)
3536

‎README_JP6.0.md‎

Lines changed: 320 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,320 @@
1+
# Intel® RealSense™ camera driver for GMSL* interface
2+
3+
# D457 MIPI on NVIDIA® Jetson AGX Orin™ JetPack 6.x
4+
The Intel® RealSense™ MIPI platform driver enables the user to control and stream RealSense™ 3D MIPI cameras.
5+
The system shall include:
6+
* NVIDIA® Jetson™ platform Supported JetPack versions are:
7+
- [6.2 production release](https://developer.nvidia.com/embedded/jetpack-sdk-62)
8+
- [6.1 production release](https://developer.nvidia.com/embedded/jetpack-sdk-61)
9+
- [6.0 production release](https://developer.nvidia.com/embedded/jetpack-sdk-60)
10+
* RealSense™ [De-Serialize board](https://store.intelrealsense.com/buy-intel-realsense-des457.html)
11+
* Jetson AGX Orin™ Passive adapter board from [Leopard Imaging® LI-JTX1-SUB-ADPT](https://leopardimaging.com/product/accessories/adapters-carrier-boards/for-nvidia-jetson/li-jtx1-sub-adpt/)
12+
* RS MIPI camera [D457](https://store.intelrealsense.com/buy-intel-realsense-depth-camera-d457.html)
13+
14+
![orin_adapter](https://github.com/dmipx/realsense_mipi_platform_driver/assets/104717350/524e3eb6-6e6b-41cf-9562-9c0f920dd821)
15+
16+
17+
> Note: This MIPI reference driver is based on RealSense™ de-serialize board. For other de-serialize boards, modification might be needed.
18+
19+
### Links
20+
- Intel® RealSense™ camera driver for GMSL* interface [Front Page](./README.md)
21+
- NVIDIA® Jetson AGX Orin™ board setup - AGX Orin™ [JetPack 6.0](./README_JP6.0.md) setup guide
22+
- NVIDIA® Jetson AGX Xavier™ board setup - AGX Xavier™ [JetPack 5.x.2](./README_JP5.md) setup guide
23+
- NVIDIA® Jetson AGX Xavier™ board setup - AGX Xavier™ [JetPack 4.6.1](./README_JP4.md) setup guide
24+
- Build Tools manual page [Build Manual page](./README_tools.md)
25+
- Driver API manual page [Driver API page](./README_driver.md)
26+
27+
## NVIDIA® Jetson AGX Orin™ board setup
28+
29+
Please follow the [instruction](https://docs.nvidia.com/sdk-manager/install-with-sdkm-jetson/index.html) to flash JetPack to the NVIDIA® Jetson AGX Orin™ with NVIDIA® SDK Manager or other methods NVIDIA provides. Make sure the board is ready to use.
30+
31+
32+
33+
## Build environment prerequisites
34+
```
35+
sudo apt-get install -y build-essential bc wget flex bison curl libssl-dev xxd tar
36+
```
37+
## Build NVIDIA® kernel drivers, dtb and D457 driver
38+
39+
1. Clone [realsense_mipi_platform_driver](https://github.com/IntelRealSense/realsense_mipi_platform_driver.git) repo.
40+
2. The developers can set up build environment, ARM64 compiler, kernel sources and NVIDIA's Jetson git repositories by using the setup script.
41+
3. Apply patches for kernel drivers, nvidia-oot module and tegra devicetree.
42+
4. Build project
43+
5. Apply build results to target (Jetson).
44+
6. Configure target.
45+
46+
Assuming building for 6.0. One can also build for 6.1, 6.2 just replace the last parameter.
47+
```
48+
git clone --branch dev --single-branch https://github.com/IntelRealSense/realsense_mipi_platform_driver.git
49+
cd realsense_mipi_platform_driver
50+
./setup_workspace.sh 6.0
51+
./apply_patches.sh 6.0
52+
./build_all.sh 6.0
53+
```
54+
Note: dev_dbg() log support will not be enabled by default. If needed, run the `./build_all.sh` script with `--dev-dbg` option like below.
55+
```
56+
./build_all.sh --dev-dbg 6.0
57+
```
58+
59+
60+
61+
## JetPack manual build - cross compile x86-64 (CI deploy)
62+
63+
[NVIDIA® JetPack 6.2: Jetson Linux 36.4.3](https://developer.nvidia.com/embedded/jetson-linux-r3643)
64+
1. Download Jetson Linux Driver Package - [JetPack 6.2 BSP sources](https://developer.nvidia.com/downloads/embedded/l4t/r36_release_v4.3/release/jetson_linux_r36.4.3_aarch64.tbz2)
65+
2. Download Toolchain ARM64 compiler - [Bootlin Toolchain gcc 11.3](https://developer.nvidia.com/downloads/embedded/l4t/r36_release_v3.0/toolchain/aarch64--glibc--stable-2022.08-1.tar.bz2)
66+
3. Apply patches for kernel drivers, nvidia-oot module and tegra devicetree.
67+
4. Build cross-compile project on host (Build PC) or natively on target (Jetson).
68+
5. Apply build results to target (Jetson).
69+
6. Configure target.
70+
71+
```
72+
# JetPack 6.0
73+
mkdir -p l4t-gcc/6.x
74+
cd ./l4t-gcc/6.x
75+
wget https://developer.nvidia.com/downloads/embedded/l4t/r36_release_v3.0/toolchain/aarch64--glibc--stable-2022.08-1.tar.bz2 -O aarch64--glibc--stable-final.tar.bz2
76+
tar xf aarch64--glibc--stable-final.tar.bz2 --strip-components 1
77+
cd ../..
78+
wget https://developer.nvidia.com/downloads/embedded/l4t/r36_release_v4.3/sources/public_sources.tbz2
79+
tar xjf public_sources.tbz2
80+
cd Linux_for_Tegra/source
81+
tar xjf kernel_src.tbz2
82+
tar xjf kernel_oot_modules_src.tbz2
83+
tar xjf nvidia_kernel_display_driver_source.tbz2
84+
cd ../..
85+
86+
./apply_patches_ext.sh 6.0 Linux_for_Tegra/source
87+
88+
cp ./nvidia-oot/Makefile Linux_for_Tegra/source
89+
cp ./kernel/kernel-jammy-src/Makefile Linux_for_Tegra/source/kernel
90+
91+
# build kernel, dtb and D457 driver
92+
./build_all.sh 6.0 ./Linux_for_Tegra/source
93+
```
94+
Note: dev_dbg() log support will not be enabled by default. If needed, run the `./build_all.sh` script with `--dev-dbg` option like below.
95+
```
96+
./build_all.sh --dev-dbg 6.0 ./Linux_for_Tegra/source
97+
```
98+
99+
## Archive JetPack 6.x build results (optional)
100+
For 6.0 the kernel version is 5.15.136-tegra.
101+
- kernel image : `images/6.0/rootfs/boot/Image`
102+
- dtb: `images/6.0/rootfs/boot/dtb/tegra234-p3737-0000+p3701-0000-nv.dtb`
103+
- dtb overlay: `images/6.0/rootfs/boot/tegra234-camera-d4xx-overlay.dtbo`
104+
- dtb dual camera overlay: `images/6.0/rootfs/boot/tegra234-camera-d4xx-overlay-dual.dtbo`
105+
- kernel modules: `images/6.0/rootfs/lib/modules/5.15.136-tegra`
106+
107+
## Backup JetPack 6.0 boot configuration and drivers (optional)
108+
```
109+
echo "Backup boot configuration"
110+
sudo cp /boot/tegra234-p3737-0000+p3701-0000-nv.dtb /boot/tegra234-p3737-0000+p3701-0000-nv-bkp.dtb
111+
# Note: If using a production board and not a dev kit copy the relevant dtb file below
112+
sudo cp /boot/tegra234-p3737-0000+p3701-0005-nv.dtb /boot/tegra234-p3737-0000+p3701-0005-nv-bkp.dtb
113+
```
114+
115+
## Deploy build results on Jetson target
116+
On build host, copy build results to the right places.
117+
Assuming user 'nvidia' on Jetson with ip: `10.0.0.116` (if building natively on Jetson use $USER@localhost):
118+
119+
```
120+
# Configuration files
121+
tar czf rootfs.tar.gz -C images/6.2/rootfs boot lib
122+
scp rootfs.tar.gz nvidia@10.0.0.116:
123+
```
124+
125+
## Install kernel drivers, extra modules and device-tree to Jetson AGX Orin
126+
127+
Following steps required:
128+
129+
1. Copy build artifacts:
130+
If you build locally use those commands:
131+
```
132+
sudo cp -r ./images/6.0/rootfs/lib/modules/5.15.136-tegra /lib/modules/.
133+
sudo cp ./images/6.0/rootfs/boot/tegra234-camera-d4xx-overlay.dtbo /boot/.
134+
sudo cp ./images/6.0/rootfs/boot/dtb/tegra234-p3737-0000+p3701-0000-nv.dtb /boot/dtb/.
135+
sudo cp ./images/6.0/rootfs/boot/Image /boot/
136+
```
137+
In case of scp copy from host use this commands:
138+
```
139+
tar xf rootfs.tar.gz
140+
sudo cp -r ./lib/modules/5.15.136-tegra /lib/modules/.
141+
sudo cp ./boot/tegra234-camera-d4xx-overlay.dtbo /boot/.
142+
sudo cp ./boot/dtb/tegra234-p3737-0000+p3701-0000-nv.dtb /boot/dtb/.
143+
sudo cp ./boot/Image /boot/
144+
```
145+
2. Run $ `sudo /opt/nvidia/jetson-io/jetson-io.py`, to exit choose save & reboot:
146+
1. Configure Jetson AGX CSI Connector
147+
2. Configure for compatible hardware
148+
3. Choose appropriate configuration:
149+
i. Jetson RealSense Camera D457
150+
ii. Jetson RealSense Camera D457 dual
151+
5. Choose to save & reboot
152+
153+
3. Enable and run depmod scan for "extra" & "kernel" modules
154+
```
155+
# enable extra & kernel modules
156+
# original file content: cat /etc/depmod.d/ubuntu.conf -- search updates ubuntu built-in
157+
sudo sed -i 's/search updates/search extra updates kernel/g' /etc/depmod.d/ubuntu.conf
158+
# update driver cache
159+
sudo depmod
160+
echo "d4xx" | sudo tee /etc/modules-load.d/d4xx.conf
161+
```
162+
4.
163+
Verify bootloader configuration
164+
```
165+
cat /boot/extlinux/extlinux.conf
166+
----<CUT>----
167+
LABEL JetsonIO
168+
MENU LABEL Custom Header Config: <CSI Jetson RealSense Camera D457>
169+
LINUX /boot/Image
170+
FDT /boot/dtb/kernel_tegra234-p3737-0000+p3701-0000-nv.dtb
171+
APPEND ${cbootargs} root=PARTUUID=bbb3b34e-......
172+
OVERLAYS /boot/tegra234-camera-d4xx-overlay.dtbo
173+
----<CUT>----
174+
```
175+
On Jetson target (user home folder) assuming backup step was followed:
176+
177+
### Verify driver loaded - on Jetson:
178+
- Driver API manual page [Driver API page](./README_driver.md)
179+
180+
```
181+
nvidia@ubuntu:~$ sudo dmesg | grep tegra-capture-vi
182+
[ 9.357521] platform 13e00000.host1x:nvcsi@15a00000: Fixing up cyclic dependency with tegra-capture-vi
183+
[ 9.419926] tegra-camrtc-capture-vi tegra-capture-vi: ep of_device is not enabled endpoint.
184+
[ 9.419932] tegra-camrtc-capture-vi tegra-capture-vi: ep of_device is not enabled endpoint.
185+
[ 10.001170] tegra-camrtc-capture-vi tegra-capture-vi: subdev DS5 mux 9-001a bound
186+
[ 10.025295] tegra-camrtc-capture-vi tegra-capture-vi: subdev DS5 mux 12-001a bound
187+
[ 10.040934] tegra-camrtc-capture-vi tegra-capture-vi: subdev DS5 mux 13-001a bound
188+
[ 10.056151] tegra-camrtc-capture-vi tegra-capture-vi: subdev DS5 mux 14-001a bound
189+
[ 10.288088] tegra-camrtc-capture-vi tegra-capture-vi: subdev 13e00000.host1x:nvcsi@15a00000- bound
190+
[ 10.324025] tegra-camrtc-capture-vi tegra-capture-vi: subdev 13e00000.host1x:nvcsi@15a00000- bound
191+
[ 10.324631] tegra-camrtc-capture-vi tegra-capture-vi: subdev 13e00000.host1x:nvcsi@15a00000- bound
192+
[ 10.325056] tegra-camrtc-capture-vi tegra-capture-vi: subdev 13e00000.host1x:nvcsi@15a00000- bound
193+
194+
nvidia@ubuntu:~$ sudo dmesg | grep d4xx
195+
[ 9.443608] d4xx 9-001a: Probing driver for D45x
196+
[ 9.983168] d4xx 9-001a: ds5_chrdev_init() class_create
197+
[ 9.989521] d4xx 9-001a: D4XX Sensor: DEPTH, firmware build: 5.15.1.0
198+
[ 10.007813] d4xx 12-001a: Probing driver for D45x
199+
[ 10.013899] d4xx 12-001a: D4XX Sensor: RGB, firmware build: 5.15.1.0
200+
[ 10.025787] d4xx 13-001a: Probing driver for D45x
201+
[ 10.029095] d4xx 13-001a: D4XX Sensor: Y8, firmware build: 5.15.1.0
202+
[ 10.041282] d4xx 14-001a: Probing driver for D45x
203+
[ 10.044759] d4xx 14-001a: D4XX Sensor: IMU, firmware build: 5.15.1.0
204+
205+
```
206+
207+
### Known issues
208+
- Camera not recognized
209+
Verify I2C MUX detected. If "probe failed" reported, replace extension board adapter (LI-JTX1-SUB-ADPT).
210+
```
211+
nvidia@ubuntu:~$ sudo dmesg | grep pca954x
212+
[ 3.933113] pca954x 2-0072: probe failed
213+
```
214+
215+
- Configuration with jetson-io tool system fail to boot with message "couldn't find root partition"
216+
Verify bootloader configuration
217+
`/boot/extlinux/extlinux.conf`
218+
Sometimes configuration tool missing APPEND parameters. Duplicate `primary` section `APPEND` line to `JetsonIO` `APPEND` section, verify it's similar.
219+
220+
Example Bad:
221+
```
222+
LABEL primary
223+
MENU LABEL primary kernel
224+
LINUX /boot/Image
225+
INITRD /boot/initrd
226+
APPEND ${cbootargs} root=PARTUUID=634b7e44-aacc-4dd9-a769-3a664b83b159 rw rootwait rootfstype=ext4 mminit_loglevel=4 console=ttyTCU0,115200 console=ttyAMA0,115200 firmware_class.path=/etc/firmware fbcon=map:0 net.ifnames=0 nospectre_bhb video=efifb:off console=tty0 nv-auto-config
227+
228+
LABEL JetsonIO
229+
MENU LABEL Custom Header Config: <CSI Jetson RealSense Camera D457 dual>
230+
LINUX /boot/Image
231+
FDT /boot/dtb/kernel_tegra234-p3737-0000+p3701-0000-nv.dtb
232+
INITRD /boot/initrd
233+
APPEND ${cbootargs}
234+
OVERLAYS /boot/tegra234-camera-d4xx-overlay-dual.dtbo
235+
```
236+
Example Good:
237+
```
238+
LABEL primary
239+
MENU LABEL primary kernel
240+
LINUX /boot/Image
241+
INITRD /boot/initrd
242+
APPEND ${cbootargs} root=PARTUUID=634b7e44-aacc-4dd9-a769-3a664b83b159 rw rootwait rootfstype=ext4 mminit_loglevel=4 console=ttyTCU0,115200 console=ttyAMA0,115200 firmware_class.path=/etc/firmware fbcon=map:0 net.ifnames=0 nospectre_bhb video=efifb:off console=tty0 nv-auto-config
243+
244+
LABEL JetsonIO
245+
MENU LABEL Custom Header Config: <CSI Jetson RealSense Camera D457 dual>
246+
LINUX /boot/Image
247+
FDT /boot/dtb/kernel_tegra234-p3737-0000+p3701-0000-nv.dtb
248+
INITRD /boot/initrd
249+
APPEND ${cbootargs} root=PARTUUID=634b7e44-aacc-4dd9-a769-3a664b83b159 rw rootwait rootfstype=ext4 mminit_loglevel=4 console=ttyTCU0,115200 console=ttyAMA0,115200 firmware_class.path=/etc/firmware fbcon=map:0 net.ifnames=0 nospectre_bhb video=efifb:off console=tty0 nv-auto-config
250+
OVERLAYS /boot/tegra234-camera-d4xx-overlay-dual.dtbo
251+
```
252+
- Configuration tool jetson-io terminates without configuration menu.
253+
verify that `/boot/dtb` has only one dtb file
254+
```
255+
nvidia@ubuntu:~$ ls /boot/dtb/
256+
kernel_tegra234-p3737-0000+p3701-0000-nv.dtb
257+
```
258+
259+
- kernel does not recognize the I2C device
260+
```
261+
# Make sure which Jetson Carrier board is used:
262+
# p3701-0000 → Dev kit carrier board
263+
# p3701-0005 → Production carrier board or custom carrier
264+
# if you have the *0005* board, replace the relevant dtb file in in the instructions above
265+
266+
Example:
267+
sudo cat /proc/device-tree/compatible
268+
269+
Output:
270+
nvidia,p3701-0000
271+
```
272+
### Notes
273+
- With the introduction of meta data support for depth IR starting from release r/1.0.1.27, calibration format streaming requires a separate DTB that disables meta data since the camera does not metadata in calibration mode.
274+
- If calibration is needed, it's recommended to add `d457_calib` boot option as shown below.
275+
276+
```
277+
LABEL primary
278+
MENU LABEL primary kernel
279+
LINUX /boot/Image
280+
INITRD /boot/initrd
281+
APPEND ${cbootargs} root=PARTUUID=634b7e44-aacc-4dd9-a769-3a664b83b159 rw rootwait rootfstype=ext4 mminit_loglevel=4 console=ttyTCU0,115200 console=ttyAMA0,115200 firmware_class.path=/etc/firmware fbcon=map:0 net.ifnames=0 nospectre_bhb video=efifb:off console=tty0 nv-auto-config
282+
283+
LABEL JetsonIO
284+
MENU LABEL Custom Header Config: <CSI Jetson RealSense Camera D457 dual>
285+
LINUX /boot/Image
286+
FDT /boot/dtb/kernel_tegra234-p3737-0000+p3701-0000-nv.dtb
287+
INITRD /boot/initrd
288+
APPEND ${cbootargs} root=PARTUUID=634b7e44-aacc-4dd9-a769-3a664b83b159 rw rootwait rootfstype=ext4 mminit_loglevel=4 console=ttyTCU0,115200 console=ttyAMA0,115200 firmware_class.path=/etc/firmware fbcon=map:0 net.ifnames=0 nospectre_bhb video=efifb:off console=tty0 nv-auto-config
289+
OVERLAYS /boot/tegra234-camera-d4xx-overlay-dual.dtbo
290+
291+
LABEL JetsonIO_calib
292+
MENU LABEL Custom Header Config: <CSI Jetson RealSense Camera D457 dual - Calibration>
293+
LINUX /boot/Image
294+
FDT /boot/dtb/kernel_tegra234-p3737-0000+p3701-0000-nv.dtb
295+
INITRD /boot/initrd
296+
APPEND ${cbootargs} root=PARTUUID=634b7e44-aacc-4dd9-a769-3a664b83b159 rw rootwait rootfstype=ext4 mminit_loglevel=4 console=ttyTCU0,115200 console=ttyAMA0,115200 firmware_class.path=/etc/firmware fbcon=map:0 net.ifnames=0 nospectre_bhb video=efifb:off console=tty0 nv-auto-config
297+
OVERLAYS /boot/tegra234-camera-d4xx-overlay-dual.calib.dtbo
298+
```
299+
300+
- To generate `tegra234-camera-d4xx-overlay.calib.dtbo` and `tegra234-camera-d4xx-overlay-dual.calib.dtbo` on the Host build system, replace the contents of the below files:
301+
`tegra234-camera-d4xx-overlay.dts` by the contents of `tegra234-camera-d4xx-overlay.calib.dts`
302+
303+
`tegra234-camera-d4xx-overlay-dual.dts` by the contents of `tegra234-camera-d4xx-overlay-dual.calib.dts`
304+
305+
- Reset and reapply patches and rebuild driver:
306+
```
307+
./apply_patches.sh 6.0 reset
308+
309+
./apply_patches.sh 6.0
310+
311+
./build_all.sh 6.0
312+
```
313+
314+
- Deploy 2 DTBO files on the Jetson:
315+
- In `./images/6.0/rootfs/boot/` folder, locate and rename the following DTBO files:
316+
- Rename `tegra234-camera-d4xx-overlay.dtbo` to `tegra234-camera-d4xx-overlay.calib.dtbo`
317+
- Rename `tegra234-camera-d4xx-overlay-dual.dtbo` to `tegra234-camera-d4xx-overlay-dual.calib.dtbo`
318+
- Copy the two DTBO files from the build Host to `/boot/` on the Jetson.
319+
320+
---

0 commit comments

Comments
 (0)