RealSense™ camera driver for GMSL* interface

D457 MIPI on NVIDIA® Jetson AGX Orin™ JetPack 6.x

The RealSense™ MIPI platform driver enables the user to control and stream RealSense™ 3D MIPI cameras. The system shall include:

NVIDIA® Jetson™ platform Supported JetPack versions are:
RealSense™ De-Serialize board
Jetson AGX Orin™ Passive adapter board from Leopard Imaging® LI-JTX1-SUB-ADPT
RS MIPI camera D457

Note: This MIPI reference driver is based on RealSense™ de-serialize board. For other de-serialize boards, modification might be needed.

NVIDIA® Jetson AGX Orin™ board setup

Please follow the instruction to flash JetPack to the NVIDIA® Jetson AGX Orin™ with NVIDIA® SDK Manager or other methods NVIDIA provides. Make sure the board is ready to use.

Build environment prerequisites

sudo apt-get install -y build-essential bc wget flex bison curl libssl-dev xxd tar

Build NVIDIA® kernel drivers, dtb and D457 driver

Clone realsense_mipi_platform_driver repo.
The developers can set up build environment, ARM64 compiler, kernel sources and NVIDIA's Jetson git repositories by using the setup script.
Apply patches for kernel drivers, nvidia-oot module and tegra devicetree.
Build project
Apply build results to target (Jetson).
Configure target.

Assuming building for 6.2. One can also build for 6.1, 6.0 just replace the last parameter.

git clone --branch dev --single-branch https://github.com/realsenseai/realsense_mipi_platform_driver.git
cd realsense_mipi_platform_driver
./setup_workspace.sh 6.2
./apply_patches.sh 6.2
./build_all.sh 6.2

Note: dev_dbg() log support will not be enabled by default. If needed, run the ./build_all.sh script with --dev-dbg option like below.

./build_all.sh --dev-dbg 6.2

JetPack manual build - cross compile x86-64 (CI deploy)

NVIDIA® JetPack 6.2: Jetson Linux 36.4.3

Download Jetson Linux Driver Package - JetPack 6.2 BSP sources
Download Toolchain ARM64 compiler - Bootlin Toolchain gcc 11.3
Apply patches for kernel drivers, nvidia-oot module and tegra devicetree.
Build cross-compile project on host (Build PC) or natively on target (Jetson).
Apply build results to target (Jetson).
Configure target.

# JetPack 6.2
mkdir -p l4t-gcc/6.x
cd ./l4t-gcc/6.x
wget https://developer.nvidia.com/downloads/embedded/l4t/r36_release_v3.0/toolchain/aarch64--glibc--stable-2022.08-1.tar.bz2 -O aarch64--glibc--stable-final.tar.bz2
tar xf aarch64--glibc--stable-final.tar.bz2 --strip-components 1
cd ../..
wget https://developer.nvidia.com/downloads/embedded/l4t/r36_release_v4.3/sources/public_sources.tbz2
tar xjf public_sources.tbz2
cd Linux_for_Tegra/source
tar xjf kernel_src.tbz2
tar xjf kernel_oot_modules_src.tbz2
tar xjf nvidia_kernel_display_driver_source.tbz2
cd ../..

./apply_patches_ext.sh 6.2 Linux_for_Tegra/source

cp ./nvidia-oot/Makefile Linux_for_Tegra/source
cp ./kernel/kernel-jammy-src/Makefile Linux_for_Tegra/source/kernel

# build kernel, dtb and D457 driver
./build_all.sh 6.2 ./Linux_for_Tegra/source

Note: dev_dbg() log support will not be enabled by default. If needed, run the ./build_all.sh script with --dev-dbg option like below.

./build_all.sh --dev-dbg 6.2 ./Linux_for_Tegra/source

Archive JetPack 6.x build results (optional)

Assuming 6.2 (or 6.1) build the kernel version is 5.15.148-tegra. For 6.0 the kernel version is 5.15.136-tegra.

kernel image : images/6.2/rootfs/boot/Image
dtb: images/6.2/rootfs/boot/dtb/tegra234-p3737-0000+p3701-0000-nv.dtb
dtb overlay: images/6.2/rootfs/boot/tegra234-camera-d4xx-overlay.dtbo
dtb dual camera overlay: images/6.2/rootfs/boot/tegra234-camera-d4xx-overlay-dual.dtbo
dtb fg12-16ch camera overlay: images/6.0/rootfs/boot/tegra234-camera-d4xx-overlay-fg12-16ch.dtbo
dtb fg12-16ch dual camera overlay: images/6.0/rootfs/boot/tegra234-camera-d4xx-overlay-fg12-16ch-dual.dtbo
kernel modules: images/6.2/rootfs/lib/modules/5.15.148-tegra

Backup JetPack 6.2 boot configuration and drivers (optional)

echo "Backup boot configuration"
sudo cp /boot/tegra234-p3737-0000+p3701-0000-nv.dtb /boot/tegra234-p3737-0000+p3701-0000-nv-bkp.dtb
# Note: If using a production board and not a dev kit copy the relevant dtb file below
sudo cp /boot/tegra234-p3737-0000+p3701-0005-nv.dtb /boot/tegra234-p3737-0000+p3701-0005-nv-bkp.dtb

Install kernel drivers, extra modules and device-tree to Jetson AGX Orin

Following steps required:

Create "dev" directory in boot (in order to not override the default kernel)

sudo mkdir /boot/dev

Copy build artifacts: If you build locally use those commands:

sudo cp -r ./images/6.2/rootfs/lib/modules/5.15.148-tegra /lib/modules/.
sudo cp    ./images/6.2/rootfs/boot/tegra234-camera-d4xx-overlay*.dtbo /boot/dev/.
sudo cp    ./images/6.2/rootfs/boot/dtb/tegra234-p3737-0000+p3701-0000-nv.dtb /boot/dtb/.
# For production carrier boards (p3701-0005):
sudo cp    ./images/6.2/rootfs/boot/dtb/tegra234-p3737-0000+p3701-0005-nv.dtb /boot/dtb/.
sudo cp    ./images/6.2/rootfs/boot/Image /boot/dev/.

In case of scp copy from host use this commands:

tar xf rootfs.tar.gz
sudo cp -r ./lib/modules/5.15.148-tegra /lib/modules/.
sudo cp    ./boot/tegra234-camera-d4xx-overlay*.dtbo /boot/dev/.
sudo cp    ./boot/dtb/tegra234-p3737-0000+p3701-0000-nv.dtb /boot/dtb/.
# For production carrier boards (p3701-0005):
sudo cp    ./boot/dtb/tegra234-p3737-0000+p3701-0005-nv.dtb /boot/dtb/.
sudo cp    ./boot/Image /boot/dev/.

Run depmod

sudo depmod

Update initrd (regenerate kernel modules)

The kernel patches modify the I2C subsystem header (i2c.h), which changes the CRC of all exported I2C symbols. The boot initrd contains cached kernel modules that must be regenerated to match the new kernel, otherwise modules like ucsi_ccg will fail to load with "disagrees about version of symbol" errors.

sudo update-initramfs -u -k 5.15.148-tegra
sudo rm -f /boot/initrd
sudo ln -s /boot/initrd.img-5.15.148-tegra /boot/initrd

Modify bootloader configuration:

open /boot/extlinux/extlinux.conf for editing using your preferred editor
Copy existing primary kernel and rename the copy to "dev"
Change the "MENU LABEL" to a meaningful label (e.g "development kernel")
Change the "LINUX" line to point to the newly copied /boot/dev/Image
Add the "FDT" line pointing at the newly copied device tree "/boot/dtb/tegra234-p3737-0000+p3701-0000-nv.dtb" (or tegra234-p3737-0000+p3701-0005-nv.dtb for production boards)
add the "OVERLAYS" line pointing to the required overlay "tegra234-camera-d4xx-overlay/dual/else>.dtb
Select the new label as the default

The result should be:

...
DEFAULT dev

LABEL primary
    MENU LABEL primary kernel
    LINUX /boot/Image
    INITRD /boot/initrd
    APPEND ${cbootargs} root=...

LABEL dev
    MENU LABEL development kernel
    LINUX /boot/dev/Image
    INITRD /boot/initrd
    APPEND ${cbootargs} root=...
    FDT /boot/dtb/tegra234-p3737-0000+p3701-0000-nv.dtb
    OVERLAYS /boot/dev/tegra234-camera-d4xx-overlay.dtbo

Reboot

sudo reboot

For fg12-16ch/dual/quad, use tegra234-camera-d4xx-overlay-fg12-16ch</dual/quad>.dtbo

On Jetson target (user home folder) assuming backup step was followed:

Verify driver loaded - on Jetson:

Driver API manual page Driver API page

nvidia@ubuntu:~$ sudo dmesg | grep tegra-capture-vi
[    9.357521] platform 13e00000.host1x:nvcsi@15a00000: Fixing up cyclic dependency with tegra-capture-vi
[    9.419926] tegra-camrtc-capture-vi tegra-capture-vi: ep of_device is not enabled endpoint.
[    9.419932] tegra-camrtc-capture-vi tegra-capture-vi: ep of_device is not enabled endpoint.
[   10.001170] tegra-camrtc-capture-vi tegra-capture-vi: subdev DS5 mux 9-001a bound
[   10.025295] tegra-camrtc-capture-vi tegra-capture-vi: subdev DS5 mux 12-001a bound
[   10.040934] tegra-camrtc-capture-vi tegra-capture-vi: subdev DS5 mux 13-001a bound
[   10.056151] tegra-camrtc-capture-vi tegra-capture-vi: subdev DS5 mux 14-001a bound
[   10.288088] tegra-camrtc-capture-vi tegra-capture-vi: subdev 13e00000.host1x:nvcsi@15a00000- bound
[   10.324025] tegra-camrtc-capture-vi tegra-capture-vi: subdev 13e00000.host1x:nvcsi@15a00000- bound
[   10.324631] tegra-camrtc-capture-vi tegra-capture-vi: subdev 13e00000.host1x:nvcsi@15a00000- bound
[   10.325056] tegra-camrtc-capture-vi tegra-capture-vi: subdev 13e00000.host1x:nvcsi@15a00000- bound

nvidia@ubuntu:~$ sudo dmesg | grep d4xx
[    9.443608] d4xx 9-001a: Probing driver for D45x
[    9.983168] d4xx 9-001a: ds5_chrdev_init() class_create
[    9.989521] d4xx 9-001a: D4XX Sensor: DEPTH, firmware build: 5.15.1.0
[   10.007813] d4xx 12-001a: Probing driver for D45x
[   10.013899] d4xx 12-001a: D4XX Sensor: RGB, firmware build: 5.15.1.0
[   10.025787] d4xx 13-001a: Probing driver for D45x
[   10.029095] d4xx 13-001a: D4XX Sensor: Y8, firmware build: 5.15.1.0
[   10.041282] d4xx 14-001a: Probing driver for D45x
[   10.044759] d4xx 14-001a: D4XX Sensor: IMU, firmware build: 5.15.1.0

Known issues

Camera not recognized Verify I2C MUX detected. If "probe failed" reported, replace extension board adapter (LI-JTX1-SUB-ADPT).

nvidia@ubuntu:~$ sudo dmesg | grep pca954x
[    3.933113] pca954x 2-0072: probe failed

kernel does not recognize the I2C device

# Make sure which Jetson Carrier board is used:
#   p3701-0000 → Dev kit carrier board
#   p3701-0005 → Production carrier board or custom carrier
# if you have the *0005* board, replace the relevant dtb file in in the instructions above

Example: 
sudo cat /proc/device-tree/compatible

Output:
nvidia,p3701-0000

Notes

With the introduction of meta data support for depth IR starting from release r/1.0.1.27, calibration format streaming requires a separate DTB that disables meta data since the camera does not metadata in calibration mode.
If calibration is needed, it's recommended to add d457_calib boot option as shown below.

LABEL primary
    MENU LABEL primary kernel
    LINUX /boot/Image
    INITRD /boot/initrd
    APPEND ${cbootargs} root=PARTUUID=634b7e44-aacc-4dd9-a769-3a664b83b159 rw rootwait rootfstype=ext4 mminit_loglevel=4 console=ttyTCU0,115200 console=ttyAMA0,115200 firmware_class.path=/etc/firmware fbcon=map:0 net.ifnames=0 nospectre_bhb video=efifb:off console=tty0 nv-auto-config

LABEL JetsonIO
    MENU LABEL Custom Header Config: <CSI Jetson RealSense Camera D457 dual>
    LINUX /boot/Image
    FDT /boot/dtb/kernel_tegra234-p3737-0000+p3701-0000-nv.dtb
    INITRD /boot/initrd
    APPEND ${cbootargs} root=PARTUUID=634b7e44-aacc-4dd9-a769-3a664b83b159 rw rootwait rootfstype=ext4 mminit_loglevel=4 console=ttyTCU0,115200 console=ttyAMA0,115200 firmware_class.path=/etc/firmware fbcon=map:0 net.ifnames=0 nospectre_bhb video=efifb:off console=tty0 nv-auto-config
    OVERLAYS /boot/tegra234-camera-d4xx-overlay-dual.dtbo

LABEL JetsonIO_calib
    MENU LABEL Custom Header Config: <CSI Jetson RealSense Camera D457 dual - Calibration>
    LINUX /boot/Image
    FDT /boot/dtb/kernel_tegra234-p3737-0000+p3701-0000-nv.dtb
    INITRD /boot/initrd
    APPEND ${cbootargs} root=PARTUUID=634b7e44-aacc-4dd9-a769-3a664b83b159 rw rootwait rootfstype=ext4 mminit_loglevel=4 console=ttyTCU0,115200 console=ttyAMA0,115200 firmware_class.path=/etc/firmware fbcon=map:0 net.ifnames=0 nospectre_bhb video=efifb:off console=tty0 nv-auto-config
    OVERLAYS /boot/tegra234-camera-d4xx-overlay-dual.calib.dtbo

To generate tegra234-camera-d4xx-overlay.calib.dtbo and tegra234-camera-d4xx-overlay-dual.calib.dtbo on the Host build system, replace the contents of the below files: tegra234-camera-d4xx-overlay.dts by the contents of tegra234-camera-d4xx-overlay.calib.dts

tegra234-camera-d4xx-overlay-dual.dts by the contents of tegra234-camera-d4xx-overlay-dual.calib.dts
Reset and reapply patches and rebuild driver:

./apply_patches.sh 6.2 reset

./apply_patches.sh 6.2

./build_all.sh 6.2

Deploy 2 DTBO files on the Jetson:
In ./images/6.2/rootfs/boot/ folder, locate and rename the following DTBO files:
- Rename tegra234-camera-d4xx-overlay.dtbo to tegra234-camera-d4xx-overlay.calib.dtbo
- Rename tegra234-camera-d4xx-overlay-dual.dtbo to tegra234-camera-d4xx-overlay-dual.calib.dtbo
Copy the two DTBO files from the build Host to /boot/ on the Jetson.

Navigation Menu

Uh oh!

RealSense™ camera driver for GMSL* interface

D457 MIPI on NVIDIA® Jetson AGX Orin™ JetPack 6.x

Links

NVIDIA® Jetson AGX Orin™ board setup

Build environment prerequisites

Build NVIDIA® kernel drivers, dtb and D457 driver

JetPack manual build - cross compile x86-64 (CI deploy)

Archive JetPack 6.x build results (optional)

Backup JetPack 6.2 boot configuration and drivers (optional)

Install kernel drivers, extra modules and device-tree to Jetson AGX Orin

Verify driver loaded - on Jetson:

Known issues

Notes

Uh oh!

FilesExpand file tree

README_JP6.2.md

Latest commit

History

README_JP6.2.md

File metadata and controls

RealSense™ camera driver for GMSL* interface

D457 MIPI on NVIDIA® Jetson AGX Orin™ JetPack 6.x

Links

NVIDIA® Jetson AGX Orin™ board setup

Build environment prerequisites

Build NVIDIA® kernel drivers, dtb and D457 driver

JetPack manual build - cross compile x86-64 (CI deploy)

Archive JetPack 6.x build results (optional)

Backup JetPack 6.2 boot configuration and drivers (optional)

Install kernel drivers, extra modules and device-tree to Jetson AGX Orin

Verify driver loaded - on Jetson:

Known issues

Notes