Facing Issues getting RGB frames using Intel Realsense D435i on raspberry pi4 running ubuntu server 22.04 and ros humble

[parapara29]

Hi there, I am facing a weird issue on my when running the command:

ros2 launch realsense2_camera rs_launch.py device_type:=d435

On running this, the following is my terminal output:

[realsense2_camera_node-1]: process started with pid [12969]
[realsense2_camera_node-1] [INFO] [1715494534.811703285] [camera.camera]: RealSense ROS v4.54.1
[realsense2_camera_node-1] [INFO] [1715494534.811993027] [camera.camera]: Built with LibRealSense v2.54.1
[realsense2_camera_node-1] [INFO] [1715494534.812059612] [camera.camera]: Running with LibRealSense v2.54.1
[realsense2_camera_node-1] [INFO] [1715494534.887157295] [camera.camera]: Device with serial number 207522076538 was found.
[realsense2_camera_node-1] 
[realsense2_camera_node-1] [INFO] [1715494534.887414857] [camera.camera]: Device with physical ID /sys/devices/platform/scb/fd500000.pcie/pci0000:00/0000:00:00.0/0000:01:00.0/usb2/2-1/2-1:1.0/video4linux/video0 was found.
[realsense2_camera_node-1] [INFO] [1715494534.887475220] [camera.camera]: Device with name Intel RealSense D435I was found.
[realsense2_camera_node-1] [INFO] [1715494534.888246330] [camera.camera]: Device with port number 2-1 was found.
[realsense2_camera_node-1] [INFO] [1715494534.888472876] [camera.camera]: Device USB type: 3.2
[realsense2_camera_node-1] [INFO] [1715494534.891755496] [camera.camera]: getParameters...
[realsense2_camera_node-1] [INFO] [1715494534.892870934] [camera.camera]: JSON file is not provided
[realsense2_camera_node-1] [INFO] [1715494534.892992771] [camera.camera]: Device Name: Intel RealSense D435I
[realsense2_camera_node-1] [INFO] [1715494534.893050449] [camera.camera]: Device Serial No: 207522076538
[realsense2_camera_node-1] [INFO] [1715494534.893093499] [camera.camera]: Device physical port: /sys/devices/platform/scb/fd500000.pcie/pci0000:00/0000:00:00.0/0000:01:00.0/usb2/2-1/2-1:1.0/video4linux/video0
[realsense2_camera_node-1] [INFO] [1715494534.893136254] [camera.camera]: Device FW version: 5.15.0.2
[realsense2_camera_node-1] [INFO] [1715494534.893179341] [camera.camera]: Device Product ID: 0x0B3A
[realsense2_camera_node-1] [INFO] [1715494534.893218669] [camera.camera]: Sync Mode: Off
[realsense2_camera_node-1] [INFO] [1715494534.939557350] [camera.camera]: Set ROS param depth_module.profile to default: 848x480x30
[realsense2_camera_node-1] [WARN] [1715494534.952684180] [camera.camera]: Could not set param: rgb_camera.power_line_frequency with 3 Range: [0, 2]: parameter 'rgb_camera.power_line_frequency' could not be set: Parameter {rgb_camera.power_line_frequency} doesn't comply with integer range.
[realsense2_camera_node-1] [INFO] [1715494534.977290373] [camera.camera]: Set ROS param rgb_camera.profile to default: 1280x720x30
[realsense2_camera_node-1] [INFO] [1715494534.979336365] [camera.camera]: Set ROS param gyro_fps to default: 200
[realsense2_camera_node-1] [INFO] [1715494534.979568948] [camera.camera]: Set ROS param accel_fps to default: 100
[realsense2_camera_node-1] [INFO] [1715494535.037563328] [camera.camera]: Stopping Sensor: Depth Module
[realsense2_camera_node-1] [INFO] [1715494535.181536256] [camera.camera]: Starting Sensor: Depth Module
[realsense2_camera_node-1]  12/05 06:15:40,230 ERROR [281472543877344] (librealsense-exception.h:52) get_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: Connection timed out
[realsense2_camera_node-1] [INFO] [1715494540.230628177] [camera.camera]: Open profile: stream_type: Depth(0), Format: Z16, Width: 848, Height: 480, FPS: 30
[realsense2_camera_node-1] [INFO] [1715494540.231321482] [camera.camera]: Stopping Sensor: RGB Camera
[realsense2_camera_node-1] [INFO] [1715494540.244345135] [camera.camera]: Starting Sensor: RGB Camera
[realsense2_camera_node-1] [WARN] [1715494540.249828141] [camera.camera]: 
[realsense2_camera_node-1] [WARN] [1715494540.250018951] [camera.camera]: frame's time domain is HARDWARE_CLOCK. Timestamps may reset periodically.
[realsense2_camera_node-1]  12/05 06:15:45,357 ERROR [281472543877344] (librealsense-exception.h:52) xioctl(VIDIOC_S_FMT) failed, errno=5 Last Error: Input/output error
[realsense2_camera_node-1] [ERROR] [1715494545.358344559] [camera.camera]: ./src/rs_node_setup.cpp:370:An exception has been thrown: xioctl(VIDIOC_S_FMT) failed, errno=5 Last Error: Input/output error
[realsense2_camera_node-1] Failed to start device: xioctl(VIDIOC_S_FMT) failed, errno=5 Last Error: Input/output error
[realsense2_camera_node-1] [INFO] [1715494545.359954774] [camera.camera]: Stopping Sensor: RGB Camera
[realsense2_camera_node-1]  12/05 06:15:45,366 ERROR [281472510060768] (librealsense-exception.h:52) get_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: No such file or directory
[realsense2_camera_node-1]  12/05 06:15:45,366 ERROR [281472510060768] (error-handling.cpp:96) Error during polling error handler: get_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: No such file or directory
[realsense2_camera_node-1] [INFO] [1715494545.372495589] [camera.camera]: Starting Sensor: RGB Camera
[realsense2_camera_node-1]  12/05 06:15:50,470 ERROR [281472392751328] (librealsense-exception.h:52) xioctl(VIDIOC_S_FMT) failed, errno=5 Last Error: Input/output error
[realsense2_camera_node-1] [ERROR] [1715494550.470772143] [camera.camera]: ./src/rs_node_setup.cpp:370:An exception has been thrown: xioctl(VIDIOC_S_FMT) failed, errno=5 Last Error: Input/output error
[realsense2_camera_node-1] [ERROR] [1715494550.471136285] [camera.camera]: Error updating the sensors: xioctl(VIDIOC_S_FMT) failed, errno=5 Last Error: Input/output error
[realsense2_camera_node-1]  12/05 06:15:50,485 ERROR [281472535423200] (librealsense-exception.h:52) set_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: Connection timed out
[realsense2_camera_node-1]  12/05 06:15:50,486 ERROR [281472535423200] (global_timestamp_reader.cpp:239) Error during time_diff_keeper polling: set_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: Connection timed out
[realsense2_camera_node-1]  12/05 06:15:55,598 ERROR [281472510060768] (librealsense-exception.h:52) get_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: No such file or directory
[realsense2_camera_node-1]  12/05 06:15:55,598 ERROR [281472510060768] (error-handling.cpp:96) Error during polling error handler: get_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: No such file or directory

About my system, I am using a Raspberry pi 4 with 8GB ram, and I've increased the swap size to 2048. The OS is ubuntu server 22.04 and I am using ROS humble. I am using ssh to connect to the raspberry pi, and also running the ros2 launch commands while in ssh. My kernel is also the latest version, doing uname -r gives me 5.15.0-1053-raspi.

The ouput of dmesg is also filled with errors which you can see below:

usb 2-1: Failed to query (GET_CUR) UVC control 1 on unit 3: -32 (exp. 1024).
[ 8085.311722] uvcvideo 2-1:1.4: Failed to set UVC probe control : -110 (exp. 48).
[ 8085.311755] usb 2-1: Failed to query (GET_LEN) UVC control 7 on unit 3: -110 (exp. 2).
[ 8090.432326] usb 2-1: Failed to query (SET_CUR) UVC control 1 on unit 3: -110 (exp. 1024).
[ 8090.432332] uvcvideo 2-1:1.4: Failed to set UVC probe control : -110 (exp. 48).
[ 8095.552893] usb 2-1: Failed to query (GET_LEN) UVC control 7 on unit 3: -110 (exp. 2).
[ 8100.673535] usb 2-1: Failed to query (SET_CUR) UVC control 1 on unit 3: -110 (exp. 1024).
[ 8105.794102] usb 2-1: Failed to query (GET_LEN) UVC control 7 on unit 3: -110 (exp. 2).
[ 8110.914723] usb 2-1: Failed to query (SET_CUR) UVC control 1 on unit 3: -110 (exp. 1024).

I can see the depth topics and the infra topics on rviz2, but not the color/image_raw topic. I followed the official documentation to install the ros2 wrapper for intelrealsense but I am still getting this error. Searching the net for the same error also didn't give me any feasible solutions. Can anyone please help me figure out what the problem might be?

[MartyG-RealSense]

Hi @parapara29 xioctl(UVCIOC_CTRL_QUERY) errors typically indicate a conflict between the librealsense SDK and the kernel.

What method did you use to install librealsense, please? A method that historically works well on Raspberry Pi is to use the libuvc backend approach at the link below, as it bypasses the kernel and does not require the use of a kernel patch script.

https://github.com/IntelRealSense/librealsense/blob/master/doc/libuvc_installation.md

[parapara29]

Thanks for your reply!
Yes, I used libuvc installation method and am able to run rs-save-to-disk, and am able to see the RGB picture and the depth picture. After installing via the libuvc, I installed the ros wrapper using sudo apt install ros-humble-realsense2-* and sudo apt install ros-humble-librealsense2*

[MartyG-RealSense]

After using libuvc installation, sudo apt install ros-humble-librealsense2* should not have been used as librealsense had already been installed with libuvc. So installing it again risks having two separate installations of librealsense on the same computer that may conflict with each other.

If you still have the libuvc installation on your computer, what you could try is removing all RealSense related packages with the command below:

dpkg -l | grep "realsense" | cut -d " " -f 3 | xargs sudo dpkg --purge

and then re-install the ROS wrapper with sudo apt install ros-humble-realsense2-*

[parapara29]

I just tried that, and unfortunately i'm still getting the same error.

[MartyG-RealSense]

You should not need to specify the device_type in your rso2 launch instruction if you only have one camera attached.

Does the problem still occur if you use ros2 run instead of ros2 launch?

ros2 run realsense2_camera realsense2_camera_node

[parapara29]

It seems the problem was with the cable, turning around the type c fixed my issue after reinstalling like you said. However, the fps I'm getting is very very less. Is there anyway to mitigate that?

Thanks for the help so far!

[MartyG-RealSense]

You are very welcome. I'm pleased to hear that you have made progress by reversing the cable.

Does FPS improve if you disable the RGB option auto_exposure_priority

ros2 launch realsense2_camera rs_launch.py auto_exposure_priority:=false

[parapara29]

It's a little better but now I am getting the following error msg:

(librealsense-exception.h:52) set_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: Connection timed out

[MartyG-RealSense]

What happens if you reset the camera at launch with initial_reset and sync the timestamps of the different stream types with enable_sync

ros2 launch realsense2_camera rs_launch.py auto_exposure_priority:=false initial_reset:=true enable_sync:=true

[parapara29]

Ok, this is pretty weird but now when I run the command rs-save-to-disk, I am able to save the image and see it as well, but when I run
ros2 launch realsense2_camera rs_launch.py auto_exposure_priority:=false initial_reset:=true enable_sync:=true, I am getting

ERROR [281472917563616] (sensor.cpp:661) acquire_power failed: /dev/video10 is no video capture device Last Error: No such file or directory
[realsense2_camera_node-1]  12/05 09:43:26,578 WARNING [281472917563616] (rs.cpp:312) null pointer passed for argument "device"
[realsense2_camera_node-1] [WARN] [1715507006.578826891] [camera.camera]: Device 1/6 failed with exception: /dev/video10 is no video capture device Last Error: No such file or directory
[realsense2_camera_node-1]  12/05 09:43:26,588 ERROR [281472917563616] (librealsense-exception.h:52) /dev/video12 is no video capture device Last Error: No such file or directory
[realsense2_camera_node-1]  12/05 09:43:26,588 ERROR [281472917563616] (sensor.cpp:661) acquire_power failed: /dev/video12 is no video capture device Last Error: No such file or directory
[realsense2_camera_node-1]  12/05 09:43:26,680 WARNING [281472917563616] (rs.cpp:312) null pointer passed for argument "device"
[realsense2_camera_node-1] [WARN] [1715507006.680713225] [camera.camera]: Device 2/6 failed with exception: /dev/video12 is no video capture device Last Error: No such file or directory
[realsense2_camera_node-1]  12/05 09:43:26,693 ERROR [281472917563616] (librealsense-exception.h:52) /dev/video18 is no video capture device Last Error: No such file or directory
[realsense2_camera_node-1]  12/05 09:43:26,693 ERROR [281472917563616] (sensor.cpp:661) acquire_power failed: /dev/video18 is no video capture device Last Error: No such file or directory
[realsense2_camera_node-1]  12/05 09:43:26,782 WARNING [281472917563616] (rs.cpp:312) null pointer passed for argument "device"
[realsense2_camera_node-1] [WARN] [1715507006.782735319] [camera.camera]: Device 3/6 failed with exception: /dev/video18 is no video capture device Last Error: No such file or directory

Im not really sure why this is happening. I tested the realsense camera on my host laptop and everything was working fine, so there shouldn't be an issue with the camera or the cable.

[MartyG-RealSense]

Have you got any RealSense SDK programs such as save-to-disk running at the same time as the ROS wrapper? If you do then I recommend only having the wrapper active to ensure that access to the camera has not already been exckusively claimed by another program.

[parapara29]

Not really, I close them before accessing ros2 run or ros2 launch. Wierd thing is sometimes I can run my camera but sometimes I get the xioctl(UVCIOC_CTRL_QUERY) error. What can be the problem?

[parapara29]

I am also getting the following errors:

set_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: Connection timed out
 13/05 07:14:54,090 ERROR [281472712173792] (global_timestamp_reader.cpp:239) Error during time_diff_keeper polling: set_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: Connection timed out
 13/05 07:14:59,207 ERROR [281472351004896] (librealsense-exception.h:52) get_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: Connection timed out
 13/05 07:14:59,208 ERROR [281472351004896] (error-handling.cpp:96) Error during polling error handler: get_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: Connection timed out

After ive been able to successfully launch my realsense node.

[MartyG-RealSense]

Does performance improve if you reduce the resolution and FPS to 640x480 and 15 FPS?

ros2 launch realsense2_camera rs_launch.py auto_exposure_priority:=false depth_module.depth_profile:=640x480x15