Time synchronization of binocular images and release of image data and internal parameters

[sun-under-bird]

I would like to ask if you have used non-integrated cameras before? Cameras like D435i are all integrated and have been calibrated, so they can publish image topics and intrinsic parameters, and it's very easy to run rtabmap. However, my current camera is a global shutter type and has hardware synchronization, but it uses YUYV or MJPG format images for stitching. I need to split them and publish my own calibrated camera intrinsic and extrinsic parameters. This process will result in a delay and cannot publish at a stable frame rate like D435i does. Moreover, the mapping effect is not ideal, and it loses tracking when I move slightly faster. When I use D435i, there is no such problem and the mapping effect is very good. Is there any good solution?

[matlabbe]

In theory, if:

• cameras are hardware synchronized
• correctly calibrated (intrinsics and stereo extrinsics) and stereo rectified (see stereo_image_proc)

You can follow same rtabmap examples than D435i (in particular the stereo ones) but just change the topics to yours. If you can, you can provide a small rosbag of your current data while moving the camera, and we could see if there are some issues with the calibration or the time synchronization.