About RealSense wrist camera mount design

[mcres]

The documentation of the Wrist Camera Mount for Intel® RealSense™ D435/D435i states:

The camera is centered for depth sensing, but can also function with RGB only.

This makes the RGB image misaligned, i.e. this image shows both gripper fingers on the depth image, but only the left one on the RGB image.

It is my understanding that the wrist images provided to the model should ideally see both fingers moving, e.g. see this dataset.

Also, AFAIK, models supported by lerobot do not make use of depth data. See huggingface/lerobot#1144 (comment).

My question is the following: is there any reason I am not aware of for the part to be designed like so?

[yquanli]

Your observations are very realistic, and the d435 camera has a natural bias between RGB observation and depth observation due to the design of the hardware system.
Perhaps you can try the Intel RealSense D405, which uses the same set of hardware modules for depth and RGB observation, and the practice of using it as a wrist camera has gradually become the mainstream.

[mcres]

@yquanli thanks for your answer.

I have modified the original mount such that the RGB optic is centered. However, I see your comment on the D405 model and the fact that it is kind of mainstream. Perhaps I'll use that in the future.

[yquanli]

@mcres a good compromise if you don't want to integrate depth measurements for now.🚀