Adding UMI-Gripper

[omarrayyann]

Hello, I want to add the UMI-Gripper (https://umi-gripper.github.io). I imported the URDF and got the joints working. I would like to ask if there is any way to add mirrors in MuJoCo? The other option would be to add cameras where the mirrors are (would be enough since the position of the UMI camera is fixed relative to the mirror) and overlay the camera outputs over the UMI view. Let me know if you have any other suggestions.

Screen.Recording.2024-09-25.at.10.56.47.PM.mov

[yuvaltassa]

1. This is awesome!
2. I don't understand the question about mirrors? You want a reflective surface? The answer is yes, you can have one such surface.

[omarrayyann]

I came across this: "Only the first reflective geom in the model is rendered as such". Is there no other way to have 2?

The UMI gripper has a mirror on each of the sides (trick to increase what it sees and give a virtual depth) as show here:

I got one to work but the other does not reflect (probably has to do with the thing i mentioned up).

Also, It might be a minor detail to add the mirrors anyways.

[yuvaltassa]

As I said, and as the docs say, you can have one such surface. I could try and increase it to two, not sure if anything would break.

In the meantime, place cameras in the correct locations such that the reflected image is the thing rendered by the camera and see that you get the correct match (use the one reflection you have now to debug this)

[kevinzakka]

Nice job @omarrayyann !

[omarrayyann]

Opened a PR for this here: #106