Under-Actuation Mechanism for Shadow Hand

[JYChen18]

Intro

Hi!

I am a graduate student at Peking University, I use MuJoCo for my research on dexterous grasping and manipulation.

My setup

MuJoCo 3.2.6, python, Linux Ubuntu 20.04

My question

The under-actuation mechanism in the MuJoCo simulation differs significantly from that of the real product, contributing to a large sim2real gap, especially in precise grasping and manipulation tasks.

In the simulation, the motion of FFJ1 and FFJ2 is always identical, with each joint receiving roughly half of the combined control signal FFJ0, mediated by the tendon.

<tendon>
    <fixed name="rh_FFJ0">
      <joint joint="rh_FFJ2" coef="1"/>
      <joint joint="rh_FFJ1" coef="1"/>
    </fixed>
</tendon>

However, on the real Shadow Hand, the behavior is different: when the control signal FFJ0 is less than 90 degrees, FFJ1 remains nearly stable (almost equal to 0) while FFJ2 increases from 0 to 90. Once FFJ0 exceeds 90 degrees, FFJ1 begins to increase from 0 to 90 and FFJ2 stops moving at 90 degrees.

Given that the real product’s behavior cannot be altered, is there a mechanism in MuJoCo that can better approximate this phenomenon to reduce the sim2real gap?

Minimal model and/or code that explain my question

No response

Confirmations

• I searched the latest documentation thoroughly before posting.
• I searched previous Issues and Discussions, I am certain this has not been raised before.

[kevinzakka]

Hi @JYChen18, we actually have a model that reproduces the real world behavior, it's actually from the OpenAI paper from 2019. I can try to find some time to create a version in Menagerie in the next couple weeks.

[kevinzakka]

@JYChen18 Is this behavior what you're looking for?

hand_h.mp4

[JYChen18]

That’s exactly what I need! Thanks!
However, since I am working under a tight deadline, several weeks might be a bit late. If you could provide some guidances or a rough XML to get started, I’d be happy to implement it myself.

[JYChen18]

@kevinzakka Hey! Just checking in—any updates on this question? If you're busy, a dirty XML or a quick pointer would help me get started. Thanks!

[kevinzakka]

@JYChen18 https://github.com/kevinzakka/menagerie_custom

If you can upstream that to Menagerie that would be great, I'm very swamped right now.