speeding up env.step() for realtime teleoperation

[leonmkim]

Hello again! I am attempting to collect simulated demos on a custom tabletop franka environment using a spacemouse to command delta poses to the end effector. Effectively, I have a script that maps spacemouse inputs to the action space of the pd_ee_target_delta_pose controller and loops through env.steps while rendering the environment to display.

However, I've found for the somewhat complex scene I have that sim struggles to keep up with realtime. Having gone through some quick tuning following your guide, I've been able to achieve slight speedups (reducing contact_offset, simplifying collision geoms). However, I've found rendering to display to have a far greater impact on how quickly I can step through the env. For example, rendering the GUI using env.render_human() slows sim down to ~0.6x real-time factor, even for simpler environments like LiftPegUpright-v1 and using minimal rendering. It seems like the only way I can get close to real time is by rendering a low-res (256x256) human render camera in a cv2 window.

While this setup is workable, it would be much more convenient if I could look at the GUI while I am teleoperating demos. I am wondering if you have any general tips/advice to robustly achieve 1x real-time factor.

Thank you as always for your fastidious support and apologies for the recent influx of questions from me.

[leonmkim]

Woop! I was computing realtime factor incorrectly (forgot to factor in control freq in env.step) and I realize I was actually getting around 30x RTF for my environment. My mistake!