Skip to content

Commit 0dd1950

Browse files
authored
TeleopPanel: SE(3) gizmo + ghost hand for interactive teleop (#13)
* Add TeleopPanel: SE(3) gizmo + ghost hand for interactive teleop - GhostHand: extracts gripper collision meshes from MuJoCo model, merges into single trimesh, renders as transparent overlay in the EE frame. Generic — works for any gripper body prefix. - TeleopPanel(PanelBase): Viser TransformControls gizmo + GUI (activate/deactivate, snap to EE, toggle gripper, record, safety mode dropdown). Drives TeleopController.set_target_pose() from gizmo on_update callback. - on_sync steps the controller and updates ghost pose each frame. * Fix: remove unsupported opacity arg from add_mesh_trimesh * Fix ghost hand: make child of gizmo, increase gizmo scale - Ghost mesh is now a child of the gizmo scene node — moves automatically when the gizmo is dragged, no manual pose updates - Gizmo scale 0.15 → 0.3 for easier interaction * Add debug prints for teleop callback tracing * Step controller directly in gizmo callback, sync viewer * Use background thread for teleop step loop The gizmo callback runs in Viser's thread and can't safely touch MuJoCo data. Now the callback only buffers the target pose, and a background thread at ~30Hz calls controller.step() + viewer.sync(). The thread starts on activate and stops on deactivate. * Remove debug prints from teleop panel * Fix recursion: on_sync no-op, teleop loop handles stepping * Use controller.toggle_gripper() instead of kinematic close/open * Check controller.is_active in teleop loop * Reset panel UI when teleop loop exits (abort/error) * Add abort_fn to TeleopPanel, check in loop instead of controller * Clear abort on teleop activate via clear_abort_fn * Add status label showing tracking/collision/unreachable state * Reset status to Idle on deactivate and loop exit * Update safety dropdown to allow/reject * Colored status: red for collision, orange for unreachable, markdown rendering * Use emoji circles for status colors (no inline HTML) * Unique gizmo names per arm for bimanual teleop * Add shared sim lock for bimanual teleop thread safety * Wrap each arm's controls in a labeled folder * Cleanup: remove dead set_color(), remove on_sync no-op override
1 parent e27b8c0 commit 0dd1950

2 files changed

Lines changed: 399 additions & 1 deletion

File tree

src/mj_viser/__init__.py

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,14 @@
22

33
from mj_viser.panels import PanelBase
44
from mj_viser.sensor_panel import SensorChannel, SensorPanel
5+
from mj_viser.teleop_panel import GhostHand, TeleopPanel
56
from mj_viser.viewer import MujocoViewer
67

7-
__all__ = ["MujocoViewer", "PanelBase", "SensorChannel", "SensorPanel"]
8+
__all__ = [
9+
"GhostHand",
10+
"MujocoViewer",
11+
"PanelBase",
12+
"SensorChannel",
13+
"SensorPanel",
14+
"TeleopPanel",
15+
]

0 commit comments

Comments
 (0)