Adds DisplayPort cable insertion envs#6398
Conversation
Greptile SummaryThis PR adds DisplayPort cable insertion environments and tooling. The main changes are:
Confidence Score: 4/5The task-space training path needs a config fix before merging.
source/isaaclab_tasks/isaaclab_tasks/contrib/deploy/cable_insertion/config/displayport_rizon_4s/task_space_env_cfg.py Important Files Changed
Flowchart%%{init: {'theme': 'neutral'}}%%
flowchart TD
A[DisplayPort Gym task id] --> B[DisplayportInsertionEnv]
B --> C[Environment config]
C --> D[Reset events]
D --> E[reset_scene_to_default]
E --> F[randomize_socket_pose]
F --> G[reset_plug_at_goal_curriculum]
G --> H[set_robot_to_object_grasp_pose]
C --> I[Policy observations and actions]
C --> J[Rewards and terminations]
B --> K[Success metric logging]
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
flowchart TD
A[DisplayPort Gym task id] --> B[DisplayportInsertionEnv]
B --> C[Environment config]
C --> D[Reset events]
D --> E[reset_scene_to_default]
E --> F[randomize_socket_pose]
F --> G[reset_plug_at_goal_curriculum]
G --> H[set_robot_to_object_grasp_pose]
C --> I[Policy observations and actions]
C --> J[Rewards and terminations]
B --> K[Success metric logging]
Reviews (1): Last reviewed commit: "Relocate DisplayPort cable_insertion und..." | Re-trigger Greptile |
| "at_goal_prob": 0.0, | ||
| "at_goal_prob_final": 0.0, |
There was a problem hiding this comment.
The task-space config documents and wires an at-goal curriculum, but both probabilities are 0.0. reset_plug_at_goal_curriculum only enters the at-goal branch when the current probability is positive, so every reset uses the normal approach-pose randomization and the task-space policy never receives the seeded insertion starts this config describes.
| "at_goal_prob": 0.0, | |
| "at_goal_prob_final": 0.0, | |
| "at_goal_prob": 0.8, | |
| "at_goal_prob_final": 0.0, |
The Rizon 4s cable-insertion task was failing every episode at reset: the IK + open-loop close sequence put the gripper next to the plug instead of around it, the fingers closed on empty space, and the plug fell to the floor (or twisted away when a finger clipped its edge). Root cause was a geometry mismatch between the un-flattened plug USD and the IK target. The un-flattened plug has its visible/collision ``Mesh`` offset ~3 cm from the rigid-body origin (``bbox_center_local = (0.0312, 0.0009, -0.0107)`` in body frame). ``set_robot_to_object_grasp_pose`` snapped the plug body to the gripper-achieved pose, but the achieved pose was a function of the *body* origin, not the visible centre, so the fingertips landed ~3 cm from the actual plug. Switch to the flattened plug + socket USDs (``flattened/`` directory) where ``bbox_center == (0, 0, 0)`` and the visible mesh coincides with the rigid-body origin. With that, the grasp parameters reduce to the gripper tool length only: * ``grasp_offset = (0, 0, -0.13)`` -- only backs ``link7`` off by the Grav tool length so the fingertips land at the plug body origin (which is now also the visible centre). * ``grasp_rot_offset = (-0.707, 0.707, 0, 0)`` (unchanged) keeps ``link7`` pointing straight down with the gripper finger axis perpendicular to the plug's long body Y axis. * Plug + socket use identity ``init_state.rot``; the flattened USDs already align the long axis with body Y. * Plug spawns 5 cm above the socket so the policy has descent room. * ``hand_close_width = 0.0`` matches the gear-assembly small-gear close target. ``-0.155`` (the joint hard limit) was tried and ejected the plug because the drive over-commanded the fingers. Also extend ``set_robot_to_object_grasp_pose``: * Snap the held object to the achieved hand pose after IK so any IK residual error or geometry offset is washed out before the gripper closes. * One-shot ``[GRASP-DBG]`` log dumps the initial pose, the IK target, the achieved hand pose and the snapped object pose -- invaluable for diagnosing future grasp regressions.
The Flexiv Rizon 4s in the NVIDIA Hubble Lab is wall-mounted with the arm rotated about the negative X-axis. Update the ROS inference env config so the simulator's reset state matches the physical robot: * Set the home joint pose to the lab's joint convention. * Set the robot base orientation to the wall-mount quaternion so the arm hangs the same way in sim and in the real cell. This keeps the policy roll-out and the live deployment in the same robot frame, which is required for the policy obs (computed relative to ``base_link``) to be valid at inference time.
Three small scripts under ``scripts/tools/`` to diagnose and repair USD assets whose visible/collision geometry is offset from the rigid-body origin -- the failure mode that broke the Rizon 4s cable-insertion grasp. * ``inspect_cable_usd.py``: standalone Pixar-USD walker. Lists xform ops, ``RigidBodyAPI`` / ``MassAPI`` / ``CollisionAPI`` markers and per-mesh ``bbox_center_local`` / ``bbox_size``. Use this from any shell that already has ``pxr`` on PYTHONPATH (e.g. a Kit Python). * ``inspect_cable_usd_kit.py``: same walker run inside an IsaacLab ``AppLauncher`` context so ``pxr`` is available without extra setup. This is what we actually used during debugging from the IsaacLab conda env. * ``flatten_cable_usd.py``: bake the chain of intermediate xforms down into the leaf mesh's points, then reset all xform ops on the rigid-body and its children to identity. The result is a USD whose visible mesh is centered at the rigid-body origin -- exactly the invariant the IK-based grasp event assumes. Supports an optional ``--rotate-z-deg`` knob for assets whose body-local pose needs an additional rotation baked in. Both inspectors print the bounding-box centre in body-local frame so ``grasp_offset`` can be set deterministically from the asset rather than tuned by trial and error.
The ``-Play-v0`` variants for Rizon 4s and UR10e gear assembly omitted ``rsl_rl_cfg_entry_point`` in their gym registration, so ``play.py`` could not resolve the agent config and would error out with ``KeyError`` before ever loading a checkpoint. Add the entry-point so ``./isaaclab.sh -p .../play.py --task Isaac-Deploy-GearAssembly-...-Play-v0`` works end-to-end. Also pick up a pre-commit whitespace cleanup in ``play_hardcoded.py``.
Required for training: env cfg references these files but they were excluded by the *.usd gitignore rule.
…eam rebase. Upstream moved deploy tasks from manager_based/manipulation/deploy to contrib/deploy; update module paths and scripts so DP env registration and imports match the new layout while keeping existing Isaac-Deploy task IDs.
Remove debug scripts, asset tooling, passive drop-test env, legacy GB300 configs, unused assets, and disabled sysid action plumbing. Keep only the displayport_rizon_4s joint/task/ROS env variants and the two SDF USD assets used by the training envs.
7ccabce to
2765083
Compare
Description
Added envs to train and run inference for the new task of DisplayPort cable insertion task with the Flexiv Rizon 4s robot. Supports multiple env variants based on policy design choices such as operating in joint space vs task space and the observation space used.
Added documentation for a tutorial on training a DisplayPort cable insertion policy with Flexiv Rizon 4s.
Type of change
Checklist
pre-commitchecks with./isaaclab.sh --formatconfig/extension.tomlfileCONTRIBUTORS.mdor my name already exists there