Franka Sim to SIm#6457
Conversation
github-actions
Bot
added
asset
Greptile SummaryThis PR adds Franka Panda sim-to-sim support for the dexsuite lift/reorient tasks, including robot-specific configs, a new conditional-reset system with Warp-based collision validation, and physics parameter updates for both Franka and Kuka-Allegro.
Confidence Score: 1/5Not safe to merge: the Franka environments fail to import due to a missing module, and the base Franka robot config change may break all existing Franka tasks. The new Franka dexsuite environments import
Important Files Changed
Sequence Diagram%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
participant RL as RL Trainer
participant EM as EventManager
participant CR as conditional_reset
participant BUF as State Buffer
participant MC as mesh_clearance
participant SC as slab_clearance
Note over CR,BUF: First reset - prefill phase
RL->>EM: reset(env_ids)
EM->>CR: call(env_ids, terms, valid_criteria)
loop until buffer full per group
CR->>CR: roll_once - apply nested reset terms
CR->>MC: call(env, env_ids) returns bool mask
CR->>SC: call(env, env_ids) returns bool mask
CR->>BUF: get_reset_state for valid_ids
end
CR->>CR: terms.clear and valid_criteria.clear
Note over CR,BUF: Subsequent resets - replay phase
RL->>EM: reset(env_ids)
EM->>CR: call(env_ids)
CR->>BUF: sample rows by asset-combination group
CR->>CR: set_reset_state with sampled states
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
participant RL as RL Trainer
participant EM as EventManager
participant CR as conditional_reset
participant BUF as State Buffer
participant MC as mesh_clearance
participant SC as slab_clearance
Note over CR,BUF: First reset - prefill phase
RL->>EM: reset(env_ids)
EM->>CR: call(env_ids, terms, valid_criteria)
loop until buffer full per group
CR->>CR: roll_once - apply nested reset terms
CR->>MC: call(env, env_ids) returns bool mask
CR->>SC: call(env, env_ids) returns bool mask
CR->>BUF: get_reset_state for valid_ids
end
CR->>CR: terms.clear and valid_criteria.clear
Note over CR,BUF: Subsequent resets - replay phase
RL->>EM: reset(env_ids)
EM->>CR: call(env_ids)
CR->>BUF: sample rows by asset-combination group
CR->>CR: set_reset_state with sampled states
|
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| # keep descending: articulation links may be authored as nested rigid bodies | ||
| # (e.g. MuJoCo-converted assets), so a rigid body can have rigid-body children | ||
| all_prims += child_prim.GetChildren() |
There was a problem hiding this comment.
The previous code stopped descending into the children of any prim that already carries RigidBodyAPI — a correct optimisation for standard PhysX assets where nested rigid bodies are disallowed. The new code always appends children regardless, so for a standard USD articulation (e.g. a Franka loaded from panda_instanceable) every rigid-body link now adds its own children back to the queue even though those children will never have a second RigidBodyAPI. For a 7-link arm with ~10 geometry children per link this multiplies prims visited by roughly 10×. The stated justification ("articulation links may be authored as nested rigid bodies in MuJoCo-converted assets") applies only to those assets but is now applied unconditionally to every asset type.
AntoineRichard
left a comment
AntoineRichard
left a comment
There was a problem hiding this comment.
That's not the So101 keyboard? @ooctipus
4b82c0f to
1831952
Compare
|
@greptile-apps review pls |
|
|
||
| from ... import dexsuite_env_cfg as dexsuite | ||
| from ... import mdp | ||
| from .camera_cfg import StateObservationCfg |
There was a problem hiding this comment.
camera_cfg module causes import failure
camera_cfg.py does not exist in the franka config directory (config/franka/), so this import will raise ModuleNotFoundError at module load time, making every gym environment registered in config/franka/__init__.py (Isaac-Reorient-Franka, Isaac-Lift-Franka, etc.) fail to instantiate. The analogous kuka_allegro/camera_cfg.py exists and defines StateObservationCfg; the franka equivalent is absent. The PR description notes a dependency on #6481, which may add this file — but as submitted the module is unrunnable.
724f646 to
a46fe3b
Compare
Sync PR isaac-sim#6481 review feedback: rename motor to joint velocity terminology, warn when an implicit actuator sets only velocity_limit, and fix partial velocity_limit dicts zeroing unmatched joints.
Partial-dict zero-fill is the uniform convention of _parse_joint_parameter for all actuator parameters; the carve-out made dict semantics inconsistent across actuator types.
The panda_shoulder and panda_forearm actuators were merged into panda_arm in FRANKA_PANDA_CFG. Update the three test helpers that still referenced the old keys, and add the required isaaclab_assets changelog fragment.
The Nucleus server replaced panda_instanceable.usd with franka_panda.usda; the cabinet direct task carried the last stale reference. Joint naming is unchanged between the assets.
The original panda_instanceable.usd moved to Legacy/ on the Nucleus server; the cabinet task keeps the same robot model at its new home.
The Nucleus asset reorganization moved panda_instanceable.usd under Robots/FrankaEmika/Legacy/. Point FRANKA_PANDA_CFG at that new location and drop the panda_arm actuator rework from it, so upstream franka tasks, mimic demos, and determinism baselines keep the exact robot they were built on. The menagerie-converted franka_panda.usda and its calibrated actuators move into the dexsuite franka config as FRANKA_PANDA_DEXSUITE_CFG, which is the only consumer. This also reverts the panda_arm rename adaptations in the OSC tests and reach task, and removes the now change-free newton/physx changelog fragments. Also add the missing camera_cfg module the dexsuite franka config imports; it previously only existed as a local build artifact.
Reverting the operational-space test adaptation left the package with no changes, so the placeholder fragment no longer belongs.
3 participants
Description
Depends on #6481
Screenshots
Please attach before and after screenshots of the change if applicable.
Checklist
pre-commitchecks with./isaaclab.sh --formatconfig/extension.tomlfileCONTRIBUTORS.mdor my name already exists there