[Task Clean-up][Manager] Dexterous Part 9/11: Add Shadow state and OpenAI manager counterparts#6419
[Task Clean-up][Manager] Dexterous Part 9/11: Add Shadow state and OpenAI manager counterparts#6419hujc7 wants to merge 7 commits into
Conversation
Manager-based counterparts for the state-based and OpenAI (feed-forward and recurrent) Shadow Hand reorientation tasks, built on the shared reorientation MDP terms. Adds the RSL-RL LSTM runner configuration used by both the Direct and manager OpenAI-LSTM tasks, and observation-group mappings so asymmetric critics see the critic observation group. Validated by full manager training on PhysX and Newton (success streaks met; convergence-gap diagnosis vs Direct tracked as follow-up).
Greptile SummaryThis PR adds manager-based counterparts for the state-based and OpenAI (feed-forward and recurrent) Shadow Hand reorientation tasks, wiring them to the shared MDP terms from Part 8. It also introduces the
Confidence Score: 3/5The PR is largely well-structured, but the standard Shadow Hand manager env has a likely reward misconfiguration that would cause training behaviour to silently diverge from the Direct task it mirrors. The standard RewardsCfg passes a non-zero action_penalty_scale to DirectReorientReward without the action_name key that the OpenAI variant explicitly provides to the same function. If the reward function resolves the last-action tensor through action_name, the penalty is silently zeroed for the state-based manager env — a hidden behavioural gap from the Direct baseline that would not surface as an error during training. The agents and runner configs, the NoisyEMAJointPositionToLimitsAction implementation, and the gym registrations all look correct. shadow_hand_manager_env_cfg.py — specifically RewardsCfg (missing action_name) and the scene PresetCfg classes (missing replicate_physics and newton_kamino alias). Important Files Changed
Flowchart%%{init: {'theme': 'neutral'}}%%
flowchart TD
A[Isaac-Reorient-Cube-Shadow] --> B[ShadowHandManagerEnvCfg]
C[Isaac-Reorient-Cube-Shadow-OpenAI-FF] --> D[ShadowHandOpenAIManagerEnvCfg]
E[Isaac-Reorient-Cube-Shadow-OpenAI-LSTM] --> D
B --> F[ShadowHandManagerSceneCfg]
B --> G[ObservationsCfg 157-dim]
B --> H[ActionsCfg EMA]
B --> I[RewardsCfg - missing action_name]
D --> K[ShadowHandOpenAIManagerSceneCfg]
D --> L[OpenAIObservationsCfg 42+187-dim]
D --> M[OpenAIActionsCfg NoisyEMA]
D --> N[OpenAIRewardsCfg + action_name]
M --> P[NoisyEMAJointPositionToLimitsAction]
A --> Q[ShadowHandPPORunnerCfg]
C --> R[ShadowHandAsymFFPPORunnerCfg]
E --> S[ShadowHandAsymLSTMPPORunnerCfg]
S --> R
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
flowchart TD
A[Isaac-Reorient-Cube-Shadow] --> B[ShadowHandManagerEnvCfg]
C[Isaac-Reorient-Cube-Shadow-OpenAI-FF] --> D[ShadowHandOpenAIManagerEnvCfg]
E[Isaac-Reorient-Cube-Shadow-OpenAI-LSTM] --> D
B --> F[ShadowHandManagerSceneCfg]
B --> G[ObservationsCfg 157-dim]
B --> H[ActionsCfg EMA]
B --> I[RewardsCfg - missing action_name]
D --> K[ShadowHandOpenAIManagerSceneCfg]
D --> L[OpenAIObservationsCfg 42+187-dim]
D --> M[OpenAIActionsCfg NoisyEMA]
D --> N[OpenAIRewardsCfg + action_name]
M --> P[NoisyEMAJointPositionToLimitsAction]
A --> Q[ShadowHandPPORunnerCfg]
C --> R[ShadowHandAsymFFPPORunnerCfg]
E --> S[ShadowHandAsymLSTMPPORunnerCfg]
S --> R
Reviews (1): Last reviewed commit: "Add Shadow state and OpenAI manager coun..." | Re-trigger Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| "rotation_scale": _DIRECT_CFG.rot_reward_scale, | ||
| "rotation_epsilon": _DIRECT_CFG.rot_eps, | ||
| "action_penalty_scale": _DIRECT_CFG.action_penalty_scale, | ||
| "success_tolerance": _DIRECT_CFG.success_tolerance, | ||
| "success_bonus": _DIRECT_CFG.reach_goal_bonus, | ||
| "fall_distance": _DIRECT_CFG.fall_dist, | ||
| "fall_penalty": _DIRECT_CFG.fall_penalty, | ||
| "averaging_factor": _DIRECT_CFG.av_factor, | ||
| "success_count_threshold": _DIRECT_CFG.success_count_threshold, | ||
| "object_cfg": SceneEntityCfg("object"), | ||
| }, | ||
| ) | ||
|
|
||
|
|
||
| @configclass | ||
| class TerminationsCfg: | ||
| """Termination conditions matching the Direct task.""" | ||
|
|
||
| object_out_of_reach = DoneTerm( |
There was a problem hiding this comment.
action_name in RewardsCfg while action_penalty_scale is non-zero
RewardsCfg passes action_penalty_scale = _DIRECT_CFG.action_penalty_scale (which is -0.0002) to mdp.DirectReorientReward but omits the "action_name" key. The OpenAIRewardsCfg that calls the same function under the same penalty scale explicitly adds "action_name": "joint_pos". If DirectReorientReward resolves the last-action tensor via action_name, this omission silently drops the action penalty from the standard Shadow Hand manager env, making its reward landscape diverge from the Direct task it is designed to mirror.
| @configclass | ||
| class ShadowHandManagerSceneCfg(PresetCfg): | ||
| """Backend-specific scene cloning settings matching the Direct task.""" | ||
|
|
||
| physx = _ShadowHandManagerSceneCfg(num_envs=8192, env_spacing=0.75, clone_in_fabric=True) | ||
| newton_mjwarp = _ShadowHandManagerSceneCfg(num_envs=8192, env_spacing=0.75, clone_in_fabric=False) | ||
| ovphysx = _ShadowHandManagerSceneCfg(num_envs=8192, env_spacing=0.75, clone_in_fabric=True) |
There was a problem hiding this comment.
replicate_physics not forwarded to manager scene configs
The Direct task's ShadowHandSceneCfg always sets replicate_physics=True on its InteractiveSceneCfg presets, but _ShadowHandManagerSceneCfg and all of its wrappers omit it. With 8 192 parallel environments, missing replicate_physics=True means each environment gets an independent physics data copy instead of sharing it, which can noticeably degrade simulation throughput.
| class CriticCfg(FullStateWithoutActionCfg): | ||
| fingertip_wrench = ObsTerm( | ||
| func=mdp.fingertip_wrench, | ||
| scale=_OPENAI_DIRECT_CFG.force_torque_obs_scale, | ||
| params={ | ||
| "sensor_cfg": SceneEntityCfg("joint_wrench", body_names=_FINGERTIP_BODY_NAMES, preserve_order=False) | ||
| }, | ||
| ) | ||
| last_action = ObsTerm(func=mdp.reorient_last_action, params={"action_name": "joint_pos"}) | ||
|
|
There was a problem hiding this comment.
newton_kamino preset absent from OpenAIEventCfg (and both scene PresetCfgs)
ShadowHandEventCfg (Direct) defines newton_kamino = newton_mjwarp as an explicit alias so the Kamino solver backend resolves cleanly. OpenAIEventCfg, ShadowHandManagerSceneCfg, and ShadowHandOpenAIManagerSceneCfg all omit this alias, so selecting the Kamino backend at runtime would fall through to an unknown preset and likely raise a lookup error.
The noisy EMA action term and its cfg move into the task mdp package, replicate_physics is forwarded on the manager scenes, the newton_kamino alias resolves the Kamino backend, and the goal-marker position comes from the shared constants module.
The Shadow and OpenAI manager cfgs read the shared scalar values and noise-model cfgs from the constants module and the Direct cfg module scope, the parity-only direct_timeout custom is replaced by the framework time_out term, and the OpenAI policy observation folds into its term class with a Warp quat-error kernel and caller-owned buffer.
The Shadow and OpenAI manager cfgs read the shared scalar values and noise-model cfgs from the constants module and the Direct cfg module scope, the parity-only direct_timeout custom is replaced by the framework time_out term, and the OpenAI policy observation folds into its term class with a Warp quat-error kernel and caller-owned buffer.
The Shadow and OpenAI manager cfgs read the shared scalar values and noise-model cfgs from the constants module and the Direct cfg module scope, the parity-only direct_timeout custom is replaced by the framework time_out term, and the OpenAI policy observation folds into its term class with a Warp quat-error kernel and caller-owned buffer.
The Shadow manager cfg reads the simulation timestep from the shared task constants instead of a literal, keeping the manager and Direct variants single-sourced, and the shared observation terms pick up the Warp kernel conversion.
The Shadow and OpenAI manager configurations declare their scalar task parameters at the term declarations (matching the Direct values), and the observation terms pick up the family kernel module imports.
…nager runtime (#6412) ## Summary - Fixes OVPhysX actuator joint indices to follow the common actuator indexing contract. - Fixes OVPhysX initialization alongside Kit by reusing Kit's registered PhysX schema provider. - Fixes the OVPhysX manager to support both the declared public runtime API and the current runtime API. - Regression tests included. Validated by full dexterous training runs on the OVPhysX backend; split out of the lumped validation branch #6324 (Part 2 of 11). ## Dependencies - None. ## Series review map Full integrated diff + training/validation evidence: the lumped validation PR #6324 (DO-NOT-MERGE). | Part | PR | |---|---| | Docs: regenerate the environment overview table | #6410 | | Part 1/11: Newton runtime fixes (cloner rows, cubric fallback, viz teardown) | #6411 | | **Part 2/11: OVPhysX runtime fixes (this PR)** | #6412 | | Part 3/11: success-rate metrics for the Direct reorientation tasks | #6413 | | Part 4/11: RSL-RL training for the handover Direct task | #6414 | | Part 5/11: success-rate support in the benchmark utilities | #6415 | | Part 6/11: renderer presets for the Direct camera task | #6416 | | Part 7/11: OVPhysX presets for the dexterous tasks | #6417 | | Part 8/11: Allegro manager counterpart | #6418 | | Part 9/11: Shadow + OpenAI manager counterparts | #6419 | | Part 10/11: Shadow camera manager counterpart | #6420 | | Part 11/11: Shadow handover manager counterpart | #6421 | --- ### Exact changes in this PR - OVPhysX backend changes + tests: 1f7a433
1 participant
Summary
Isaac-Reorient-Cube-Shadow,-OpenAI-FF, and-OpenAI-LSTMwith benchmark rows.Dependencies
Review updates (2026-07-09)
direct_timeoutcustom with the frameworktime_outterm; the OpenAI policy observation folds into its term class with a Warp quat-error kernel and caller-owned buffer.Review updates (2026-07-11)
SHADOW_SIM_DTin the shared task constants, and the shared observation terms pick up the Warp kernel conversion.Series review map
Full integrated diff + training/validation evidence: the lumped validation PR #6324 (DO-NOT-MERGE).
Builds on: #6413. This PR's own commits relative to each parent branch: