Add input_ref_frame = base option for whole body ik and devices#754
Add input_ref_frame = base option for whole body ik and devices#754kevin-thankyou-lin merged 11 commits intomasterfrom
Conversation
…dd-input-ref-frame-whole-body-ik
robosuite/devices/device.py
Outdated
| else: | ||
| arm_norm_delta = np.concatenate([dpos, drotation]) | ||
|
|
||
| if robot.torso is not None: |
There was a problem hiding this comment.
I feel like it doesnt make the most sense to keep to have this code in the device class? This code just keeps it in the same place right - doesnt look like it has anything to do with the device input?
There was a problem hiding this comment.
the function is input2action() where the output is "a valid action sequence that can be fed into an env.step() call" - where else should this code go?
There was a problem hiding this comment.
yes, it keeps it in the same place; for delta action it's not needed because 0 implicitly keeps things in the same place, but for absolute position control the value would typically be non-zero
There was a problem hiding this comment.
Feel like we have added similar functionality in the collect_human_demonstrations.py script: https://github.com/ARISE-Initiative/robosuite/blob/master/robosuite/scripts/collect_human_demonstrations.py#L46 and. While its not the best place to have this type of code (which keeps joints in the same state) at least its in one place.
This is how I feel, but if you disagree, I will defer to your opinion on this.
There was a problem hiding this comment.
At some point we should update the control function in robot.py to be more flexible, so that we dont always need to specify a no-op explicity when its not delta control, but this can be a separate PR
There was a problem hiding this comment.
I see; prev_gripper code assumes the hand uses absolute position control (which is true in all? the configs we have).
I'll move things to collect_human_demonstrations, the cons are mainly:
i) it'd need to be duplicated across multiple repos that have their own collect_human_demonstrations (in this case, it'll just be robocasa and robosuite).
ii) the collect demos script will look more bloated.
There was a problem hiding this comment.
That said, I just realized we don't have a torso action in device.py. I'll think about where to add that and then we can decide where to put things
update the control function in robot.py to be more flexible, so that we dont always need to specify a no-op explicity
agreed
There was a problem hiding this comment.
Seem like the arm has 'prev_target' inside the device i.e. there's an inconsistency between gripper and arm. For the torso height, I've followed arm conventions w/ a new get_torso_action() method
…E-Initiative#754) * Add input_ref_frame = base option for whole body ik and devices * Correct goal_update_name: desired -> target * Add more finegrained input_ref_frame for ik controller * Update function name to controller_base_pose * Check robot has torso * Allow torso action as part of base actions * Enable torso action for joint position delta controller * Add assert for WHOLE_BODY_IK has 1 robot only * Format * Update default goal update mode
2 participants
What this does
This pull request introduces support for specifying the input reference frame (
input_ref_frame) for whole-body inverse kinematics (IK) controllers, allowing actions and goals to be interpreted in either the world or base frame. It also adds a newgoal_update_modeparameter to device interfaces, enabling more flexible goal updates based on either desired or achieved states. These changes are reflected in controller configuration, device logic, and demonstration collection scripts.Whole-Body IK Reference Frame Support
input_ref_frameas a configurable parameter inrobosuite/controllers/composite/composite_controller.py,robosuite/utils/ik_utils.py, and controller config JSONs, enabling IK actions to be interpreted in either the "world" or "base" frame. [1] [2] [3] [4]robosuite/utils/ik_utils.pyto transform target poses between frames as needed, ensuring correct pose computation when using the base frame.Device and Action Interface Enhancements
robosuite/devices/device.py,robosuite/devices/mjgui.py) to support a newgoal_update_modeargument, allowing goal updates based on either desired or achieved states, and integrated logic for handling torso actions and reference frames. [1] [2] [3] [4] [5] [6]Demonstration Collection Improvements
goal_update_modeas a command-line argument inrobosuite/scripts/collect_human_demonstrations.py, propagating it through the demonstration collection pipeline for more flexible data gathering. [1] [2] [3] [4]These changes collectively provide more control over how actions and goals are interpreted and updated within the whole-body IK framework and demonstration collection workflows.
How it was tested
Examples:
Then, press
bto switch between arm and base modes. Note: the controller can do both base and arm control simultaneously, currently that's not enabled in the teleop script.