ManiSkill v3.0.0b22 Released. Leaving "beta" soon!

[StoneT2000]

Overview

The latest b22 version brings a lot of bug fixes and stability improvements to ManiSkill that were being stress tested on the nightly package (thank you everyone!). Incredible to also see that we have nearly 20 new contributors since the last beta release, thank you everyone for helping maintain and improve ManiSkill and we hope to grow the manipulation research community with this.

That being said ManiSkill has been forever stuck on a "beta" version labeling but we do want to move to a full stable release. Currently the plan before we move finally to an actual "v3.1.0" version release is pending working on the following features:

• Add VR related teleop tooling / link docs to external repos for that
• Another attempt at supporting gymnasium > 1 and < 1

We had a plan to migrate ManiSkill2 soft body envs to be integrated with ManiSkill 3 APIs but that is a larger effort that is partly on hold now due to prioritization of other issues. We still just recommend you to use ManiSkill2 soft body environments which still work great (and any maniskill3 version would largely behave the same).

Highlights

• There is now some initial lerobot sim2real deployment support through a "sim in the loop" approach, which can help find sim2real bugs earlier and help reduce controller dynamics sim2real gaps (for mostly static manipulation tasks). The technique and approach is documented here and can be applied to other robots (we may be adding some examples for the Franka robot in the near future) https://maniskill.readthedocs.io/en/latest/user_guide/tutorials/sim2real/index.html

• GPU and CPU simulation IK solvers for end-effector control of robot arms have been fixed and are much more closely aligned, namely taking the same action in a CPU sim vs GPU sim will largely lead to the same result when using the IK based controllers.

• There is now a new useful wrapper ActionRepeatWrapper: https://maniskill.readthedocs.io/en/latest/user_guide/wrappers/action_repeat.html

• You can now have more fine grained control over joint position target/joint velocity target setting

• There is now a CachedResetWrapper which can help minimize overhead spent resetting environments by setting a state from a large reset pool of states: https://maniskill.readthedocs.io/en/latest/user_guide/wrappers/cached_reset.html

• Improved multi-gpu support, there is also an example multi-gpu RL setup via Puffer Lib here: https://github.com/PufferAI/PufferLib/tree/3.0/pufferlib/environments/mani_skill

• Much more simplified teleop+motion planning code with some better more unified interfaces

• Numpy 2.0 support

• A few new robots like XLeRobot and the WidowXAI platform

• Several new tasks, including more with dexterous hands (which will come with teleop system support in the future)

Additions

• Several new useful wrappers, ActionRepeat and CachedResets
• Sim2real deployment tooling for LeRobot
• Improved API documentation and auto-generated robot/task documentation
• ManiSkill to LeRobot dataset format conversion tool
• More fine-grained control over joint position target/joint velocity target setting
• Initial refactor of a number of typings to use native python dict, list, etc. instead of using the ones from the typings package.

Various bug fixes

Many of them, see full release notes: https://github.com/haosulab/ManiSkill/releases/tag/v3.0.0b22

The most critical ones were

• GPU and CPU IK solver alignment (see comment about it in the highlights section above)
• Various fixes to make multi-gpu RL training more easy
• Fixed SAC baseline to be deterministic
• Fixes some critical bugs around modelling mimic joints for robots like the inspire hand, leading to more stable sim + less NaNs.

Breaking Changes

None!

Full release notes here: https://github.com/haosulab/ManiSkill/releases/tag/v3.0.0b22