Policy for Unitree Go2

[SMEAC]

Does anyone have a well trained policy for the Unitree Go2?

I am hoping to use the Unitree Go2, in a similar extension to that of the Quadruped extension in the Robotics examples, for the Spot Robot from Boston Dynamics. This robot comes pre-trained and you can start working with it straight away. But for the Unitree Go2 I have to run through the reinforcement training tutorials, to get a policy, and regardless of how long I train or how I fine tune the parameters, I have never been completely happy with the results.
Noting that the kinematic training isn’t even my area of work, I just seem to be wasting hours trying to get a start point for a well known and often used robot.

Many thanks in advance

[baryn]

Hi,
In this package a model trained with PPO on rough terrain for Go2 is provided. model_7850.pt.
I have not tested it yet. The package uses IsaacLab but I guess it might be possible to create a UnitreeGo2RoughTerrainPolicy inspired from IsaacSim SpotFlatTerrainPolicy that loads a model like so by replacing the current model with model_7850.pt:

self.load_policy(
    assets_root_path + "/Isaac/Samples/Policies/Spot_Policies/spot_policy.pt",
    assets_root_path + "/Isaac/Samples/Policies/Spot_Policies/spot_env.yaml",
)

Let me know if you succeed in applying this approach.

Cheers
Bryan

[baryn]

Hi,

I made the policy work in IsaacSim using the Spot example as mentioned before.

• First I ran the following training policy in IsaacLab:

./isaaclab.sh -p scripts/reinforcement_learning/rsl_rl/train.py --task Isaac-Velocity-Flat-Unitree-Go2-v0 --headless --max_iterations 2000

• To check that it runs well:

./isaaclab.sh -p scripts/reinforcement_learning/rsl_rl/play.py  --task Isaac-Velocity-Flat-Unitree-Go2-v0 --num_envs 1 

• Now, copy the policy.pt that appears in the logs/ folder in IsaacLab along with the env.yaml and paste them in your project. Now, load them like so:

self.load_policy(
            "[YOUR_PATH_TO]/policy.pt",
            "[YOUR_PATH_TO]/env.yaml"
        )

• In general you have to apply scaling factor to the actions and observations. In this case the action scale is 0.25 (given in env.yaml). One has:

self._action_scale = 0.25

• If you open env.yaml, you will see that the physics engine is expected to run at 200Hz (1/0.005). So, do so in your scene:

self.world = World(stage_units_in_meters=1.0, physics_dt= 1/200, rendering_dt = 1/50)

• I initialize the robot height at 0.4 as it was done during training (see scene:robot:spawn:init_state:pos) and scale down the keyboard inputs to 1.0 otherwise the robot may crash frequently. For higher velocity commands you will have to adapt the training policy.

self._input_keyboard_mapping = {
            # forward command
            "NUMPAD_8": [1.0, 0.0, 0.0],
            "UP": [1.0, 0.0, 0.0],
            # back command
            "NUMPAD_2": [-1.0, 0.0, 0.0],
            "DOWN": [-1.0, 0.0, 0.0],
            # left command
            "NUMPAD_6": [0.0, -1.0, 0.0],
            "RIGHT": [0.0, -1.0, 0.0],
            # right command
            "NUMPAD_4": [0.0, 1.0, 0.0],
            "LEFT": [0.0, 1.0, 0.0],
            # yaw command (positive)
            "NUMPAD_7": [0.0, 0.0, 1.0],
            "N": [0.0, 0.0, 1.0],
            # yaw command (negative)
            "NUMPAD_9": [0.0, 0.0, -1.0],
            "M": [0.0, 0.0, -1.0],
        }

Hope this help. Find the training results attached go2_policy.zip

Cheers,
Bryan