Skip to content

Latest commit

 

History

History
215 lines (172 loc) · 7.16 KB

README.md

File metadata and controls

215 lines (172 loc) · 7.16 KB

FALCON Deployment Guide

This folder provides seamless sim2sim/sim2real deployment scripts for both Unitree G1 (H1, H1-2) and Booster T1. The key idea is to simulate the real robot's state publisher and command receiver in Mujoco:

Table of Contents

Pre-Configuration

Here, we use config/g1/g1_29dof.yaml. Before testing sim2sim/sim2real, check the ROBOT_TYPE, SDK_TYPE, and INTERFACE in the .yaml file:

ROBOT_TYPE: 'g1_29dof' # Robot name, "t1_29dof", "g1_29dof"...

ROBOT_SCENE: "../humanoidverse/data/robots/g1/scene_29dof_freebase.xml" # Robot scene, for Sim2Sim

ASSET_ROOT: "../humanoidverse/data/robots/g1" # Robot Asset Root

DOMAIN_ID: 0 # Domain id
# IP Interface 
# For sim2sim, 'lo' is for linux and 'lo0' is for mac.
# For sim2real, this needs to be specified according to your host ip, e.g., 'en0'
INTERFACE: "lo"

SDK_TYPE: "unitree" # SDK type, "unitree", "booster"
MOTOR_TYPE: "serial" # Motor type, "serial" or "parallel"

USE_JOYSTICK: 0 # Simulate Unitree WirelessController using a gamepad (0: disable, 1: enable)
JOYSTICK_TYPE: "xbox" # support "xbox" and "switch" gamepad layout; Unitree WirelessController is "xbox" layout.

Installation

Prebuild environment

  • OS (Ubuntu 22.04 LTS)
  • CPU (aarch64 and x86_64)
  • Compiler (gcc version 11.4.0)

Create a conda env

conda create -n fcreal python=3.10
conda activate fcreal

Install Pinocchio for Inverse Kinematics

conda install pinocchio=3.2.0 -c conda-forge

Install unitree_sdk2_python for Unitree G1 deployment

git clone https://github.com/unitreerobotics/unitree_sdk2_python.git
cd unitree_sdk2_python
pip install -e .

Install booster_robotics_sdk for Booster T1 deployment

Note that the official booster_robotics_sdk does not provide state publisher and command receiver, so I improve the repo a bit and add them in my forked repo. Also booster sdk is NOT supported on Mac OS yet.

git clone https://github.com/hang0610/booster_robotics_sdk
# Install python package for building python binding locally
pip3 install pybind11
pip3 install pybind11-stubgen
# Build & Install
mkdir build
cd build
cmake .. -DBUILD_PYTHON_BINDING=on
make
sudo make install

Install others

cd sim2real
pip install -r requirements.txt

Deployment

Important

For sim2sim, you need to start Mujoco and then launch the policy, but for sim2real, you only need to launch the policy. Make sure you read the keyboard and joystick control protocol in sim2real/rl_policy/base_policy.py. All the deployment scripts are running under sim2real, so do cd sim2real first.

TEST with G1_29DoF Locomotion

G1 29DoF Locomotion

Here, we fix the upper body target joint angles to the default, and the policy only outputs the lower body action.

1. Start Mujoco Env (ONLY for Sim2Sim)

python sim_env/base_sim.py \
--config=config/g1/g1_29dof.yaml

2. Launch the Policy

python rl_policy/dec_loco/dec_loco.py \
--config=config/g1/g1_29dof.yaml \
--model_path=models/dec_loco/g1_29dof.onnx
dec_loco_g1.mov

Here are some keyboard shortcuts:

Keyboard Shortcuts in Mujoco
  • 7: raise elastic band height
  • 8: lower elastic band height
  • 9: toggle elastic band
  • backspace: reset simulation
Keyboard Shortcuts in Policy Terminal
  • ]: start using policy actions
  • o: stop using policy action and set actions to zero
  • =: switch between standing and stepping
  • w: increase linear velocity in x direction
  • s: decrease linear velocity in x direction
  • a: increase linear velocity in y direction
  • d: decrease linear velocity in y direction
  • q: decrease angular velocity in z direction
  • e: increase angular velocity in z direction
  • z: set velocity to zero
  • 1: increase base height (if the policy allows)
  • 2: decrease base height (if the policy allows)
  • 5: decrease kp scale by 0.01
  • 6: increase kp scale by 0.01
  • 4: decrease kp scale by 0.1
  • 7: increase kp scale by 0.1
  • 0: reset kp scale to 1.0

G1 29DoF FALCON

1. Start Mujoco Env (ONLY for Sim2Sim)

python sim_env/loco_manip.py \
--config=config/g1/g1_29dof_falcon.yaml

2. Launch the Policy

python rl_policy/loco_manip/loco_manip.py \
--config=config/g1/g1_29dof_falcon.yaml \
--model_path=models/falcon/g1_29dof.onnx
falcon_g1.mp4

T1 29DoF FALCON

1. Start Mujoco Env (ONLY for Sim2Sim)

python sim_env/loco_manip.py \
--config=config/t1/t1_29dof_falcon.yaml

2. Luanch the Policy

python rl_policy/loco_manip/loco_manip.py \
--config=config/t1/t1_29dof_falcon.yaml \
--model_path=models/falcon/t1_29dof.onnx
falcon_t1.mp4

Sim2Real Tips

Caution

FALCON is a strong policy trained for robust locomotion and manipulation. Before deploying to real robots, ensure:

Network Configuration

  • Set correct INTERFACE in config file (e.g., 'en0', 'eth0')
  • Verify network connectivity between computer and robot
  • Check firewall settings if using specific ports

Testing Protocol

  1. Always do sim2sim before real-robot deployment
  2. Start with small kp, kd gains
  3. Ensure robot feet touch the ground before running falcon policies

Emergency Control

  • Keyboard: Press 'o' to stop policy actions
  • Joystick: Press 'B+Y' to stop policy actions

Real-time Inference

  • unitree_sdk2_python can not guarantee real-time inference on Jetson Orin inside of Unitree G1 as unitree_sdk2_python is fully written in python, while booster_robotics_sdk works fine as its backend is written in cpp with a pybinding wrapper.
  • It's recommended to use unitree_sdk2 for real-time onboard inference on Unitree G1. Please check this repo for the pybinding wrapper I wrote for unitree_sdk2.

I recommend you to use unitree_sdk2 for real-time inference onboard. I have written a pybinding wrapper for it. Please check this repo (currently no README for this pybinding wrapper, but will update soon).

Acknowledgement

We thank the following open-sourced repos that we build upon: