[MISC] Vision-based manipulation using batch render and two-stage policy training

[yun-long]

Description

This PR adds an end‑to‑end training script for a two‑stage manipulation pipeline:

• Stage 1 – Privileged RL (teacher): Train a PPO policy with state privileges using rsl-rl-lib’s OnPolicyRunner.

• Stage 2 – Vision Behavior Cloning (student): Distill the teacher into a vision-based policy that takes RGB + low‑dim state using a lightweight CNN encoder.

Motivation

• To demonstrate using madrona render for vision-based manipulation
• Provide a simple, reproducible path to train a performant teacher and distill it into a student policy suitable for vision-only or vision+state deployment. To simulate RGB images, it uses gs-madrona batch render.

The policy controls the robot from a stereo camera (left and right) fixed in the real world.

How Has This Been / Can This Be Tested?

To test the code, you need to run the training script and inspect the training curve / final policy performance manually.

Train privileged RL

python examples/manipulation/grasp_train.py --stage=rl

Test privileged RL

python examples/manipulation/grasp_eval.py --stage=rl

Train Vision-based Policy using BC

python examples/manipulation/grasp_train.py --stage=bc 

Test Vision-based Policy

python examples/manipulation/grasp_eval.py --stage=bc

Screenshots

evaluation.mp4

Checklist:

• I read the CONTRIBUTING document.
• I followed the Submitting Code Changes section of CONTRIBUTING document.
• I tagged the title correctly (including BUG FIX/FEATURE/MISC/BREAKING)
• I updated the documentation accordingly or no change is needed.
• I tested my changes and added instructions on how to test it for reviewers.

[YilingQiao]

can you provide some instructions about the commands that I need to run it on my side?

[yun-long]

can you provide some instructions about the commands that I need to run it on my side?

I have updated the descriptions. You can to test the example via the provided commands.

[YilingQiao]

Could you please add the description in this PR as a READE.md to this folder? Such that users have an idea how to use it.

[yun-long]

Could you please add the description in this PR as a READE.md to this folder? Such that users have an idea how to use it.

Yes. I will add proper documentation this week.

[yun-long]

See this PR for updated documentations.