Skip to content

README.md

Latest commit

 

History

History
executable file
·
309 lines (222 loc) · 10.5 KB

README.md

File metadata and controls

executable file
·
309 lines (222 loc) · 10.5 KB
Raw

Data Collection

A data collection system for robotic simulation tasks using Isaac Sim and cuRobo.

Table of Contents

Prerequisites

  • Docker (for containerized deployment)
  • NVIDIA GPU with CUDA support (40 series GPU recommended, for 50 series GPU (SM_120) may not be able to install cuRobo)
  • Python 3.11
  • Conda (for local deployment)

Getting Started

Note: Before running, geniesim_assets must be pip-installed (editable) on the host. The CLI discovers it (find_spec, like geniesim docker), bind-mounts it to /geniesim_assets, and the entrypoint editable-installs it:

pip install -e /path/to/geniesim_assets

Option 1: Docker Container (Recommended)

Building the Docker Image

Recommended — via the CLI (wraps the build with the right context and tag):

geniesim autocollect build

Or directly — assuming the benchmark base image registry.agibot.com/genie-sim/geniesim3:latest is built:

docker build -f ./dockerfile -t registry.agibot.com/genie-sim/geniesim3-data-collection:latest .

Note: For cuRobo installation, the Dockerfile is configured for RTX 4090D by default. If you're using a different GPU model, you need to modify the TORCH_CUDA_ARCH_LIST environment variable in the Dockerfile, 50 series GPU (SM_120) may not be able to install cuRobo, this needs a compatibility update by the cuRobo team.

One-Click Data Collection (Recommended)

The recommended entry point is the geniesim autocollect CLI (ships with geniesim_cli; see the repo README § 3.8). It auto-discovers the editable-installed geniesim_assets, resolves the task by name, and wraps the host orchestrator below:

geniesim autocollect list --robot=g2 <substr>            # discover tasks
geniesim autocollect run <TASK> --headless --standalone  # collect (add --dry-run to preview)

Or call the underlying script directly:

Usage:

./scripts/run_data_collection.sh [OPTIONS]

Options:

  • --headless - Run in headless mode (default: false)
  • --no-record - Disable recording (default: record enabled)
  • --task TASK_PATH - Task template path (e.g. tasks/geniesim_2025/sort_fruit/g2/sort_the_fruit_into_the_box_apple_g2.json)
  • --standalone - Run in standalone mode (only save logs, no terminal output) (default: false)
  • --container-name NAME - Container name (default: data_collection_open_source)
  • --help, -h - Show help message

Assets:

  • geniesim_assets must be pip-installed (editable) on the host. The CLI auto-discovers it and mounts it at /geniesim_assets — no manual SIM_ASSETS export needed.

Examples:

# Run with default task in GUI mode
./scripts/run_data_collection.sh

# Run in headless mode with custom task
./scripts/run_data_collection.sh --headless --task tasks/geniesim_2025/sort_fruit/g2/sort_the_fruit_into_the_box_apple_g2.json

# Run in standalone headless mode (logs only, no terminal output)
./scripts/run_data_collection.sh --standalone --headless

# Run without recording
./scripts/run_data_collection.sh --no-record

Logs: Logs are saved to logs/{TASK_NAME}/ directory:

  • run_data_collection_sh.log - Script output
  • container.log - Container logs
  • data_collector_server.log - Data collector server logs (if available)
  • run_data_collection.log - data collection application logs (if available)

Outputs: Outputs are save to recording_data/[{TASK_NAME}_{INDEX}]/ directory

Interactive Mode

Via the CLI: geniesim autocollect up creates/starts the GUI container and geniesim autocollect into drops you into a shell in it (wrapping the script below).

Or use scripts/start_gui.sh directly to launch an interactive container for debugging or development.

Usage:

./scripts/start_gui.sh [ACTION] [CONTAINER_NAME]

Actions:

  • run (default) - Create and run a new container
  • exec - Enter an existing container
  • start - Start a stopped container
  • restart - Restart a container

Parameters:

  • ACTION - One of: exec, start, restart, run (default: run)
  • CONTAINER_NAME - Container name (default: data_collection_open_source)

Examples:

# Create and run a new container (default)
./scripts/start_gui.sh run my_container

# Enter an existing container
./scripts/start_gui.sh exec my_container

# Start a stopped container
./scripts/start_gui.sh start my_container

# Restart a container
./scripts/start_gui.sh restart my_container

Running Services Inside Container:

After entering the container using exec, you need to start two services in separate terminals:

Terminal 1 - Start the container and run data collector server:

# Enter the container
./scripts/start_gui.sh exec my_container

# Inside container, start data collector server
python scripts/data_collector_server.py --enable_physics --enable_curobo --publish_ros

Terminal 2 - Enter the same container and run data collection application:

# Enter the same container (in a new terminal)
./scripts/start_gui.sh exec my_container

# Inside container, run data collection application
python scripts/run_data_collection.py --task_template tasks/geniesim_2025/sort_fruit/g2/sort_the_fruit_into_the_box_apple_g2.json --use_recording

Note: Both terminals need to exec into the same container. Make sure the container is running before executing these commands.

Option 2: Local Deployment

1. Create Conda Environment

conda create -n data_collect python=3.11
conda activate data_collect

2. Install Dependencies

pip install -r requirements.txt
pip install "isaacsim[all,extscache]==5.1.0" --extra-index-url https://pypi.nvidia.com

3. Install cuRobo

Clone cuRobo and install:

# Set CUROBO_DIR to your cuRobo installation directory
export CUROBO_DIR=/path/to/cuRobo

# Copy robot assets and configs
cp -r ${SIM_ASSETS}/robot/curobo_robot/assets/robot ${CUROBO_DIR}/src/curobo/content/assets
cp -r config/curobo/configs ${CUROBO_DIR}/src/curobo/content/

# Install cuRobo
cd ${CUROBO_DIR} && pip install -e ".[isaacsim]" --no-build-isolation

Note: Make sure to set TORCH_CUDA_ARCH_LIST according to your GPU architecture before installing cuRobo. For RTX 4090D, use:

export TORCH_CUDA_ARCH_LIST="8.9"

4. Setup ROS2

Install ROS2 on your local, who should be located in either /opt/ros/humble/ or /opt/ros/jazzy/. Then set environment variables, which can also be insert to your ~/.bashrc:

export ROS_DISTRO=jazzy # or humble
export ROS_CMD_DISTRO=${ROS_DISTRO}
export CONDA_SITE_PACKAGES=YOUR_CONDA_ENV_SITE_PACKAGES_PATH # e.g. ~/anaconda3/envs/data_collect/lib/python3.11/site-packages/
export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:${CONDA_SITE_PACKAGES}/isaacsim/exts/isaacsim.ros2.bridge/${ROS_DISTRO}/lib
export RMW_IMPLEMENTATION=rmw_cyclonedds_cpp

5. Start the Services

You need to run two services in separate terminals:

Terminal 1 - Data Collector Server:

python scripts/data_collector_server.py [OPTIONS]

Options for data_collector_server.py:

  • --headless - Run in headless mode
  • --enable_physics - Enable physics simulation
  • --enable_curobo - Enable cuRobo motion planning
  • --publish_ros - Publish ROS messages, MUST be set if recording is needed

Terminal 2 - Data Collection Application:

python scripts/run_data_collection.py [OPTIONS]

Options for run_data_collection.py:

  • --use_recording - Use recording mode, MUST be set if recording is needed
  • --task_template - Task template JSON file path

Example:

# Terminal 1
python scripts/data_collector_server.py --enable_physics --enable_curobo --publish_ros

# Terminal 2
python scripts/run_data_collection.py --task_template tasks/geniesim_2025/sort_fruit/g2/sort_the_fruit_into_the_box_apple_g2.json --use_recording

Usage Examples

Docker - Automated Data Collection

# Install geniesim_assets (editable, once on the host)
pip install -e /path/to/geniesim_assets

# Run data collection with custom task
./scripts/run_data_collection.sh \
  --headless \
  --task tasks/geniesim_2025/sort_fruit/g2/sort_the_fruit_into_the_box_apple_g2.json

Docker - Interactive Development

# Install geniesim_assets (editable, once on the host)
pip install -e /path/to/geniesim_assets

# Start interactive container
./scripts/start_gui.sh run my_container

# Terminal 1: Enter container and start data collector server
./scripts/start_gui.sh exec my_container
# Inside container:
python scripts/data_collector_server.py --enable_physics --enable_curobo --publish_ros

# Terminal 2: Enter the same container and run data collection application
./scripts/start_gui.sh exec my_container
# Inside container:
python scripts/run_data_collection.py --task_template tasks/geniesim_2025/sort_fruit/g2/sort_the_fruit_into_the_box_apple_g2.json --use_recording

Local - Full Control

# Terminal 1: Start server with physics and cuRobo
python scripts/data_collector_server.py --enable_physics --enable_curobo --publish_ros

# Terminal 2: Run data collection
python scripts/run_data_collection.py \
  --task_template tasks/geniesim_2025/sort_fruit/g2/sort_the_fruit_into_the_box_apple_g2.json \
  --use_recording

Task Configuration Guide

To create and configure data collection tasks, refer to the Task Configuration Guide. This comprehensive guide covers:

  • Creating task configuration files from scratch
  • Configuring scenes, robots, and objects
  • Setting up task stages (pick, place, insert, rotate, reset)
  • Configuring runtime checkers and data filter rules
  • Understanding action parameters and workspace types
  • Complete examples and best practices

The guide provides detailed explanations and examples for all configuration options, making it easy to create custom tasks for your data collection needs.