Skip to content

README.md

Latest commit

 

History

History
170 lines (131 loc) · 8.45 KB

README.md

File metadata and controls

170 lines (131 loc) · 8.45 KB
Raw

source/ — Genie Sim Module Map

Genie Sim is shipped as a fan-out of peer Python packages plus a couple of non-geniesim modules. Each module lives under source/<name>/ and is self-contained: it has its own pyproject.toml, its own docs, and (for the agent-driven workflows) its own skills/ directory.

This file is the entry index for GitHub readers. Start here, then click into the module you actually need.

🧞 First time? Install the CLI and bootstrap the rest of the stack following source/geniesim_cli/AGENTS.md § 0 — Fresh-machine setup — that's the canonical onboarding path.

📦 Genie Sim Python packages

All editable-installable via pip install -e source/<pkg>/. The geniesim umbrella pulls them in as deps; geniesim bootstrap runs the installs in topological order.

Module Role Entry doc Skills
🧞 geniesim_cli/ CLI dispatcher — the geniesim console script AGENTS.md —
🌐 geniesim/ Umbrella meta-package — pulls in every peer — —
🧪 geniesim_benchmark/ Benchmark tasks, scoring, LLM eval configs (legacy stack — Isaac Sim direct; will be refactored onto geniesim_ros) README.md run-benchmark, check-inference
🏗️ geniesim_generator/ Scene generation, procedural layout, LLM scene language AGENTS.md generate-scene, search-assets, deploy-generator
⚡ geniesim_ros/ Genie Sim RT Engine — realtime interactive ROS 2 simulation README.md · AGENTS.md build-workspace, launch-scene, moveit-wbc, add-robot, teleop-bridge, record-episode, debug-physics, material-override
🎮 geniesim_teleop/ VR / Pico teleoperation bridge AGENTS.md run-teleop
🌍 geniesim_world/ Multimodal spatial world model (panorama → 3D world) README.md · AGENTS.md generate-world

Where do I start?

Install + docker via geniesim_cli, then pick a stack — they are independent and parallel today:

geniesim_cli      → install + docker
   │
   ├── geniesim_benchmark   legacy benchmark runtime (Isaac Sim direct)
   │                        — run / score / evaluate policies headless
   │
   └── geniesim_ros         Genie Sim RT Engine (ROS 2 native)
                            — interactive sim, MoveIt, teleop, recording

🚧 Roadmap. geniesim_benchmark will be refactored into a benchmark layer on top of geniesim_ros so both paths share one physics + render stack. Until then, treat them as independent — don't unify their scene formats or launch graphs.

Side branches:

  • Author a new scene from a panorama → geniesim_world
  • Synthesise scenes / search assets via LLM → geniesim_generator
  • Train an RL policy in the loop → rlinf_geniesim

🔗 Module dependency DAG

How the geniesim_* peers depend on each other at build time and runtime.

Legend: -->|build| build-system requires · ==>|exec| runtime dependency · -.->|[X] extra| optional extra · -.->|refactor: layer atop| planned refactor target. Methodology + how to regenerate: see AGENTS.md § Module dependency DAG — methodology.

graph TD
  %% Auto-generated by `geniesim tool deps-dag --fix`.
  %% Edges: -->|build| build-system requires; ==>|exec| runtime deps;
  %%        -.->|[X] extra| optional extras; -.->|refactor| planned.
  %% Source: source/geniesim_*/pyproject.toml + _ANNOTATIONS in
  %% source/geniesim_cli/src/geniesim_cli/commands/tool.py.

  geniesim["<b>geniesim</b><br/>meta-package"]
  geniesim_cli["<b>geniesim_cli</b><br/>console script"]
  geniesim_benchmark["<b>geniesim_benchmark</b><br/>🚧 legacy — Isaac Sim direct"]
  geniesim_generator["<b>geniesim_generator</b><br/>🏗️ scene gen (optional extra)"]
  geniesim_ros["<b>geniesim_ros</b><br/>⚡ RT Engine"]
  geniesim_teleop["<b>geniesim_teleop</b><br/>🎮 VR / Pico"]
  geniesim_world["<b>geniesim_world</b><br/>🌍 pano → 3D world (own env)"]

  geniesim -->|build| geniesim_cli
  geniesim -->|build| geniesim_benchmark
  geniesim -->|build| geniesim_ros
  geniesim_benchmark ==>|exec| geniesim_cli
  geniesim_generator ==>|exec| geniesim_cli
  geniesim_ros ==>|exec| geniesim_cli
  geniesim_teleop ==>|exec| geniesim_cli
  geniesim_teleop ==>|exec| geniesim_ros
  geniesim -."[teleop,all,full] extra".-> geniesim_teleop
  geniesim -."[generator,all,full] extra".-> geniesim_generator
  geniesim -."[world,all,full] extra".-> geniesim_world
  geniesim_benchmark -.->|refactor: layer atop| geniesim_ros

  classDef legacy fill:#fff3cd,stroke:#856404,color:#856404
  classDef leaf fill:#e9ecef,stroke:#6c757d,color:#495057
  classDef placeholder fill:#f8d7da,stroke:#842029,color:#842029
  classDef umbrella fill:#cfe2ff,stroke:#084298,color:#084298
  class geniesim_benchmark legacy
  class geniesim_world leaf
  class geniesim umbrella
Loading

🧩 Separately-maintained modules

These directories live under source/ but are not part of the geniesim_* peer set. They have their own build / run conventions, their own release cadence, and they are not pulled in by geniesim bootstrap. Some predate the geniesim_* reorganisation; others are active out-of-band collaborations.

Directory Description
data_collection/ Data collection client/server, ROS nodes, aimdk protocol
🎓 rlinf_geniesim/ RL training (RLinf, human-in-the-loop, distributed)
scene_reconstruction/ 3D reconstruction pipeline (Dockerfile, third-party deps)
external/ Vendored third-party code (ml-sharp, DA360, …)

🤖 Agent SKILLs

Every skills/<name>/SKILL.md is self-contained and invocable by agentic coding agents (Claude Code et al.). The skill file carries its own trigger phrases, prerequisites, workflow, and copy-paste commands — so an agent can drive the simulator end-to-end without the user reading the source.

Human consumers can cat any skill file for the same recipe:

cat source/geniesim_ros/skills/launch-scene/SKILL.md

Skill conventions:

  • YAML frontmatter with name, description, license, metadata.
  • Sections: When to Use · Critical Patterns · Workflow · Commands · Notes · Resources.
  • Cross-reference related skills by name in Resources.

📜 Conventions

  • pip install -e source/<pkg>/ for every Python package — never pip install geniesim (not on PyPI).
  • AGENTS.md is the engineer-facing routing doc for each package.
  • README.md (where present) is the user-facing pitch / quickstart.
  • skills/ is the agent-facing recipe library — same target audience as a human user looking for a copy-paste guide.

When in doubt, follow the trail: root README → source/AGENTS.md → per-package AGENTS.md → per-package skills/<name>/SKILL.md.

🔗 Top-level pointers