|
1 | 1 | # 🌍 ABot-World: Real-Time Interactive World Simulation on a Single Desktop GPU |
2 | 2 |
|
3 | 3 | [](https://amap-cvlab.github.io/ABot-World/) |
| 4 | +[](https://abot-world.amap.com) |
4 | 5 | [](#) |
5 | | -[](https://github.com/amap-cvlab/ABot-World) |
6 | | -[](#) |
| 6 | +[](https://github.com/amap-cvlab/ABot-World) |
| 7 | +[](https://huggingface.co/acvlab/ABot-World-0-5B-LF) |
| 8 | +[](https://modelscope.cn/models/amap_cvlab/ABot-World-0-5B-LF) |
7 | 9 |
|
8 | | -> **TL;DR:** ABot-World is a next-generation Action-Conditioned World Model that achieves **real-time interactive inference on a single consumer-grade Desktop GPU**. |
9 | | -
|
10 | | -We are currently preparing the technical report, code, and model weights for public release. In the meantime, explore our interactive demos below. |
| 10 | +> **TL;DR:** ABot-World turns a single NVIDIA RTX 5090 desktop GPU into a real-time interactive world simulator, enabling infinite action-conditioned world rollout at 720P, 16 FPS, 1.2s latency, and 19GB GPU memory. |
11 | 11 |
|
12 | 12 | ## 🚀 Key Highlights |
13 | 13 |
|
14 | | -* ⚡ **Real-Time Desktop Inference:** Breaking the latency barrier. Through highly optimized inference architecture and compute-efficient scaling, ABot-World enables fluid, real-time rollout generation on a single desktop GPU. You no longer need a data center to simulate the world. |
15 | | -* 🧠 **Native Spatial Intelligence**: To capture the infinite variations of the physical world, ABot-World extracts robust spatial structures and complex dynamics implicitly. By natively embedding physical priors into its generation process, it seamlessly unifies visual rendering and physical simulation within the latent space. |
16 | | -* ♾️ **Long-Horizon Consistency:** Exposure bias and compounding errors have long plagued auto-regressive generation. By introducing a novel Parametric Memory module that acts as a global state sink, ABot-World maintains strict spatial topology and physical consistency over extraordinarily long rollouts. |
17 | | -* 🏆 **Top-Tier Physical Reasoning:** Evaluated against state-of-the-art models, ABot-World achieves top-tier performance on industry-standard benchmarks including [**VBench 2.0**](https://huggingface.co/spaces/Vchitect/VBench_Leaderboard) (annouced by [ABot-World v0.1](https://github.com/amap-cvlab/ABot-World)), [**WorldArena**](https://huggingface.co/spaces/WorldArena/WorldArena) & [**GigaBran**](https://huggingface.co/spaces/open-gigaai/CVPR-2026-WorldModel-Track-LeaderBoard) (announced by [ABot-PhysWorld](https://huggingface.co/spaces/WorldArena/WorldArena)) and [**WorldScore**](https://huggingface.co/spaces/Howieeeee/WorldScore_Leaderboard) (announced by [FantasyWorld 1.0](https://fantasy-amap.github.io/fantasy-world/)), particularly excelling in physics adherence and dynamic motion quality. |
| 14 | +* 🎮 **Action-Driven World Control:** Responds to user actions in real time, enabling continuous exploration instead of passive video playback. |
| 15 | +* ⚡ **Real-Time Desktop Inference:** Runs at 720p and 16 FPS on a single NVIDIA RTX 5090 desktop GPU, with 1.2s latency and 19GB GPU memory. |
| 16 | +* ♾️ **Infinite World Rollout:** Supports open-ended interactive world generation beyond fixed video-length limits. |
| 17 | +* 🧠 **Open-Ended World Imagination:** Expands the world with new scenes and dynamics during rollout, avoiding scene lock-in, without prompt switching. |
| 18 | + |
| 19 | +## 📢 News |
| 20 | +- 2026-07-09: We release the causal student model `ABot-World-0-5B-LF`, inference code, our local gradio demo and online playground [ABot World Studio](https://abot-world.amap.com). |
| 21 | + |
| 22 | +## 🛠️ Setup |
| 23 | + |
| 24 | +> This installation was tested on: Ubuntu 22.04, CUDA 13.3, NVIDIA RTX 5090. |
| 25 | +
|
| 26 | +1. Clone the repository: |
| 27 | + |
| 28 | +```bash |
| 29 | +git clone https://github.com/amap-cvlab/ABot-World.git |
| 30 | +cd ABot-World |
| 31 | +``` |
| 32 | + |
| 33 | +2. Install dependencies using conda: |
| 34 | + |
| 35 | +```bash |
| 36 | +conda create -n aworld python=3.12 -y |
| 37 | +conda activate aworld |
| 38 | +pip install -r requirements.txt |
| 39 | +``` |
| 40 | + |
| 41 | +3. Download checkpoints: |
| 42 | + |
| 43 | +```bash |
| 44 | +pip install -U "huggingface_hub" |
| 45 | +hf download <HF_REPO_ID> --local-dir checkpoints |
| 46 | +``` |
| 47 | + |
| 48 | +After downloading, the project should have the following checkpoint structure: |
| 49 | + |
| 50 | +```text |
| 51 | +checkpoints/ |
| 52 | +└── ABot-World-0-5B-LF/ |
| 53 | + ├── Wan2.2_VAE.pth |
| 54 | + ├── taew2_2.pth |
| 55 | + ├── models_t5_umt5-xxl-enc-bf16.pth |
| 56 | + ├── diffusion_pytorch_model.safetensors |
| 57 | + └── google/umt5-xxl/ |
| 58 | +``` |
| 59 | + |
| 60 | +The checkpoint paths are configured in `configs/long_forcing_dmd.yaml` and |
| 61 | +`configs/default_config.yaml`. The distilled generator weights are already |
| 62 | +merged into `ABot-World-0-5B-LF/diffusion_pytorch_model.safetensors`. |
18 | 63 |
|
19 | | -## 🕹️ Interactive Demonstrations |
| 64 | +## 🤗 Gradio Demo |
20 | 65 |
|
21 | | -### 1. Open World Exploration |
22 | | -Experience fluid and consistent generation as you interact with the environment. Whether navigating via first-person camera adjustments or driving a character through diverse scenes, the simulation responds dynamically to your inputs. The model natively understands spatial depth, object permanence, and complex environmental interactions, delivering an immersive experience from any viewpoint. |
| 66 | +1. Run the demo online: 此处预留hf demo 链接 |
| 67 | +2. Run it locally: |
23 | 68 |
|
24 | | -*<img src="assets/gifs/open-world-exploration-main.gif" width="800" alt="More Open World Exploration Demo">* |
| 69 | +```bash |
| 70 | +bash web_client/run.sh |
| 71 | +``` |
25 | 72 |
|
26 | | -#### More Demos |
27 | | -*<img src="assets/gifs/open-world-exploration.gif" width="800" alt="More Open World Exploration Demo">* |
| 73 | +Select a GPU with: |
28 | 74 |
|
29 | | -### 2. Long-Horizon World Exploration via a single Desktop GPU |
30 | | -A continuous, uninterrupted rollout showcasing the power of our Parametric Memory. Through a highly compute-efficient architecture, ABot-World sustains strict global state topology and physical consistency over extended durations, generating expansive environments locally on a single Desktop GPU. |
| 75 | +```bash |
| 76 | +CUDA_ID=0 bash web_client/run.sh |
| 77 | +``` |
31 | 78 |
|
32 | | -*<img src="assets/gifs/long-horizon.gif" width="800" alt="Long Horizon World Exploration Demo">* |
| 79 | +## License |
33 | 80 |
|
34 | | -🔗 **[Explore our previous work FantasyWorld](https://fantasy-amap.github.io/fantasy-world/)** |
| 81 | +This project is released under the Apache License 2.0. See `LICENSE`, `NOTICE`, |
| 82 | +and `THIRD_PARTY_NOTICES.md` for copyright and third-party attribution details. |
35 | 83 |
|
36 | | -### 3. Embodied Interaction & Physics Alignment (ABot-PhysWorld) |
37 | | -A specialized variant of our foundation model, **ABot-PhysWorld**, focuses on robotic manipulation. It uses a novel DPO-based post-training framework with decoupled discriminators to actively suppress unphysical behaviors like object penetration and anti-gravity motion. A parallel context block enables precise spatial action injection for cross-embodiment control. |
| 84 | +## 🤝 Acknowledgement |
38 | 85 |
|
39 | | -🔗 **[Explore the ABot-PhysWorld Repository](https://github.com/amap-cvlab/ABot-PhysWorld)** |
| 86 | +This project builds on and is inspired by the following open-source projects: [Causal Forcing](https://github.com/thu-ml/Causal-Forcing), [AngelSlim](https://github.com/tencent/AngelSlim), [LightX2V](https://github.com/ModelTC/LightX2V), [taehv](https://github.com/madebyollin/taehv), [Wan2.2](https://github.com/Wan-Video/Wan2.2), [Helios](https://github.com/PKU-YuanGroup/Helios), from which the optimized Triton RoPE and normalization kernels in `wan/modules/helios_kernels` are derived. |
40 | 87 |
|
41 | 88 | ## 🗓️ Roadmap |
42 | | -- [x] Teaser & Demo Release |
| 89 | +- [x] Interactive Web Playground (ABot World Studio) |
| 90 | +- [x] Inference Code Release |
| 91 | +- [x] Local Gradio Demo Release |
| 92 | +- [x] Causal Student Model Release |
| 93 | +- [ ] Bidirectional Teacher Model Release |
43 | 94 | - [ ] Technical Report (Arxiv) |
44 | | -- [ ] Inference Code Release |
45 | | -- [ ] Model Weights (Open Source) |
46 | | -- [ ] Interactive Web Playground |
47 | 95 |
|
48 | 96 | ## 📝 Citation |
49 | | -If you find our work helpful, please stay tuned for our upcoming paper. |
| 97 | +If you find our work helpful, please cite our paper: |
| 98 | + |
| 99 | +``` |
| 100 | +@article{abot-world-0, |
| 101 | + title={ABot-World-0: Real-Time Interactive World Simulation on a Single Desktop GPU}, |
| 102 | + author={ABot-World Team}, |
| 103 | + year={2026} |
| 104 | +} |
| 105 | +``` |
0 commit comments