|
2 | 2 | <img src="assets/nvidia-cosmos-header.png" alt="NVIDIA Cosmos Header"> |
3 | 3 | </p> |
4 | 4 |
|
5 | | -Thank everybody for the feedback. We have learned your feedback and decided to simplify our code structure to make it easier to use. |
6 | | -Code is this repo **will be deprecated soon**! Please move to the new codebase that is actively developed and maintained. The list of the new |
| 5 | +Thank you all for the valuable feedback! We have restructured the codebase to make it easier to use and contribute to. |
7 | 6 |
|
8 | | -| Cosmos World Foundation Model Family||||| |
9 | | -| ----------------- | ----------------- | ----------------- | ----------------- |----------------- | |
10 | | -| [Cosmos-Predict1](https://github.com/nvidia-cosmos/cosmos-predict1) | World Models | [Code](https://github.com/nvidia-cosmos/cosmos-predict1) | [Models](https://huggingface.co/collections/nvidia/cosmos-predict1-67c9d1b97678dbf7669c89a7) | [Paper](https://arxiv.org/abs/2501.03575) | |
11 | | -| [Cosmos-Transfer1](https://github.com/nvidia-cosmos/cosmos-transfer1) | Control Nets | [Code](https://github.com/nvidia-cosmos/cosmos-transfer1) | [Models](https://huggingface.co/collections/nvidia/cosmos-transfer1-67c9d328196453be6e568d3e) | [Paper](https://arxiv.org/abs/2503.14492) | |
12 | | -| [Cosmos-Reason1](https://github.com/nvidia-cosmos/cosmos-reason1) | Reasoning Models | Coming soon | Coming soon | [Paper](https://arxiv.org/abs/2503.15558) | |
13 | | -| | | | |
14 | | - |
15 | | -[NVIDIA Cosmos](https://www.nvidia.com/cosmos/) is a developer-first world foundation model platform designed to help Physical AI developers build their Physical AI systems better and faster. Cosmos contains |
16 | | - |
17 | | -1. Pre-trained models (available via Hugging Face) under the [NVIDIA Open Model License](https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-open-model-license/) that allows free commercial use. |
18 | | -2. Pre-training, post-training, and inference code (available in native PyTorch) under the [Apache 2 License](https://www.apache.org/licenses/LICENSE-2.0). |
19 | | - |
20 | | -There are three main model families in Cosmos World Foundation Model Platform. |
21 | | - |
22 | | -1. [Cosmos Predict](https://github.com/nvidia-cosmos/cosmos-predict1): a collection of general-purpose world models for future state prediction. |
23 | | - |
24 | | -2. [Cosmos Transfer](https://github.com/nvidia-cosmos/cosmos-transfer1): a collection of multimodal conditional world generation model for various domain transfer applications such as Sim2Real. |
25 | | - |
26 | | -3. [Cosmos Reason](https://github.com/nvidia-cosmos/cosmos-reason1): a collection of Physical AI reasoning models for planning and critics. |
27 | | - |
28 | | -Being a minimalist, we have these individual models in individual repositories under [nvidia-github](https://github.com/nvidia-cosmos). |
29 | | - |
30 | | - |
31 | | -### Example Model Behavior |
32 | | - [Cosmos-Predict Text2World](https://github.com/nvidia-cosmos/cosmos-predict1) |
33 | | - |
34 | | -<video src="https://github.com/user-attachments/assets/b001966c-5f5e-4927-a3fe-44d142dd0ab1"> Your browser does not support the video tag.</video> |
35 | | - |
36 | | -[Cosmos-Predict Video2World](https://github.com/nvidia-cosmos/cosmos-predict1) |
37 | | - |
38 | | -<video src="https://github.com/user-attachments/assets/0bbba982-c6fd-4388-a46f-bf91ce4099ad"> Your browser does not support the video tag. </video> |
39 | | - |
40 | | -[Cosmos-Transfer LiDAR + HDMap Conditional Inputs -> World](https://github.com/nvidia-cosmos/cosmos-transfer1) |
41 | | - |
42 | | -<video src="https://github.com/user-attachments/assets/71faa274-a238-47c9-b2ae-5b3ea08cb643"> Your browser does not support the video tag. </video> |
43 | | - |
44 | | -[Cosmos-Transfer Multimodal Conditional Inputs -> World](https://github.com/nvidia-cosmos/cosmos-transfer1) |
45 | | - |
46 | | -<video src="https://github.com/user-attachments/assets/f04f430a-dc64-4ef8-b66a-70625edf860c"> Your browser does not support the video tag. </video> |
47 | | - |
48 | | -[Cosmos-Reason Physical AI Planning](https://github.com/nvidia-cosmos/cosmos-transfer1) |
| 7 | +<p align="center"> |
| 8 | + <span style="font-size: 18px;">New GitHub page for NVIDIA Cosmos: <b><a href="https://github.com/nvidia-cosmos">https://github.com/nvidia-cosmos</a></b></span> |
| 9 | +</p> |
49 | 10 |
|
50 | | -<video src="https://github.com/user-attachments/assets/46b8088b-d8ad-46a1-b700-2ed4c8d3fc9c"> Your browser does not support the video tag. </video> |
| 11 | +NVIDIA Cosmos now includes three subprojects: |
51 | 12 |
|
52 | | -### Cosmos Publication |
| 13 | +1. [Cosmos-Predict1](https://github.com/nvidia-cosmos/cosmos-predict1) is a collection of general-purpose world foundation models for Physical AI that can be fine-tuned into customized world models for downstream applications. |
| 14 | +2. [Cosmos-Transfer1](https://github.com/nvidia-cosmos/cosmos-transfer1) is a world-to-world transfer model designed to bridge the perceptual divide between simulated and real-world environments. |
| 15 | +3. [Cosmos-Reason1](https://github.com/nvidia-cosmos/cosmos-reason1) models understand the physical common sense and generate appropriate embodied decisions in natural language through long chain-of-thought reasoning processes. |
53 | 16 |
|
54 | | -<table> |
55 | | - <tr> |
56 | | - <th width="40%">Paper Title</th> |
57 | | - <th width="30%">Summary</th> |
58 | | - <th width="15%">Authors</th> |
59 | | - <th width="15%">Date</th> |
60 | | - </tr> |
61 | | - <tr> |
62 | | - <td><a href="https://arxiv.org/abs/2503.15558">Cosmos-Reason1: From Physical Common Sense To Embodied Reasoning</a></td> |
63 | | - <td>Introduces a reasoning model for physical AI that combines common sense knowledge with embodied reasoning capabilities.</td> |
64 | | - <td>NVIDIA</td> |
65 | | - <td>2025-03-19</td> |
66 | | - </tr> |
67 | | - <tr> |
68 | | - <td><a href="https://arxiv.org/abs/2503.14492">Cosmos-Transfer1: Conditional World Generation with Adaptive Multimodal Control</a></td> |
69 | | - <td>Presents a multimodal model for conditional world generation with adaptive control mechanisms.</td> |
70 | | - <td>NVIDIA</td> |
71 | | - <td>2025-03-18</td> |
72 | | - </tr> |
73 | | - <tr> |
74 | | - <td><a href="https://arxiv.org/abs/2501.03575">Cosmos World Foundation Model Platform for Physical AI</a></td> |
75 | | - <td>Overview of the Cosmos platform, its architecture, and applications in physical AI systems. Introduction of Cosmos-Predict1 world models.</td> |
76 | | - <td>NVIDIA</td> |
77 | | - <td>2025-01-06</td> |
78 | | - </tr> |
79 | | -</table> |
| 17 | +----------------------------------------------------------- |
80 | 18 |
|
81 | | -### Developer |
82 | | -For native PyTorch developers, we provide native PyTorch training and inference scripts in [nvidia-github](https://github.com/nvidia-cosmos). For Nemo developers, please refer to [README_CES2025.md](README_CES2025.md). |
| 19 | +**This repository is no longer maintained and will soon be deprecated.** To view the initial release of NVIDIA Cosmos from this repository, please check out branch `archived-ces2025`. |
0 commit comments