A comprehensive guide for working with the NVIDIA Cosmos ecosystem—a suite of World Foundation Models (WFMs) for real-world, domain-specific applications across robotics, simulation, autonomous systems, and physical scene understanding.
📚 View the Full Documentation → — Step-by-step workflows, case studies, and technical recipes
carousel.mp4
| Date | Recipe | Model | Description |
|---|---|---|---|
| Feb 4 | Worker Safety in a Classical Warehouse | Cosmos Reason 2 | Zero-shot industrial safety compliance and hazard detection in classical warehouse environments using context-aware prompt engineering |
| Jan 30 | Prompt Guide | Cosmos Reason 2 | Inference Prompt Guide |
| Jan 29 | Video Search and Summarization with Cosmos Reason | Cosmos Reason 2 | GPU-accelerated video analysis pipeline for large-scale video summarization, Q&A, and live stream alerts across warehouses, factories, retail, and smart cities |
| Jan 28 | Cosmos Policy: Fine-Tuning Video Models for Visuomotor Control and Planning | Cosmos Predict 2 | State-of-the-art robot policy through latent frame injection for visuomotor control achieving 98.5% on LIBERO, 67.1% on RoboCasa, and 93.6% on ALOHA |
| Jan 27 | Physical Plausibility Prediction with Cosmos Reason 2 | Cosmos Reason 2 | Supervised fine-tuning for physical plausibility prediction using VideoPhy-2 dataset with improved zero-shot and SFT performance |
| Jan 26 | Intelligent Transportation Post-Training with Cosmos Reason 2 | Cosmos Reason 2 | Post-training Cosmos Reason 2 for intelligent transportation scene understanding using WovenTraffic Safety dataset |
Register for NVIDIA GTC happening March 16–19, 2026, and add the Cosmos sessions to your calendar. Don't miss the must-see keynote from CEO Jensen Huang at SAP Center on Monday, March 16 at 11:00 a.m. PT.
Introducing the NVIDIA Cosmos Cookoff — a virtual, four-week physical AI challenge running January 29 – February 26 for robotics, AV, and vision AI builders.
Build with NVIDIA Cosmos Reason and Cosmos Cookbook recipes—from egocentric robot reasoning to physical plausibility checks and traffic-aware models for a chance to win $5,000, an NVIDIA DGX Spark, and more!
Sponsored by Nebius and Milestone.
| Use Case | Linux (Ubuntu) | macOS | Windows |
|---|---|---|---|
| Running cookbook recipes (GPU workflows) | ✅ Supported | ❌ | ❌ |
| Local documentation & contribution | ✅ Supported | ✅ Supported |
- Git with Git LFS
- Python: Version 3.10+
- Internet access for cloning and dependencies
Full GPU workflows require an Ubuntu Linux environment with NVIDIA GPUs.
→ See Getting Started for complete hardware and software requirements.
→ Or Deploy on Cloud (Nebius, Brev, and more to come) for ready-to-launch GPU instances.
⚠️ Important: This repository contains many media files (videos, images, demonstrations). Git LFS is required to clone and work with this repository properly.
# Ubuntu/Debian (recommended)
sudo apt update && sudo apt install git-lfs
# Enable Git LFS globally
git lfs installFor other platforms (macOS, Windows, Fedora), see the official installation guide at git-lfs.com.
If you've already cloned without LFS, fetch the media files with:
git lfs pull# Install uv (fast Python package manager)
curl -LsSf https://astral.sh/uv/install.sh | sh
source $HOME/.local/bin/env
# Install just (command runner)
uv tool install -U rust-justFor other platforms, see astral.sh/uv for installation instructions.
# Clone the repository
git clone https://github.com/nvidia-cosmos/cosmos-cookbook.git
cd cosmos-cookbook
# Install dependencies and setup
just install# Serve documentation locally
just serve-external # For public documentation
# or
just serve-internal # For internal documentation (if applicable)Then open http://localhost:8000 in your browser.
The Cosmos Cookbook is organized into two main directories:
Contains the source documentation in markdown files:
- Technical guides and workflows
- End-to-end examples and case studies
- Step-by-step recipes and tutorials
- Getting started guides
Contains executable scripts referenced throughout the cookbook:
- Data processing and curation pipelines
- Model evaluation and quality control scripts
- Configuration files for post-training tasks
- Automation tools and utilities
This structure separates documentation from implementation, making it easy to navigate between reading about workflows and executing the corresponding scripts.
When contributing media files, prefer .mp4 over .gif:
- Better quality — MP4 supports full color depth vs GIF's 256-color limit
- Smaller file size — Modern video codecs compress far more efficiently
- Audio support — MP4 can include narration when needed
Use H.264 encoding for universal browser compatibility.
# Development
just install # Install dependencies and setup
just setup # Setup pre-commit hooks
just serve-external # Serve public documentation locally
just serve-internal # Serve internal documentation locally
# Quality Control
just lint # Run linting and formatting
just test # Run all tests and validation
# Continuous Integration
just ci-lint # Run CI linting checks
just ci-deploy-internal # Deploy internal documentation
just ci-deploy-external # Deploy external documentation- Contributing Guide - How to contribute to the cookbook
- Report Issues: Use GitHub Issues for bugs and feature requests
- Share Success Stories: We love hearing how you use Cosmos models creatively
This project will download and install additional third-party open source software projects. Review the license terms of these open source projects before use.
NVIDIA Cosmos source code is released under the Apache 2 License.
NVIDIA Cosmos models are released under the NVIDIA Open Model License. For a custom license, please contact [email protected].