Skip to content

docs: update NeMo RL tutorial container #3078

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
ananthsub wants to merge 1 commit into
base: main
Choose a base branch
from
+7 −7
Open

docs: update NeMo RL tutorial container #3078

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions fern/versions/latest/pages/training-tutorials/nemo-rl-grpo/multi-node-training.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -57,14 +57,14 @@ cd /path/to/nemo/rl
# WANDB_API_KEY: Your Weights & Biases API key for logging
# EXP_NAME: Experiment name
# NUM_ACTOR_NODES: Number of GPU nodes to use (2, 4, 8, etc.)
# CONTAINER_IMAGE_PATH: Nano recipe container (see RL Framework Compatibility table)
# CONTAINER_IMAGE_PATH: NeMo RL release container
# SLURM_ACCOUNT: Slurm account
# SLURM_PARTITION: Slurm partition
WANDB_API_KEY={your W&B API key} \
EXP_NAME=nemo_gym_grpo/nemotron_nano_v2_9b/2nodes/workplace_assistant_001 \
NUM_ACTOR_NODES=2 \
REPO_LOCATION=$PWD \
CONTAINER_IMAGE_PATH=nvcr.io/nvidia/nemo-rl:v0.4.0.nemotron_3_nano \
CONTAINER_IMAGE_PATH=nvcr.io/nvidia/nemo-rl:v0.7.0 \
SLURM_ACCOUNT={your Slurm account} \
SLURM_PARTITION={your Slurm partition} \
examples/nemo_gym/launch_nemo_gym_multinode_training.sh \
Expand All @@ -77,7 +77,7 @@ SLURM_PARTITION={your Slurm partition} \
If you are using enroot following the steps in the [Setup](/tutorials/training-tutorials/nemo-rl-grpo/setup) doc and downloaded the container locally, use the local container filepath instead:

```bash
CONTAINER_IMAGE_PATH=$PWD/../nvcr.io/nvidia/nemo-rl:v0.4.0.nemotron_3_nano \
CONTAINER_IMAGE_PATH=$PWD/../nvcr.io/nvidia/nemo-rl:v0.7.0 \
```

</Tip>
Expand Down
8 changes: 4 additions & 4 deletions fern/versions/latest/pages/training-tutorials/nemo-rl-grpo/setup.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -112,13 +112,13 @@ CONTAINER_IMAGE_PATH=./$CONTAINER_IMAGE_PATH
</Tip>

<Note>
This tutorial trains [Nemotron Nano 9B v2](https://huggingface.co/nvidia/NVIDIA-Nemotron-Nano-9B-v2), not Nemotron 3 Ultra. Use the Nano NGC container from the [RL Framework Compatibility](/reference/rl-framework-compatibility) table (v0.1.1 / Nemotron 3 Nano row). For Nemotron 3 Ultra training with NeMo Gym v0.3.0, build the `ultra-v3` Dockerfile instead.
This tutorial trains [Nemotron Nano 9B v2](https://huggingface.co/nvidia/NVIDIA-Nemotron-Nano-9B-v2), not Nemotron 3 Ultra. The container and NeMo RL checkout below use the same release so their Python and uv versions remain compatible. For Nemotron 3 Ultra training with NeMo Gym v0.3.0, build the `ultra-v3` Dockerfile instead.
</Note>

```bash
# Nano recipe container — see RL Framework Compatibility table
# NeMo RL release container
# https://catalog.ngc.nvidia.com/orgs/nvidia/containers/nemo-rl
CONTAINER_IMAGE_PATH=nvcr.io/nvidia/nemo-rl:v0.4.0.nemotron_3_nano
CONTAINER_IMAGE_PATH=nvcr.io/nvidia/nemo-rl:v0.7.0

NUM_ACTOR_NODES=1
ACCOUNT=<ACCOUNT_NAME>
Expand Down Expand Up @@ -153,7 +153,7 @@ For the first setup on your local filesystem:

```bash
# Clone NeMo RL repository
git clone https://github.com/NVIDIA-NeMo/RL
git clone --branch v0.7.0 https://github.com/NVIDIA-NeMo/RL
cd RL

# Initialize all submodules (Gym, Megatron, AutoModel, etc.)
Expand Down
Loading