Skip to content

Commit 3b097b6

Browse files
committed
v1.1.1: air-gapped Slurm support, dual-mode runtime, GRPO multi-env hardening
- Air-gapped Slurm support: pre-built nemo-skills/nemo-rl/vLLM/vllm-grpo containers with offline env vars (HF_*_OFFLINE, UV_OFFLINE, TIKTOKEN_*); reproducible Dockerfiles + build instructions - Dual-mode runtime: nvflow/lib/runtime.py centralises venv/python resolution so the same code path runs in airgap and dev modes; checkpoint converter, vLLM serving, training, and SDG scripts all use it - sbatch arg propagation: nvflow/lib/sbatch.py + tests plumb extra_sbatch_args through every Slurm submission - GRPO eval per-environment: outputs split by env (equivalence_llm_judge, finance_sec_search); restored sequence_packing / logprob_chunk_size / make_sequence_length_divisible_by for the equivalence_llm_judge training policy - Eval / SDG / SFT fixes: skip tiktoken download when cache env vars pre-configured, eval base/demo config refinements, removed unused nemotron_nano_9b SFT config - CI: lightweight unit-tests workflow on both GitLab and GitHub. tests/test_sbatch.py reads source files via Path(...).read_text() instead of importing modules that transitively need nemo_skills / typer; both runners invoke `uv run --no-sync pytest tests/ -v --tb=short` so the --no-deps install is authoritative - Documentation: INSTALL.md rewrite, dockerfiles/README + docker_instructions, finance troubleshooting guide, cluster-configuration docs Signed-off-by: Pritam Gundecha <pgundecha@nvidia.com>
1 parent 080ee83 commit 3b097b6

38 files changed

Lines changed: 1776 additions & 410 deletions

.github/workflows/unit-tests.yml

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -41,4 +41,7 @@ jobs:
4141
4242
- name: Test
4343
run: |
44-
.venv/bin/pytest tests/ -v --tb=short
44+
# --no-sync mirrors the GitLab CI invocation and prevents `uv run`
45+
# from implicitly auto-installing heavy core deps that we
46+
# deliberately skipped above.
47+
uv run --no-sync pytest tests/ -v --tb=short

.gitlab-ci.yml

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -59,7 +59,10 @@ test:
5959
- uv venv --python 3.12
6060
- uv pip install -e ".[dev]" --no-deps
6161
- uv pip install pytest pytest-cov pytest-xdist pytest-timeout PyYAML omegaconf rich
62-
- uv run pytest tests/ -v --tb=short
62+
# --no-sync prevents `uv run` from implicitly auto-installing heavy core
63+
# deps (nemo-skills etc) that we deliberately skipped above. Without it,
64+
# tests would pass here for the wrong reason and fail on GitHub CI.
65+
- uv run --no-sync pytest tests/ -v --tb=short
6366
rules:
6467
- if: $CI_PIPELINE_SOURCE == "merge_request_event"
6568
- if: $CI_COMMIT_BRANCH == "main"

INSTALL.md

Lines changed: 187 additions & 127 deletions
Large diffs are not rendered by default.

README.md

Lines changed: 12 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -123,9 +123,18 @@ pytest
123123

124124
## 🔧 Cluster Setup
125125

126-
To run workflows on a Slurm cluster, you need to configure containers and cluster settings.
127-
128-
> **See [INSTALL.md](INSTALL.md)** for complete cluster setup (containers, cluster configuration, verification).
126+
To run workflows on a Slurm cluster you need to: (1) build the four NVFlow
127+
container images from the Dockerfiles in [`dockerfiles/`](dockerfiles/),
128+
(2) convert them to `.sqsh` for Slurm, and (3) write a cluster config
129+
(`cluster_configs/my_cluster.yaml`). The containers are self-sufficient —
130+
all dependencies are pre-installed, so no runtime downloads are needed.
131+
132+
> **See [INSTALL.md](INSTALL.md)** for the complete setup guide
133+
> (build, sanity-check, `.sqsh` conversion, model staging, cluster
134+
> configuration, and verification).
135+
>
136+
> **See [`dockerfiles/docker_instructions.md`](dockerfiles/docker_instructions.md)**
137+
> for the build / multi-arch / sanity-check reference.
129138
130139
Once cluster setup is complete, set the config directory:
131140

cluster_configs/containers.yaml

Lines changed: 29 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -10,33 +10,47 @@
1010
# # Edit my_containers.yaml with your registry paths
1111
# sbatch --account=<acct> scripts/setup_containers.sh --config cluster_configs/my_containers.yaml ./containers
1212
#
13+
# Naming:
14+
# YAML KEYS (nemo-skills, nemo-rl, vllm, vllm-grpo, sglang) match the
15+
# short names the workflow code uses to look up containers -- do NOT rename
16+
# them. Only the values (registry/tag refs) change between deployments.
17+
#
1318
# Format:
14-
# - Simple string: image reference (e.g., your-registry/nemo-skills:latest)
19+
# - Simple string: image reference (e.g., your-registry/nvflow-nemo-skills:0229040)
1520
# - Nested object: Platform-specific tags (amd64/arm64 keys)
21+
#
22+
# Self-sufficient containers (all deps pre-installed, no runtime downloads):
23+
# nemo-rl -> dockerfiles/Dockerfile.nemo-rl (base: nvcr.io/nvidia/nemo-rl:v0.6.0)
24+
# nemo-skills -> dockerfiles/Dockerfile.nemo-skills (base: ubuntu:22.04, NeMo-Skills @ 0229040)
25+
# vllm -> dockerfiles/Dockerfile.vllm (base: vllm/vllm-openai:v0.18.1)
26+
# vllm-grpo -> dockerfiles/Dockerfile.vllm-grpo (base: vllm/vllm-openai:v0.17.1)
27+
# sglang -> pulled as-is from Docker Hub (no custom Dockerfile)
1628

1729
containers:
1830
# ---------------------------------------------------------------------------
19-
# Required: Build from NeMo-Skills Dockerfiles (see INSTALL.md Step 1a)
31+
# Required: Built locally from dockerfiles/, then pushed to your registry
32+
# (see INSTALL.md Step 1 and Step 2).
2033
# ---------------------------------------------------------------------------
21-
# Tested: NeMo-Skills @ 0229040
22-
nemo-skills: <YOUR_REGISTRY>/nemo-skills:<TAG>
34+
# Tested: nvflow-nemo-rl:v0.6.0 (extends nvcr.io/nvidia/nemo-rl:v0.6.0)
35+
nemo-rl: <YOUR_REGISTRY>/nvflow-nemo-rl:v0.6.0
36+
37+
# Tested: nvflow-nemo-skills:0229040 (NeMo-Skills @ commit 0229040)
38+
nemo-skills: <YOUR_REGISTRY>/nvflow-nemo-skills:0229040
39+
40+
# Tested: nvflow-vllm:v0.18.1 (extends vllm/vllm-openai:v0.18.1; pre-cached
41+
# tiktoken + openai_harmony; multi-arch amd64 + arm64)
42+
vllm: <YOUR_REGISTRY>/nvflow-vllm:v0.18.1
43+
44+
# Tested: nvflow-vllm-grpo:v0.17.1 (extends vllm/vllm-openai:v0.17.1; pinned
45+
# to match NeMo-RL v0.6.0 colocated vLLM)
46+
vllm-grpo: <YOUR_REGISTRY>/nvflow-vllm-grpo:v0.17.1
2347

2448
# ---------------------------------------------------------------------------
25-
# Required: Pull pre-built from Docker Hub / NGC (no build needed)
49+
# Required: Pulled as-is from Docker Hub (no custom Dockerfile)
2650
# ---------------------------------------------------------------------------
27-
# Tested: vllm/vllm-openai:v0.18.1 (standalone SDG/eval)
28-
vllm: <YOUR_REGISTRY>/nemo-skills-vllm:<TAG>
29-
# Tested: vllm/vllm-openai:v0.17.1 (standalone GRPO rollouts/judge)
30-
vllm-grpo: vllm/vllm-openai:v0.17.1
3151
# Tested: lmsysorg/sglang:v0.5.10.post1
3252
sglang: lmsysorg/sglang:v0.5.10.post1
3353

34-
# ---------------------------------------------------------------------------
35-
# Required: Pull from NGC (see INSTALL.md Step 1b)
36-
# ---------------------------------------------------------------------------
37-
# Tested: nvcr.io/nvidia/nemo-rl:v0.6.0
38-
nemo-rl: nvcr.io/nvidia/nemo-rl:v0.6.0
39-
4054
# ---------------------------------------------------------------------------
4155
# Optional: Not currently used by NVFlow recipes
4256
# Uncomment and update if needed for your workflows.

cluster_configs/template-slurm.yaml

Lines changed: 42 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -5,12 +5,12 @@
55
#
66
# Then update all <PLACEHOLDER> values with your settings.
77
#
8-
# Container versions tested with this release:
9-
# nemo-skills: NeMo-Skills @ 0229040
10-
# vllm: vllm/vllm-openai v0.18.1 (standalone SDG/eval)
11-
# vllm-grpo: vllm/vllm-openai v0.17.1 (standalone GRPO rollouts/judge)
8+
# Container versions tested with this release (self-sufficient, no runtime downloads):
9+
# nemo-skills: nvflow-nemo-skills (NeMo-Skills @ 0229040)
10+
# vllm: nvflow-vllm (base vllm/vllm-openai v0.18.1, standalone SDG/eval)
11+
# vllm-grpo: nvflow-vllm-grpo (base vllm/vllm-openai v0.17.1, GRPO rollouts/judge)
1212
# sglang: lmsysorg/sglang v0.5.10.post1
13-
# nemo-rl: nvcr.io/nvidia/nemo-rl:v0.6.0 (includes vLLM 0.17.1 colocated)
13+
# nemo-rl: nvflow-nemo-rl (base nvcr.io/nvidia/nemo-rl:v0.6.0, pre-built venvs + Gym)
1414
#
1515
# Reference: https://github.com/NVIDIA/NeMo-Skills
1616

@@ -79,12 +79,12 @@ extra_sandbox_args:
7979
# =============================================================================
8080
# After converting containers to .sqsh format (see INSTALL.md), paste paths here.
8181
containers:
82-
# Required containers
83-
nemo-skills: <PATH_TO_CONTAINERS>/nemo-skills.sqsh # Orchestration client (eval, SDG, data prep)
84-
vllm: <PATH_TO_CONTAINERS>/vllm.sqsh # vLLM v0.18.1 standalone (SDG, eval)
85-
vllm-grpo: <PATH_TO_CONTAINERS>/vllm-grpo.sqsh # vLLM v0.17.1 standalone (GRPO rollouts, judge)
86-
sglang: <PATH_TO_CONTAINERS>/sglang.sqsh # sglang inference server (SDG stages 3-5)
87-
nemo-rl: <PATH_TO_CONTAINERS>/nemo-rl.sqsh # NeMo-RL v0.6.0 for SFT and GRPO training
82+
# Required containers (self-sufficient — all deps pre-installed, no runtime downloads)
83+
nemo-skills: <PATH_TO_CONTAINERS>/nvflow-nemo-skills.sqsh # Orchestration client (eval, SDG, data prep)
84+
vllm: <PATH_TO_CONTAINERS>/nvflow-vllm.sqsh # vLLM v0.18.1 standalone (SDG, eval)
85+
vllm-grpo: <PATH_TO_CONTAINERS>/nvflow-vllm-grpo.sqsh # vLLM v0.17.1 standalone (GRPO rollouts, judge)
86+
sglang: <PATH_TO_CONTAINERS>/sglang.sqsh # sglang inference server (SDG stages 3-5)
87+
nemo-rl: <PATH_TO_CONTAINERS>/nvflow-nemo-rl.sqsh # NeMo-RL v0.6.0 for SFT and GRPO training
8888
# Optional containers (not currently used by NVFlow recipes)
8989
# trtllm: <PATH_TO_CONTAINERS>/trtllm.sqsh
9090
# megatron: <PATH_TO_CONTAINERS>/megatron.sqsh
@@ -98,14 +98,16 @@ containers:
9898
mounts:
9999
- <CLUSTER_PATH_TO_HF_MODELS>:/hf_models # HuggingFace models
100100
- <CLUSTER_PATH_TO_WORKSPACE>:/workspace # Your workspace
101-
# --- GRPO / RL Training (required for collect_rollouts and training stages) ---
102-
# NeMo-RL source mount: overlays the container's built-in /opt/NeMo-RL.
103-
- <PATH_TO_NEMO_RL_CLONE>:/opt/NeMo-RL
104-
# Gym overlay: mount your NeMo-Gym clone inside the NeMo-RL source tree.
105-
# Harmless for SFT/SDG/eval -- only accessed by GRPO stages.
106-
- <PATH_TO_GYM_CLONE>:/opt/NeMo-RL/3rdparty/Gym-workspace/Gym
107101
# Add more mounts as needed:
108102
# - /lustre/data:/data
103+
#
104+
# --- NeMo-RL / NeMo-Gym source overlays (dev mode only) ---
105+
# The nvflow-nemo-rl container includes NeMo-RL source and pre-built Gym
106+
# venvs. Mounting host clones here shadows the container's venvs and
107+
# breaks GRPO stages. Only uncomment for local source iteration with
108+
# NRL_FORCE_REBUILD_VENVS=true enabled below.
109+
# - <PATH_TO_NEMO_RL_CLONE>:/opt/NeMo-RL
110+
# - <PATH_TO_GYM_CLONE>:/opt/NeMo-RL/3rdparty/Gym-workspace/Gym
109111

110112
# =============================================================================
111113
# Timeouts (per partition)
@@ -131,10 +133,30 @@ env_vars:
131133
- TOKENIZERS_PARALLELISM=false # Disable HF tokenizer Rayon threads (prevents vLLM RefCell race)
132134
- VIRTUAL_ENV= # Unset to prevent host venv from interfering with container
133135
- VIRTUAL_ENV_PROMPT= # Unset venv prompt
134-
# Rebuild Ray venvs when NeMo-RL source mount changes (e.g., new branch/commit).
135-
# Without this, workers reuse stale cached venvs and may fail with import errors.
136-
# Safe to leave enabled — only rebuilds when the source tree actually changes.
136+
137+
# --- Offline enforcement (recommended) ---
138+
# Prevents accidental network calls at runtime. Containers are self-sufficient.
139+
# Clear HF_*_OFFLINE temporarily for one-time stages that download external
140+
# data (download_sec_filings, create_seed_data). UV_OFFLINE should stay set.
141+
- HF_HUB_OFFLINE=1
142+
- HF_DATASETS_OFFLINE=1
143+
- TRANSFORMERS_OFFLINE=1
144+
- UV_OFFLINE=true
145+
146+
# Pre-cached tiktoken encodings (baked into vllm/vllm-grpo containers;
147+
# set here for uniform coverage across all container types).
148+
- TIKTOKEN_CACHE_DIR=/opt/tiktoken_cache
149+
- TIKTOKEN_RS_CACHE_DIR=/opt/tiktoken_cache
150+
- TIKTOKEN_ENCODINGS_BASE=/opt/tiktoken_cache
151+
152+
# Ray worker port range — avoids collisions with OS ephemeral ports.
153+
- MIN_WORKER_PORT=7000
154+
- MAX_WORKER_PORT=8000
155+
156+
# Only enable in dev mode when iterating on NeMo-RL/Gym source overlays.
157+
# Forces Ray workers to rebuild venvs from mounted source (requires internet).
137158
# - NRL_FORCE_REBUILD_VENVS=true
159+
138160
# API keys (keep these secret, don't commit to git!)
139161
# - HF_TOKEN=<YOUR_HF_TOKEN>
140162
# - WANDB_API_KEY=<YOUR_WANDB_KEY>

dockerfiles/Dockerfile.nemo-rl

Lines changed: 131 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,131 @@
1+
# =============================================================================
2+
# NVFlow NeMo-RL Container
3+
# =============================================================================
4+
# Extends the NeMo-RL nightly container with NeMo-Skills and the NeMo-Gym
5+
# finance agent for NVFlow workflows. The base image ships with frozen
6+
# environments and pre-built Ray venvs; this Dockerfile adds the NeMo-Skills
7+
# package, replaces the Gym submodule with a feature branch that includes the
8+
# finance-SEC-search resource server and finance agent, pre-builds all Gym
9+
# component venvs, and relocates paths for Slurm/enroot compatibility.
10+
#
11+
# Build:
12+
# docker build -f dockerfiles/Dockerfile.nemo-rl -t nvflow-nemo-rl:latest .
13+
# =============================================================================
14+
15+
ARG BASE_IMAGE=nvcr.io/nvidia/nemo-rl:v0.6.0
16+
FROM ${BASE_IMAGE}
17+
18+
# --- Symlink for NeMo-Skills code that references /opt/NeMo-RL (wrong case) --
19+
RUN ln -sf /opt/nemo-rl /opt/NeMo-RL
20+
21+
# --- Upgrade uv (nemo-gym may require newer features than what the nightly ships)
22+
RUN curl -LsSf https://astral.sh/uv/install.sh | sh
23+
24+
# --- Pre-cache Python interpreter for uv (air-gapped safety net) -------------
25+
# Gym's cli_setup_command runs `uv venv --python <version>`. If
26+
# skip_venv_if_present ever misses, uv still needs a local interpreter.
27+
RUN /root/.local/bin/uv python install 3.12
28+
29+
# --- Install NeMo-Skills into the frozen venv --------------------------------
30+
ARG NEMO_SKILLS_COMMIT=022904023ad7a83a87662a313cf72e7df5891d55
31+
RUN git clone https://github.com/NVIDIA-NeMo/Skills.git /opt/NeMo-Skills && \
32+
cd /opt/NeMo-Skills && git checkout ${NEMO_SKILLS_COMMIT} && \
33+
/root/.local/bin/uv pip install --python /opt/nemo_rl_venv/bin/python .
34+
35+
# --- Replace NeMo-Gym submodule with feature branch -------------------------
36+
# The feature branch includes the finance-SEC-search resource server and
37+
# finance agent that are not yet on main.
38+
ARG NEMO_GYM_BRANCH=ude/finance-sec-search-v2
39+
RUN rm -rf /opt/nemo-rl/3rdparty/Gym-workspace/Gym && \
40+
git clone --branch ${NEMO_GYM_BRANCH} \
41+
https://github.com/NVIDIA-NeMo/Gym.git \
42+
/opt/nemo-rl/3rdparty/Gym-workspace/Gym
43+
44+
# --- Pre-build Gym venv ------------------------------------------------------
45+
WORKDIR /opt/nemo-rl/3rdparty/Gym-workspace/Gym
46+
RUN /root/.local/bin/uv venv .venv --python 3.12 && \
47+
. .venv/bin/activate && \
48+
/root/.local/bin/uv sync --active --extra dev
49+
50+
# Install finance-specific dependencies into Gym venv
51+
# uvicorn>=0.37.0 is required for timeout_worker_healthcheck support;
52+
# uv sync resolves from the parent nemo-rl workspace lock (0.35.0) instead
53+
# of the Gym lock, so we force the correct version here.
54+
RUN . .venv/bin/activate && \
55+
/root/.local/bin/uv pip install aiohttp beautifulsoup4 "tavily==1.1.0" tenacity "uvicorn>=0.37.0"
56+
57+
# --- Symlink component venvs to the main Gym venv ---------------------------
58+
# Each NeMo-Gym component expects its own .venv/; symlinking avoids multi-GB
59+
# duplication and guarantees every component runs with the same packages.
60+
RUN for component in \
61+
resources_servers/equivalence_llm_judge \
62+
resources_servers/finance_sec_search \
63+
responses_api_agents/simple_agent \
64+
responses_api_agents/finance_agent \
65+
responses_api_models/openai_model \
66+
responses_api_models/vllm_model; do \
67+
dir="/opt/nemo-rl/3rdparty/Gym-workspace/Gym/$component"; \
68+
[ -d "$dir" ] && ln -sf /opt/nemo-rl/3rdparty/Gym-workspace/Gym/.venv "$dir/.venv"; \
69+
done
70+
71+
WORKDIR /
72+
73+
# --- Install Gym into the NemoGym Ray venv ------------------------------------
74+
# The pre-built Ray venv from the base image is stale (built from the old Gym
75+
# submodule). Install the new Gym branch editable + all deps so the Ray actor
76+
# can import nemo_gym without missing modules (e.g. gprof2dot, pydot).
77+
RUN /root/.local/bin/uv pip install \
78+
--python /opt/ray_venvs/nemo_rl.environments.nemo_gym.NemoGym/bin/python \
79+
-e /opt/nemo-rl/3rdparty/Gym-workspace/Gym
80+
81+
# --- Align numpy across all Ray venvs to match the main venv ----------------
82+
# NeMo-Skills may upgrade numpy; mismatched versions cause pickle failures
83+
# when Ray serializes data between the main process and worker processes.
84+
RUN MAIN_NP=$(/opt/nemo_rl_venv/bin/python -c "import numpy; print(numpy.__version__)") && \
85+
for venv in /opt/ray_venvs/*/; do \
86+
"$venv/bin/pip" install --no-cache-dir "numpy==$MAIN_NP" 2>/dev/null || true; \
87+
done
88+
89+
# --- Relocate /root/.local/ → /opt/ -----------------------------------------
90+
# enroot/pyxis on Slurm mounts the user's home directory over /root at runtime,
91+
# which shadows everything uv installed there during the Docker build.
92+
# NOTE: Do NOT move /root/.cache/uv — base-image venvs symlink into it.
93+
RUN REAL_PYTHON=$(readlink /opt/nemo_rl_venv/bin/python) && \
94+
mv /root/.local/share/uv/python /opt/uv-python && \
95+
find /opt/uv-python -maxdepth 1 -type l | while read link; do \
96+
target=$(readlink "$link") && \
97+
new_target=$(echo "$target" | sed "s|/root/.local/share/uv/python|/opt/uv-python|") && \
98+
ln -sf "$new_target" "$link"; \
99+
done && \
100+
NEW_PYTHON=$(echo "$REAL_PYTHON" | sed "s|/root/.local/share/uv/python|/opt/uv-python|") && \
101+
ln -sf "$NEW_PYTHON" /opt/nemo_rl_venv/bin/python && \
102+
sed -i "s|/root/.local/share/uv/python|/opt/uv-python|g" /opt/nemo_rl_venv/pyvenv.cfg && \
103+
mv /root/.local/bin /opt/uv-bin
104+
105+
# --- Fix pre-built Ray venvs (same /root/ relocation) -----------------------
106+
RUN for cfg in /opt/ray_venvs/*/pyvenv.cfg; do \
107+
sed -i "s|/root/.local/share/uv/python|/opt/uv-python|g" "$cfg"; \
108+
done && \
109+
find /opt/ray_venvs/ -type l | while read link; do \
110+
target=$(readlink "$link") && \
111+
case "$target" in */root/.local/share/uv/python*) \
112+
new_target=$(echo "$target" | sed "s|/root/.local/share/uv/python|/opt/uv-python|") && \
113+
ln -sf "$new_target" "$link" ;; \
114+
esac; \
115+
done
116+
117+
# --- Fix Gym venv (same /root/ relocation) ----------------------------------
118+
RUN GYM_VENV=/opt/nemo-rl/3rdparty/Gym-workspace/Gym/.venv && \
119+
sed -i "s|/root/.local/share/uv/python|/opt/uv-python|g" "$GYM_VENV/pyvenv.cfg" && \
120+
find "$GYM_VENV" -type l | while read link; do \
121+
target=$(readlink "$link") && \
122+
case "$target" in */root/.local/share/uv/python*) \
123+
new_target=$(echo "$target" | sed "s|/root/.local/share/uv/python|/opt/uv-python|") && \
124+
ln -sf "$new_target" "$link" ;; \
125+
esac; \
126+
done
127+
128+
# --- Runtime environment -----------------------------------------------------
129+
ENV VIRTUAL_ENV=/opt/nemo_rl_venv
130+
ENV PATH=/opt/uv-bin:/opt/nemo_rl_venv/bin:$PATH
131+
ENV UV_PYTHON_INSTALL_DIR=/opt/uv-python

0 commit comments

Comments
 (0)