Skip to content

Commit 0d76eb2

Browse files
Use conda-native nanoGPT baseline instructions
1 parent 6842647 commit 0d76eb2

1 file changed

Lines changed: 76 additions & 22 deletions

File tree

baseline/nanogpt_one_head/README.md

Lines changed: 76 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -217,33 +217,85 @@ Test measurements are monitoring-only. Validation loss selects
217217
`checkpoint_best.pt`; test loss, test accuracy, test perplexity, and BLEU never
218218
change optimizer updates, schedules, stopping, or checkpoint selection.
219219

220-
## MacBook MPS workflow
220+
## Conda / local workflow
221+
222+
Use the currently activated conda environment. Do **not** create a project venv
223+
and do not run setup wrapper scripts.
221224

222225
From the repository root:
223226

224227
```bash
225228
cd baseline/nanogpt_one_head
226-
bash scripts/setup_mac.sh
227-
bash scripts/prepare_data.sh
228-
bash scripts/smoke_test.sh
229229

230+
# Install this package and any missing dependencies into the active conda env.
231+
python -m pip install -e .
232+
233+
# Keep all corpus caches and experiment outputs outside the git checkout.
230234
export RG_NANOGPT_ONE_HEAD_ROOT="$HOME/rg-nanogpt-one-head"
231-
caffeinate -dimsu bash scripts/run_all_baselines.sh \
232-
2>&1 | tee "$RG_NANOGPT_ONE_HEAD_ROOT/run_all.log"
235+
236+
# Allow unsupported individual MPS operations to fall back to CPU when needed.
237+
export PYTORCH_ENABLE_MPS_FALLBACK=1
233238
```
234239

235-
The setup script reports whether MPS is built and available. When available,
236-
`--device auto` selects it. Unsupported individual MPS operations may use
237-
PyTorch CPU fallback because `PYTORCH_ENABLE_MPS_FALLBACK=1` is set by the
238-
scripts. WeightWatcher always runs on CPU copies of the six matrices.
240+
Verify the active Python and accelerator before a long run:
239241

240-
The first data-preparation run requires internet access. Later runs reuse only a
241-
fully verified corpus under:
242+
```bash
243+
which python
244+
python -c "import torch; print(torch.__version__); print('MPS built:', torch.backends.mps.is_built()); print('MPS available:', torch.backends.mps.is_available())"
245+
```
242246

243-
```text
244-
$RG_NANOGPT_ONE_HEAD_ROOT/data
247+
### Prepare the pinned FineWeb-Edu corpus
248+
249+
The first preparation requires internet access. Later runs reuse only the fully
250+
verified cache under `$RG_NANOGPT_ONE_HEAD_ROOT/data`.
251+
252+
```bash
253+
rg-onehead-prepare --config configs/reference.yaml
254+
```
255+
256+
To deliberately replace an incompatible or stale cache:
257+
258+
```bash
259+
rg-onehead-prepare --config configs/reference.yaml --force
260+
```
261+
262+
### Run the three reference baselines directly
263+
264+
Each command runs or resumes all three canonical seeds.
265+
266+
```bash
267+
python -m rg_nanogpt_one_head.training \
268+
--config configs/reference.yaml \
269+
--optimizer sgd_momentum \
270+
--device auto
271+
272+
python -m rg_nanogpt_one_head.training \
273+
--config configs/reference.yaml \
274+
--optimizer adamw \
275+
--device auto
276+
277+
python -m rg_nanogpt_one_head.training \
278+
--config configs/reference.yaml \
279+
--optimizer muon \
280+
--device auto
245281
```
246282

283+
On an Apple Silicon Mac, `--device auto` selects MPS when available.
284+
WeightWatcher measurements are performed on CPU copies of the six matrices.
285+
286+
### Build the comparison output notebook
287+
288+
After all nine runs are complete:
289+
290+
```bash
291+
cd notebooks
292+
papermill 04_compare_baselines.ipynb 04_compare_baselines.out.ipynb
293+
```
294+
295+
The notebook requires all three optimizers × all three seeds and produces
296+
optimizer overlays plus final and validation-selected 95% confidence-interval
297+
tables.
298+
247299
## Notebook order
248300

249301
```text
@@ -253,14 +305,14 @@ notebooks/03_muon_baseline.ipynb
253305
notebooks/04_compare_baselines.ipynb
254306
```
255307

256-
The first three notebooks run or resume their optimizer's three seeds. The
257-
comparison notebook requires all nine completed runs and produces optimizer
258-
overlays plus final and validation-selected 95% confidence-interval tables.
308+
The first three notebooks are interactive entry points for the same training
309+
runtime. The direct Python commands above are the simplest path for unattended
310+
local runs.
259311

260-
Launch Jupyter with:
312+
To launch Jupyter from the active conda environment:
261313

262314
```bash
263-
.venv-one-head/bin/jupyter lab notebooks
315+
jupyter lab notebooks
264316
```
265317

266318
## Smaller development runs
@@ -271,7 +323,7 @@ Temporary pilots belong in a separate YAML file and a separate output root.
271323
Run one optimizer:
272324

273325
```bash
274-
.venv-one-head/bin/python -m rg_nanogpt_one_head.training \
326+
python -m rg_nanogpt_one_head.training \
275327
--config configs/reference.yaml \
276328
--optimizer adamw \
277329
--device auto
@@ -280,7 +332,7 @@ Run one optimizer:
280332
Run one seed:
281333

282334
```bash
283-
.venv-one-head/bin/python -m rg_nanogpt_one_head.training \
335+
python -m rg_nanogpt_one_head.training \
284336
--config configs/reference.yaml \
285337
--optimizer muon \
286338
--seeds 1337 \
@@ -289,8 +341,10 @@ Run one seed:
289341

290342
## Validation
291343

344+
No shell wrapper is required. From `baseline/nanogpt_one_head` run:
345+
292346
```bash
293-
bash scripts/smoke_test.sh
347+
python -m pytest -q tests
294348
```
295349

296350
The test suite checks the architecture, 80M-token corpus contract, all optimizer

0 commit comments

Comments
 (0)