Skip to content

Commit cacbc8d

Browse files
author
root
committed
works on a beaker session
1 parent eb70de6 commit cacbc8d

1 file changed

Lines changed: 9 additions & 0 deletions

File tree

docs/Pretraining.md

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -382,7 +382,16 @@ torchrun --nproc_per_node=8 scripts/official/base.py train custom_experiment loc
382382
--trainer.max_duration.epochs=100
383383
```
384384

385+
### Example Single GPU debug Setup
385386

387+
```bash
388+
torchrun scripts/official/base.py train custom_experiment local \
389+
--data_loader.global_batch_size=64 \
390+
--data_loader.num_workers=4 \
391+
--train_module.rank_microbatch_size=16 \
392+
--trainer.callbacks.wandb.enabled=False
393+
# Optionally --dataset.h5py_dir=/your/path/to/data \
394+
```
386395
---
387396

388397

0 commit comments

Comments
 (0)