We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
1 parent eb70de6 commit cacbc8dCopy full SHA for cacbc8d
1 file changed
docs/Pretraining.md
@@ -382,7 +382,16 @@ torchrun --nproc_per_node=8 scripts/official/base.py train custom_experiment loc
382
--trainer.max_duration.epochs=100
383
```
384
385
+### Example Single GPU debug Setup
386
387
+```bash
388
+torchrun scripts/official/base.py train custom_experiment local \
389
+ --data_loader.global_batch_size=64 \
390
+ --data_loader.num_workers=4 \
391
+ --train_module.rank_microbatch_size=16 \
392
+ --trainer.callbacks.wandb.enabled=False
393
+ # Optionally --dataset.h5py_dir=/your/path/to/data \
394
+```
395
---
396
397
0 commit comments