GPU Efficiency / Benchmarking

[neptunes5thmoon]

GPU utilization is low - need to identify and fix bottlenecks

Benchmarking Options

• nvidia-smi dmon -s u to log GPU utilization over time during training
• torch.profiler to break down data loading vs forward/backward pass
• torch.cuda.Event timing to measure per-iteration GPU compute vs idle
• Tune precache_size and precache_workers in run config

Dacapo-Toolbox

• Use dacapo-toolbox to wrap the gunpowder pipeline as a torch Dataset
• use torch Dataloaders
• compare against gunpowder's built-in PreCache

Grace Hopper

• benchmark on GH200 architecture (unified CPU-GPU memory could eliminate data transfer bottleneck
• may require manual compilation