Commit 100892f
committed
fix(scripts): single-device sharding for compute_norm_stats torch path
Salvage of #438 by @tlpss — rebased to main.
Size-1 batches cannot shard across multi-GPU meshes; pin SingleDeviceSharding
for norm-stats collection convenience.1 parent 15a9616 commit 100892f
1 file changed
Lines changed: 4 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
14 | 14 | | |
15 | 15 | | |
16 | 16 | | |
| 17 | + | |
17 | 18 | | |
18 | 19 | | |
19 | 20 | | |
| |||
47 | 48 | | |
48 | 49 | | |
49 | 50 | | |
| 51 | + | |
| 52 | + | |
50 | 53 | | |
51 | 54 | | |
52 | 55 | | |
| 56 | + | |
53 | 57 | | |
54 | 58 | | |
55 | 59 | | |
| |||
0 commit comments