Commit 01c58e6
Han Wang
Revert "perf(dpmodel): contract the DPA4 grid-branch router with matmul"
This reverts commit 7518a41.
Measurement did not support it. An op-level profile attributed the 45.6 ms
reduce to ExpandBackward0, not to this multiply, and re-benchmarking after
the change moved DPA4 eager training by nothing (1.514 -> 1.552 s/step, i.e.
run-to-run noise) while the offending kernel stayed byte-identical at
410.7 ms. The GridBranch product is well under the size that would matter.
Since matmul is autocast-listed where mul/sum are not, keeping it would have
silently moved this contraction into bf16 under the autocast region for no
measured gain. The actual site is the broadcast weight in so3.py, fixed
separately.1 parent 7518a41 commit 01c58e6
1 file changed
Lines changed: 2 additions & 16 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
401 | 401 | | |
402 | 402 | | |
403 | 403 | | |
404 | | - | |
405 | | - | |
406 | | - | |
407 | | - | |
408 | | - | |
409 | | - | |
410 | | - | |
411 | | - | |
412 | | - | |
413 | | - | |
414 | | - | |
415 | | - | |
416 | | - | |
417 | | - | |
418 | | - | |
419 | | - | |
| 404 | + | |
| 405 | + | |
420 | 406 | | |
421 | 407 | | |
422 | 408 | | |
| |||
0 commit comments