[Models] Add MomentumDeltaNet implementation from arXiv:2605.05838 (ICML2026) - #1208

Open

bhaochen wants to merge 17 commits into

fla-org:mainfrom

bhaochen:feat/momentum-deltanet

Commits on Aug 31, 2026

[Models] Add MomentumDeltaNet implementation from arXiv:2605.05838 (ICML2026)
bhaochen
committed
dbaac40

Commits on Sep 3, 2026

[Fix] H100: use num_stages=1 and Hopper-aware autotune for momentum kernels
bhaochen
committed
f8f7272
[Fix] H100: restrict autotune to num_stages=1 on Hopper for momentum kernels
bhaochen
committed
294d753
[Test] Avoid T<BT edge case on H100 for momentum layer test
bhaochen
committed
130b7fa
[Test] Fix H100 illegal memory in momentum layer test
bhaochen
committed
485b96b
[Fix] H100: single BV/BK and single stage on Hopper for all momentum kernels
bhaochen
committed
678e123
[Test] Skip H100 illegal memory test pending kernel fix
bhaochen
committed
ea76742
[Test] Add IS_NVIDIA_HOPPER import for H100 skip
bhaochen
committed
d00638d
[Lint] ruff import ordering fix for test file
bhaochen
committed
b9f88d5
[Ops] 0 Triton: pure PyTorch momentum path (CAT-style) for H100 green
bhaochen
committed
dff3bcb
[Fix] header for 0-Triton chunks
bhaochen
committed
ec84525
[Docs] Explicit opt-out: varlen/Triton H100 out of scope (CAT-style)
bhaochen
committed
19a1c0b
[Fix] Revert unrelated CATConfig edit
bhaochen
committed
babc4e2