System Info
- transformers:
main at cbc1651a03, editable install
- torch: 2.11.0+cu128
- accelerate 1.14.0, datasets 5.0.1
- GPU: one node, 8 x NVIDIA B200 (sm_100), driver 580.126.20
flash-linear-attention and causal-conv1d: not installed in the run below
The rest of the environment is in #48692.
Who can help?
No response
Information
Tasks
Reproduction
I ran run_clm.py (examples/pytorch/language-modeling/run_clm.py, full fine-tuning of Qwen/Qwen3.5-9B-Base on wikitext-103 under FSDP2, 8 processes) on 8x B200, following the example's README, and got 1.22 s per training step. A profile showed the 24 gated DeltaNet layers on the fp32 torch chunk loop in modeling_qwen3_5.py, forward and backward, for about half the step, because flash-linear-attention was not installed. With the package installed and no source change, the same step took 0.70 s. The fused norms, causal_conv1d and the Trainer trims that take it to 0.49 s are in the PR.
The full command is under Reproduce in #48692.
I opened #48692 to record my experiment: setup, measurements, traces and some potential fixes. A different fix may also well suit the codebase.
Expected behavior
A warning_once at import does say the fallback is "much slower", but it went by with the rest of the startup output on 8 ranks, and the package is in no setup.py extra and not in the example's requirements.txt, so I only found the gap by profiling. The Qwen3.5 model doc page does say both packages are needed for the fast kernels; I had not read it before running the example. The measured gap and the pip command in that warning would have saved me the profile.
System Info
mainatcbc1651a03, editable installflash-linear-attentionandcausal-conv1d: not installed in the run belowThe rest of the environment is in #48692.
Who can help?
No response
Information
Tasks
examplesfolder (such as GLUE/SQuAD, ...)Reproduction
I ran run_clm.py (
examples/pytorch/language-modeling/run_clm.py, full fine-tuning of Qwen/Qwen3.5-9B-Base on wikitext-103 under FSDP2, 8 processes) on 8x B200, following the example's README, and got 1.22 s per training step. A profile showed the 24 gated DeltaNet layers on the fp32 torch chunk loop inmodeling_qwen3_5.py, forward and backward, for about half the step, becauseflash-linear-attentionwas not installed. With the package installed and no source change, the same step took 0.70 s. The fused norms,causal_conv1dand the Trainer trims that take it to 0.49 s are in the PR.The full command is under Reproduce in #48692.
I opened #48692 to record my experiment: setup, measurements, traces and some potential fixes. A different fix may also well suit the codebase.
Expected behavior
A
warning_onceat import does say the fallback is "much slower", but it went by with the rest of the startup output on 8 ranks, and the package is in nosetup.pyextra and not in the example'srequirements.txt, so I only found the gap by profiling. The Qwen3.5 model doc page does say both packages are needed for the fast kernels; I had not read it before running the example. The measured gap and the pip command in that warning would have saved me the profile.