SLA and SageSLA (SLA based on SageAttention) Code Update #8
Pinned
jt-zhang
announced in
Announcements
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
We’ve just updated the Triton implementation of SLA. Training is now more stable, faster, and typically achieves better training results.
We’ve released the code for SageSLA, a very fast SLA (Sparse-Linear Attention) forward pass based on SageAttention. It uses some code from SpargeAttn. Please refer to the
SageSLA/directory for how to use SageSLA.Feel free to try it out!
Beta Was this translation helpful? Give feedback.
All reactions