Evolving Sparsity: Leveraging Token Importance Dynamics for Efficient LLM Decoding with Sparse Attention

Dynamic sparse attention that evolves across steps and layers to deliver high-performance, low-latency long-context LLM inference.

Authors

Ruizi Han^{1, 2}, Miao Zhang¹*, Ziyue Qiao²*, Liqiang Nie¹

¹ Harbin Institute of Technology (Shenzhen)
² Great Bay University
* Co-corresponding Authors

Links

• Code Repository: GitHub

---

Table of Contents

• Updates
• Method / Framework
• TODO
• License

---

Updates

• [04/2026] Initial release

---

Method / Framework

Figure 1. Overall framework of EvoSparse.

---

TODO

• Complete the code repository

---

License

This project is released under the Apache License 2.0.