Nunchaku Summer Development Roadmap

[lmxyy]

Hello everyone,

Thank you for supporting and using Nunchaku! Yesterday, we officially released v0.3.0. While a few items from our previous roadmap remain unfinished, we’re committed to completing them in upcoming releases. We've also received valuable feedback on v0.3.0—thank you!—and will be addressing these issues in future versions.

Our mission is to make Nunchaku a general-purpose library for running diffusion models and even LLMs efficiently with minimal memory overhead. As Nunchaku grows in functionality, it’s time to evolve our infrastructure to better support a broader range of models and applications.

In v0.3.0, we introduced extensive CI pipelines for testing and building, and significantly optimized model loading. This summer (June–August), our focus will be on improving flexibility and scalability. Planned milestones include:

🔧 Core Development

• A user-friendly deepcompressor for quantizing custom models and exporting them for use in Nunchaku @synxlin
• Modular 4-bit and FP16 attention kernels to make it easier to build custom low-bit models @lmxyy @sxtyzhangzk
• A Pythonized model forward pipeline to support custom downstream tasks @lmxyy
• Comprehensive API documentation for both Nunchaku and ComfyUI-Nunchaku to help onboard new contributors @lmxyy
• More robust Python wheel distributions and hosting on an internal PyPI server @lmxyy

🎥 Video Model Support

A major focus this season is supporting video diffusion models as promised before, especially WAN 2.1:

• Release 4-bit WAN models
• Integrate low-bit attention mechanisms @lmxyy @sxtyzhangzk
• Incorporate sparse attention kernels such as SVG and SVG2 for faster attention
• Support advanced acceleration techniques like TeaCache
• Integrate with VACE

🧠 New Models & Applications

We plan to support the following models and features in upcoming releases:

• FLUX.1-Kontext-dev (waiting for access)
• IP-Adapter (feat: enable IP-Adapter (XLabs-AI/flux-ip-adapter-v2) support nunchaku#418) @Bluear7878
• EasyControl @KBRASK
• Chroma @Aprilhuu
• Rank-64 SVDQuant models @Aprilhuu

🛠️ Bug Fixes & Improvements

LoRA-related

• More LoRA formats ([Bug] Assertion error "lora_act_out.shape[1] == rank_down" related to LoRAs with missing blocks nunchaku#278, [Bug] LoRAs trained by OneTrainer cannot work #64)
• FLUX.1-Turbo compatibility with FLUX.1-Fill ([Bug] Can't use lora with Nunchaku Flux Fill workflow #46, [Bug] Using 2 loras (one with pix2pix and another flow with character lora and|or turbo) #66)
• Others ([Bug] Loading this LoRA will cause ComfyUI to crash #155)

4-bit Text Encoder

• Support for Turing GPUs
• Proper handling of meta device during model loading

Cache System

• Fix issue that FB cache does not work in v0.3.0 @KBRASK

PuLID

• Support PuLID v0.9.1 @lmxyy
• Add support for Turing GPUs @lmxyy
• Prevent repeated model downloads @KBRASK

Some Future Features in Plan

• 8-bit model support.
• HiDream support.

---

If you're interested in quantization or machine learning systems and would like to contribute to Nunchaku, feel free to reach out at muyangli@mit.edu.

Let’s build something amazing together!

[healthyfat]

Any update on WAN 2.1 support? Thanks.

[arvnoodle]

hidream full and e1.1 will be good!

[VladiCz]

Thank you for your hard work and determination, it's very much appreciated!

I'm looking forward for WAN2.1 support as many others do and I'm happy you are focusing on it, this will be a game changer for many talented people with not so powerful cards.

What a time to be alive!

[pixaroma]

Is it possible to do 4 bit version for fluxmania model also? Is flux based and has pretty realistic result compared with flux. https://civitai.com/models/778691?modelVersionId=1769925

And of course those wan models will be great
Thank for great work

[BobMind758]

qwen-image 会做4位版本吗？

[lmxyy]

qwen-image 会做4位版本吗？

Of top priority.

[healthyfat]

Wen WAN 2.1 ? Is it still a major focus?