Accelerating the inference of Hunyuan-DiT with its skip-branches

[guanjiechen118]

Hi, I'm the first author of the paper Accelerating Vision Diffusion Transformers with Skip Branches. In this work, we proposed Skip-DiT and Skip-Cache. Skip-DiT demonstrates superior feature smoothness and improved training efficiency, while Skip-Cache achieves a 2x acceleration during inference. Notably, Skip-DiT is inspired by Hunyuan-DiT, making it particularly well-suited for integration with Hunyuan-DiT. We successfully accelerated Hunyuan-DiT inference by 1.5x without any quality degradation and achieved up to 2.0x acceleration with minimal overhead.

We have added it to our forked repository, complete with detailed code comments and a consistent coding style for easy adoption. We believe that integrating Skip-Cache into this exceptional model will be highly beneficial. To run the inference with skip-cache with the command line, you only need to add --use-cache and specify the --cache-step

python sample_t2i.py --infer-mode fa --prompt "渔舟唱晚" --use-cache --cache-step 2 # or 3,4,5,6

Below, we present both quantitative and qualitative results of Skip-Cache on Hunyuan-DiT to showcase its performance.

Qualitative Results

Visualizations
Comparison between Skip-Cache and other caching methods on HunyuanDiT:

Accelerating HunyuanDiT with 2x speedup:

[zml-ai]

Awesome, thanks! I’ll check it out soon. :)