fuse qkv/norm/rottary for z image

[devgdovg]

Motivation

optimize inference speed for Nunchaku Z-Image-Turbo by fuse qkv/norm/rottary in attention module.

Modifications

• modify attention processor, calling fused_qkv_norm_rottary method.
• add a pre forward hook for caching packed freqs_cis tensor.

Checklist

• Code is formatted using Pre-Commit hooks.
• Relevant unit tests are added in the tests directory following the guidance in Contribution Guide.
• Documentation and example scripts in examples are updated if necessary.
• Throughput/latency benchmarks and quality evaluations are included where applicable.
• For reviewers: If you're only helping merge the main branch and haven't contributed code to this PR, please remove yourself as a co-author when merging.
• Please feel free to join our Slack, Discord or WeChat to discuss your PR.

[devgdovg]

No need to add new test cases. Just run existing case for z image for verification.

[vgabbo]

Can I please ask if it influences quality? And what is the time improvement? Thank you so much for your work! @devgdovg

[lmxyy]

Can I please ask if it influences quality? And what is the time improvement? Thank you so much for your work! @devgdovg

No. This is mathematically equivalent to the previous SVDQ inference.

[lmxyy]

passing tests. approved