Why is Expert-Choice routing better suited for DDPM training, while Token-Choice routing works better for Rectified Flow?

[YunjieYu]

Hi, I have a question regarding the choice of routing strategies for different training paradigms.

Why does Expert-Choice routing seem to be more suitable for DDPM-based training, while Token-Choice routing performs better in Rectified Flow training?

Is there any theoretical explanation behind this difference, or is it mainly an empirical observation based on experiments?

Thanks in advance for any insights!

[weilllllls]

Hi, thank you for your interest in our work. Our conclusions are currently based on empirical observations. One possible analytical perspective to explain this difference lies in the trajectory paths of DDPM versus Flow Matching.

Additionally, we mainly adopt Token-Choice to align with mainstream LLM architectures. However, we believe that further exploring the effectiveness of Expert-Choice routing in this context is a valuable direction for future research.