With the release of vLLM-Omni, workloads now include fully disaggregated, any-to-any multimodal architectures (like Diffusion Transformers / DiT). llm-d-router is currently optimized purely for autoregressive LLMs and KV-cache locality. To remain the premier inference router, it must evolve to handle such any-to-any multimodal architecture.
We need to investigate how llm-d-router can support diffusion model, we can start with vllm-omni model server
With the release of vLLM-Omni, workloads now include fully disaggregated, any-to-any multimodal architectures (like Diffusion Transformers / DiT). llm-d-router is currently optimized purely for autoregressive LLMs and KV-cache locality. To remain the premier inference router, it must evolve to handle such any-to-any multimodal architecture.
We need to investigate how llm-d-router can support diffusion model, we can start with vllm-omni model server