[Feature][Cleanup]: Unify `vllm.utils.flashinfer` and `vllm.model_executor.layers.quantization.utils.flashinfer_utils`

[robertgshaw2-redhat]

🚀 The feature, motivation and pitch

its confusing to have both

Alternatives

No response

Additional context

No response

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

[rajanyadav0307]

@robertgshaw2-redhat ,
Please assign it to me, I can take this.
Thanks

[Frenzyknight]

Hey so I was looking through this and flashinfer_utils has imports from fused_moe so moving everything to vllm/utils would create circular deps right? should we just move the pure utilities and leave the layer-specific stuff in model_executor or what would be the correct approach in your opinion ?

[rajanyadav0307]

@Frenzyknight , I was working on that. Appreciate your feedback.

[rajanyadav0307]

@robertgshaw2-redhat ,
It would better if you could add some more clarification regarding the task. My understanding is we want to make a common "flashinfer.py" at the path vllm/utils/. There are couple of things I could observe:

1. If we are bringing complete vllm/model_executor/layers/quantization/utils/flashinfer_utils.py to vllm/vllm/utils/ under a new file(say, flashinfer_unified.py) that contains the API from vllm/utils/flashinfer.py and vllm/model_executor/layers/quantization/utils/flashinfer_utils.py; we may introduce circular dependency as vllm/model_executor/layers/quantization/utils/flashinfer_utils import fused_moe stuffs, and also fused_moe imports vllm/model_executor/layers/quantization/utils/flashinfer_utils stuff.

2. Considering 1, I think we can just bring the utilies API from vllm/model_executor/layers/quantization/utils/flashinfer_utils to the new file, and leave the module dependent stuffs as it is.

3. Based on your requirement description, I see you want to have a common file. Maybe, if you could add some elaboration, I could clear my thoughts if not aligned to the request.

Thanks.