Any plan to integrate it to TRT-LLM or VLLM

[lishicheng1996]

Hi, may I ask is there any plan to integrate FlatQuant to TRT-LLM or VLLM?

[snowdusky]

Hi,

Thanks for the suggestion and your interest in FlatQuant!

We’ve recently added support for fake-quantized inference with vLLM, which allows you to evaluate quantized models with minimal changes. However, true quantized execution (e.g., int4 kernels in vLLM) is not yet supported — we’re still exploring efficient deployment backends and kernel-level support.

We’ll keep you posted as soon as we make further progress.

Best,