Multi-GPU support for inference?

[bryanyzhu]

Hi can I ask does the current codebase support multi-gpu inference? Need to run 13B model on 40GB GPUs, thanks.

[liujian19911023]

Is this feature okay？