[Performance]: VLLM with DP performing worst

[akasshdeep]

Name of failing test

examples/offline_inference/data_parallel.py

Basic information

• Flaky test
• Can reproduce locally
• Caused by external libraries (e.g. bug in transformers)

🧪 Describe the failing test

I have tested DP feature with 4 x A100 card. I observed that vllm with DP 4 and api-server-count = 4 performs poor as compare to 4 x VLLM instances with 1 GPU each .

📝 History of failing test

NA

CC List.

No response

[DarkLight1337]

cc @njhill

[njhill]

@akasshdeep this is a known issue #24461. I have a fix in progress, will open a PR today.