😀 3 concurrents stream prompts running on a 3060 12gb !!! #3560
celsowm
started this conversation in
Show and tell
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
on vllm, 2 or more I always got corrupted tokens, very sad:
bug_vllm.mp4
But I got 3 fast and fine on SGLang ! Thanks all team:
3_concurrent_sglang.mp4
Beta Was this translation helpful? Give feedback.
All reactions