Does sglang support --load_format sharded_state #3197
Unanswered
wedu-nvidia
asked this question in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
I am currently deploying DeepSeek R1 bf16 with 4 nodes, but the model's loading time is extremely slow, taking approximately 1.5 hours.
Does SGLang support the --load_format sharded_state option, similar to the VLLM framework?
Beta Was this translation helpful? Give feedback.
All reactions