Commit 799d45c
[serve.llm] Prefix-aware scheduler [1/N] Adding Prefix-aware tree data structure (ray-project#52747)
Signed-off-by: Justin Ji <[email protected]>
Signed-off-by: weiran11 <[email protected]>1 parent 2372927 commit 799d45c
File tree
2 files changed
+1232
-0
lines changed- python/ray/llm
- _internal/serve/replica_scheduler/prefix_aware
- tests/serve/cpu/deployments
2 files changed
+1232
-0
lines changed
0 commit comments