Commit ba1a8df

authored

[serve.llm] Prefix-aware scheduler [1/N] Adding Prefix-aware tree data structure (#52747)

Signed-off-by: Justin Ji <[email protected]>

1 parent 0abf03b commit ba1a8dfCopy full SHA for ba1a8df

2 files changed

+1232

-0

lines changed

python/ray/llm
- _internal/serve/replica_scheduler/prefix_aware
  - prefix_tree.py
- tests/serve/cpu/deployments
  - test_prefix_tree.py

+1232

-0

lines changed

Comments

(0)