Title: Add documentation for custom dependencies in RayService and LMCache+Mooncake on Kubernetes#60035
Title: Add documentation for custom dependencies in RayService and LMCache+Mooncake on Kubernetes#60035Blaze-DSP wants to merge 3 commits intoray-project:masterfrom
Conversation
Blaze-DSP
force-pushed
the
feat/extra-dependency
branch
from
9e86c70 to
92cb1d8
Compare
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request adds valuable documentation for installing custom dependencies in RayService deployments on Kubernetes. The new sections in rayservice.md, rayserve-llm-example.md, and kv-cache-offloading.md are clear and provide useful examples for users looking to manage dependencies with args and runtime_env, including a detailed guide for LMCache and Mooncake. The cross-references in handling-dependencies.md are also a good addition. My main feedback is regarding a broken URL in the curl examples, which should be fixed to ensure users can follow the guides successfully.
| Download a basic example: | ||
|
|
||
| ```sh | ||
| curl -o ray-serve.extra-dependency.yaml https://raw.githubusercontent.com/ray-project/kuberay/master/ray-operator/config/samples/ray-serve.extra-dependency.yaml |
There was a problem hiding this comment.
The URL in this curl command points to a file that doesn't seem to exist in the master branch of the ray-project/kuberay repository. This will cause the example command to fail for users. Please verify the URL and update it to a valid location.
| Download a complete example combining both approaches: | ||
|
|
||
| ```sh | ||
| curl -o ray-serve.extra-dependency.yaml https://raw.githubusercontent.com/ray-project/kuberay/master/ray-operator/config/samples/ray-serve.extra-dependency.yaml |
There was a problem hiding this comment.
The URL in this curl command points to a file that doesn't seem to exist in the master branch of the ray-project/kuberay repository. This will cause the example command to fail for users. Please verify the URL and update it to a valid location.
Signed-off-by: DPatel_7 <dpatel@gocommotion.com>
Blaze-DSP
force-pushed
the
feat/extra-dependency
branch
from
92cb1d8 to
fa2bbf5
Compare
Signed-off-by: DPatel_7 <dpatel@gocommotion.com>
ray-gardener
bot
added
serve
kouroshHakha
left a comment
kouroshHakha
left a comment
There was a problem hiding this comment.
overall looks good. Let's make sure the full yaml service lands on kuberay repo on git and then we can cross-ref it here.
| Download a basic example: | ||
|
|
||
| ```sh | ||
| curl -o ray-serve.extra-dependency.yaml https://raw.githubusercontent.com/ray-project/kuberay/master/ray-operator/config/samples/ray-serve.extra-dependency.yaml |
There was a problem hiding this comment.
is there a pr for this?
There was a problem hiding this comment.
Signed-off-by: DPatel_7 <dpatel@gocommotion.com>
Future-Outlier
left a comment
Future-Outlier
left a comment
There was a problem hiding this comment.
cc @machichima @seanlaii @400Ping @win5923 to take a look
Future-Outlier
left a comment
Future-Outlier
left a comment
There was a problem hiding this comment.
we should build a custom image instead, since this might cause other potential problem like here
ray-project/kuberay#4362 (comment)
cc @rueian
|
This pull request has been automatically marked as stale because it has not had You can always ask for help on our discussion forum or Ray's public slack channel. If you'd like to keep this open, just leave any comment, and the stale label will be removed. |
github-actions
bot
added
the
stale
|
This pull request has been automatically closed because there has been no more activity in the 14 days Please feel free to reopen or open a new pull request if you'd still like this to be addressed. Again, you can always ask for help on our discussion forum or Ray's public slack channel. Thanks again for your contribution! |
3 participants
Description
Add documentation for installing custom dependencies in RayService deployments: