-
Notifications
You must be signed in to change notification settings - Fork 326
Description
Describe the feature
We have a small request to allow custom labels for Ray workerGroup. Our current situation is that we have a default istio side car injection to all pods, only can be disabled by specifying labels: "sidecar.istio.io/inject: "false". Unfortunately currently, there is no way to do that for the workerGroup: https://github.com/vllm-project/production-stack/blob/main/helm/templates/ray-cluster.yaml#L258
Why do you need this feature?
vllm engine initialization inside ray head got stuck because the istio-proxy container inside ray worker intercepts all traffic. As a result, we can't start vllm engine with ray pipeline parallel.
Additional context
Ray head custom label are passed via engineLabels: https://github.com/vllm-project/production-stack/blob/main/helm/templates/ray-cluster.yaml#L24 -> that should apply to workerGroup too in my opinion