fix: Replace exec liveness probe with tcpSocket for LMCache cache server - #1065
fix: Replace exec liveness probe with tcpSocket for LMCache cache server#1065Asthenia0412 wants to merge 2 commits into
Conversation
The LMCache cache server uses a TCP socket protocol, not an HTTP server. The exec liveness probe referenced a health_probe.py script at /workspace/LMCache/examples/kubernetes/health_probe.py which does not exist in the LMCache v0.5.4 Docker image, causing the liveness check to fail. Switching to a tcpSocket probe is more appropriate since: 1. The cache server communicates via TCP socket protocol 2. No external script is needed - Kubernetes can check the port directly 3. Works with all LMCache versions without requiring a specific script Closes vllm-project#1057 Signed-off-by: Asthenia <asthenia0412@gmail.com>
There was a problem hiding this comment.
Code Review
This pull request replaces the Python-based exec liveness probe in the cache server deployment with a simpler tcpSocket probe, updating the corresponding test cases as well. The reviewer recommends using a named port (e.g., 'caserver-cport') instead of referencing the port directly in both the deployment template and the test assertions to align with Kubernetes best practices.
| tcpSocket: | ||
| port: {{ .Values.cacheserverSpec.containerPort }} |
There was a problem hiding this comment.
Using a named port (e.g., caserver-cport) instead of referencing the port number directly in the tcpSocket probe is a Kubernetes best practice. This decouples the probe configuration from the specific port number, making the chart more maintainable and robust if the port number changes in the future.
tcpSocket:
port: caserver-cport| tcpSocket: | ||
| port: 8000 |
Use the named port 'caserver-cport' instead of the numeric port value in the tcpSocket liveness probe, following Kubernetes best practices. Signed-off-by: Asthenia <asthenia0412@gmail.com>
Description
The LMCache cache server livenessProbe uses an exec probe that runs
/workspace/LMCache/examples/kubernetes/health_probe.py, but this script does not exist in the LMCache v0.5.4 Docker image. The cache server communicates via TCP socket protocol, not HTTP.Changes
exectotcpSocketindeployment-cache-server.yamlcacheserver_test.yamlA
tcpSocketprobe is more appropriate because:Closes #1057