[Doc] Tutorial 17: troubleshooting the silent-miss preconditions of kvaware routing - #1062
[Doc] Tutorial 17: troubleshooting the silent-miss preconditions of kvaware routing#1062tyler2cr wants to merge 2 commits into
Conversation
…preconditions KV-aware routing fails silently - requests still succeed while every lookup misses - so the six wiring preconditions discovered across real deployments are documented in checking order: matched lmcache versions (version-locked ZMQ messages), vLLM present in the router (NONE_HASH hash-chain root), PYTHONHASHSEED parity, worker heartbeats (workers default to never sending them while the controller reaps silent workers at ~30s), one cache-owning instance per IP (QueryInstMsg attributes by IP alone), and the hybrid-KV-manager cost of LMCacheConnectorV1 on sliding-window models (silently disabled, up to ~10x KV inflation). Signed-off-by: tyler <tcr@enfuse.io>
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request adds a troubleshooting section to the KV-aware routing tutorial, detailing six common reasons why routing might silently fail. The review feedback correctly points out that the configuration keys mentioned for enabling worker heartbeats do not exist in the Helm chart's values.yaml, which should be corrected to avoid confusing users.
| 4. **Worker heartbeats must be enabled** (`lmcacheConfig.workerHeartbeatTime` | ||
| in this tutorial's values, mapping to | ||
| `LMCACHE_LMCACHE_WORKER_HEARTBEAT_TIME`). lmcache workers default to |
There was a problem hiding this comment.
The parameter lmcacheConfig.workerHeartbeatTime and environment variable LMCACHE_LMCACHE_WORKER_HEARTBEAT_TIME are not defined in the Helm chart's values.yaml. This can be very confusing for users trying to follow the troubleshooting guide. Please update the documentation to use the correct and existing configuration keys for enabling worker heartbeats.
There was a problem hiding this comment.
The keys do exist and work — they are consumed by the chart rather than declared in values.yaml defaults: helm/templates/deployment-vllm-multi.yaml lines 389–391 render LMCACHE_LMCACHE_WORKER_HEARTBEAT_TIME from lmcacheConfig.workerHeartbeatTime behind a hasKey guard, and this tutorial's own tutorials/assets/values-17-kv-aware.yaml (line 28) sets workerHeartbeatTime: "30". The kernel of truth in this comment is discoverability — the key is absent from values.yaml, which is what made it look nonexistent — so the doc now states that explicitly (added in 8ed05a0). Declaring the key in values.yaml defaults would be a reasonable chart follow-up.
… chart key Signed-off-by: tyler <tcr@enfuse.io>
2 participants
KV-aware routing fails silently — when a wiring precondition is violated, requests still succeed and the router just falls back to session/QPS, so every KV lookup misses with nothing above a debug line to explain why. This adds a troubleshooting section to tutorial 17 documenting the six preconditions, each one hit in a real deployment while validating #1045/#1060/#1061:
NONE_HASHroots the chunk-hash chain — a vllm-less router can never match)PYTHONHASHSEEDparity (builtin hash is process-seed-randomized)QueryInstMsgattributes by IP alone)LMCacheConnectorV1on sliding-window models (vLLM silently disables HMA — up to ~10× per-token KV inflation;LMCacheMPConnectoris the HMA-capable path)Docs only — no code changes.
🤖 Generated with Claude Code