Skip to content

Commit 2734681

Browse files
authored
Fix file path in tiered prefix guide (llm-d#1785)
Signed-off-by: Cong Liu <conliu@google.com>
1 parent 0808a91 commit 2734681

2 files changed

Lines changed: 1 addition & 4 deletions

File tree

guides/tiered-prefix-cache/modelserver/gpu/vllm/native/fs/gke/kustomization.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
apiVersion: kustomize.config.k8s.io/v1beta1
22
kind: Kustomization
33
resources:
4-
- ..
4+
- ../base
55
components:
66
- ../../../../../../../recipes/modelserver/components/disable-gke-nccl-tuner-patch
77
patches:

guides/tiered-prefix-cache/router/tiered-prefix-cache-cpu.values.yaml

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -18,13 +18,10 @@ router:
1818
- type: no-hit-lru-scorer
1919
- type: approx-prefix-cache-producer
2020
name: gpu-prefix-cache-producer
21-
parameters:
22-
maxPrefixTokensToMatch: 40000 # enough to cover the ~20k input length in the benchmark.
2321
- type: approx-prefix-cache-producer
2422
name: cpu-prefix-cache-producer
2523
parameters:
2624
autoTune: false
27-
maxPrefixTokensToMatch: 40000 # enough to cover the ~20k input length in the benchmark.
2825
lruCapacityPerServer: 82000 # Allocating ~200GB for Qwen-32B: 82,000 blocks * 2.5MB/block (based on 160KB/token * 16 block size).
2926
- type: queue-scorer
3027
- type: kv-cache-utilization-scorer

0 commit comments

Comments
 (0)