Skip to content

Commit 21f3af8

Browse files
docs: add release-note fragment for PR llm-d#1978 (llm-d#1985)
Signed-off-by: llm-d-router-release-notes[bot] <287676111+llm-d-router-release-notes[bot]@users.noreply.github.com> Co-authored-by: llm-d-router-release-notes[bot] <287676111+llm-d-router-release-notes[bot]@users.noreply.github.com>
1 parent 936f064 commit 21f3af8

1 file changed

Lines changed: 7 additions & 0 deletions

File tree

release-notes.d/unreleased/1978.md

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
---
2+
pr: 1978
3+
url: https://github.com/llm-d/llm-d-router/pull/1978
4+
author: ezrasilvera
5+
date: 2026-07-13
6+
---
7+
burst-prefix-cache-producer: add a `balanceBy` parameter (`requests` (default) or `tokens`). With `tokens`, placement is balanced by prefix-block (input token) load instead of request count, charging each unit's prefix once per replica and discounting blocks it already holds, to avoid KV-saturating a single replica on high prompt-length-variance workloads. Default behavior is unchanged.

0 commit comments

Comments
 (0)