Skip to content

fix(vllm): correct heterogeneous prefix-cache hashes - #451

Merged
RixinLiu merged 1 commit into
ovg-project:mainfrom
jeff3071:fix-gemma4-prefix-cache
Aug 18, 2026
Merged

fix(vllm): correct heterogeneous prefix-cache hashes#451
RixinLiu merged 1 commit into
ovg-project:mainfrom
jeff3071:fix-gemma4-prefix-cache

Conversation

@jeff3071

@jeff3071 jeff3071 commented Aug 18, 2026

Copy link
Copy Markdown
Contributor

Summary

Fix #446

Gemma 4 uses different physical block sizes across KV-cache groups, while request hashes are computed at a shared finer granularity. kvcached previously lost vLLM’s hash_block_size when replacing the native block pool, causing physical blocks to be cached under incorrect prefix hashes.

Testing

  • 40 passed in targeted prefix-cache and coordinator tests.
  • Tested google/gemma-4-12B-it with vLLM 0.24.0.

@RixinLiu RixinLiu self-assigned this Aug 18, 2026
@RixinLiu

Copy link
Copy Markdown
Collaborator

LGTM! Thanks for finding the root cause of this bug!

@RixinLiu
RixinLiu merged commit a441e02 into ovg-project:main Aug 18, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Prefix caching makes gemma-4 generations diverge from vanilla vLLM (output stays coherent, not garbled)

2 participants