Generic KvCache#3188

Open

ivarflakstad wants to merge 12 commits intomainfrom

generic-kvcache-proposal

Commits on Nov 14, 2025

Introduce KvCacheTrait so we can easily switch out KvCache implementations in models
ivarflakstad
committed

Commits on Nov 16, 2025

Impl KvCache for ScatteredKvCache
ivarflakstad
committed

Commits on Nov 17, 2025

Merge branch 'main' into generic-kvcache-proposal
ivarflakstad
authored
Finish wiring up generic KvCache in Qwen and quantized Qwen models
ivarflakstad
committed
Merge branch 'generic-kvcache-proposal' of github.com:huggingface/candle into generic-kvcache-proposal
ivarflakstad
committed

Commits on Nov 20, 2025