Skip to content

Latest commit

 

History

History
35 lines (22 loc) · 2.05 KB

File metadata and controls

35 lines (22 loc) · 2.05 KB

4-bit KV cache (experimental)

⚠️ Experimental, llama.cpp backend only. A memory tool, not a speed tool: decode is slightly slower than the default FP16 KV cache.

BONSAI_KV4=1 stores the KV cache in Q4_0 (4-bit) instead of FP16, cutting KV memory roughly 3.5x: from 64 KiB per token to about 18 KiB per token on the 27B, so a 100K-token context needs about 1.8 GiB instead of 6.3 GiB. The 27B's hybrid attention already keeps the cache small, so reach for this only at very long contexts on tight machines.

BONSAI_KV4=1 ./scripts/start_llama_server.sh

Under the hood this passes --cache-type-k q4_0 --cache-type-v q4_0 (quantized KV requires flash attention, which the scripts already enable).

Better quality: the mean-centering bias

4-bit quantization of the K cache loses a little accuracy on channels whose activations have a nonzero mean. A small model-specific calibration bias fixes most of that at zero decode-time cost (one subtract when the cache is written). Build it once:

./scripts/make_kv_bias.sh
BONSAI_KV4=1 ./scripts/start_llama_server.sh

The script runs llama-kv-mean-center (included in the prebuilt binaries) over a calibration text and writes <Model>-kv-bias.gguf next to the model weights. The server picks the bias up automatically whenever BONSAI_KV4=1 is set; without a bias, the 4-bit cache still runs, just with slightly lower quality.

Notes:

  • Calibration does not need much data. The script ships a tiny built-in synthetic corpus; for best results pass your own text file as the first argument, representative of your workload:

    ./scripts/make_kv_bias.sh my_corpus.txt
  • The bias is model-specific. Re-run the script after switching BONSAI_FAMILY / BONSAI_MODEL.

  • Calibration and inference must agree on the K-rotation state; the script and server handle the matching flags automatically, and the loader refuses a mismatched bias by design.

Full background and the manual command flow: PrismML-Eng/llama.cpp#54.