feat: KV cache compression pipeline (Attention Matching + KVTC)#418

Closed

buftar wants to merge 133 commits intojundot:mainfrom

buftar:feat/kv-cache-compression

Commits on Mar 27, 2026

research: KV cache compression spike (AM + kvtc) on MLX

Tony Sina
and
claude
committed
docs: initialize project
Tony Sina
committed
chore: add project config
Tony Sina
committed
docs: complete project research for KV cache compression pipeline

Tony Sina
and
claude
committed
docs: define v1 requirements
Tony Sina
committed
docs: create roadmap (8 phases)
Tony Sina
committed
docs(01): capture phase context
Tony Sina
committed
docs(state): record phase 1 context session
Tony Sina
committed
docs(01): research linalg foundation phase
Tony Sina
committed
docs(phase-1): add validation strategy
Tony Sina
committed
docs(01-linalg-foundation): create phase plan
Tony Sina
committed
test(01-01): add failing test scaffold for linalg_utils
Tony Sina
committed
feat(01-01): implement omlx/compression package with linalg_utils
Tony Sina
committed
chore(01-01): declare scipy>=1.7.0 in project dependencies
Tony Sina
committed
docs(01-01): complete linalg-foundation plan 01
Tony Sina
committed
docs(phase-01): complete phase execution
Tony Sina
committed
docs(02): capture phase context
Tony Sina
committed
docs(state): record phase 2 context session
Tony Sina
committed
docs(02): research am-compaction phase
Tony Sina
committed
docs(phase-2): add validation strategy
Tony Sina
committed
docs(02-am-compaction): create phase 2 plan (3 plans, 3 waves)
Tony Sina
committed
test(02-02): add failing tests and am.py skeleton for AM compaction
Tony Sina
committed
feat(02-02): implement AMCompactor with HighestAttnKeys and uniform fallback
Tony Sina
committed
docs(02-01): complete AM test scaffold plan — Wave 0 RED state

Tony Sina
and
claude
committed
docs(02-02): complete am-compaction plan 02 summary and state update
Tony Sina
committed
test(02-03): add failing tests for _compute_head_budgets method

Tony Sina
and
claude
committed
feat(02-03): implement _compute_head_budgets and entropy-proportional budgets

Tony Sina
and
claude
committed
feat(02-03): update generate_reference_queries docstring per plan spec

Tony Sina
and
claude
committed
docs(02-03): complete non-uniform head budgets plan

Tony Sina
and
claude
committed
fix(02-am): cast queries and keys to float32 before attention score computation

Tony Sina
and
claude
committed
docs(phase-02): complete phase execution
Tony Sina
committed
docs(03): capture phase context
Tony Sina
committed
docs(state): record phase 3 context session
Tony Sina
committed
docs(03): research kvtc compression phase
Tony Sina
committed
docs(phase-3): add validation strategy
Tony Sina
committed
docs(03-kvtc): create phase 3 plan (3 plans, 3 waves)
Tony Sina
committed
chore(03-01): add zstandard dependency and register pytest.mark.slow
Tony Sina
committed
test(03-01): create kvtc.py stub and test scaffold (RED state)
Tony Sina
committed
docs(03-01): complete KVTC test scaffold plan
Tony Sina
committed
feat(03-02): implement private compression primitives
Tony Sina
committed
fix(03-02): remove forbidden linalg pattern from kvtc.py comments
Tony Sina
committed
docs(03-02): complete KVTC compression primitives plan
Tony Sina
committed
feat(03-03): implement compress() -- PCA project + DP quantize + zstd + blob packing
Tony Sina
committed
feat(03-03): implement decompress() + vectorized Lloyd + fast dequant path
Tony Sina
committed
docs(03-03): complete KVTC compress/decompress plan
Tony Sina
committed
docs(phase-03): complete phase execution
Tony Sina
committed
docs(phase-03): add/update validation strategy
Tony Sina
committed
docs(04): research phase 4 PCA calibration CLI
Tony Sina
committed
docs(phase-4): add validation strategy
Tony Sina
committed
docs(04-pca-calibration-cli): create phase plan (3 plans, 3 waves)
Tony Sina
committed
test(04-01): add failing test scaffold for PCA calibration pipeline
Tony Sina
committed
feat(04-01): add calibrator.py stub and calibrate-kv CLI wiring
Tony Sina
committed
docs(04-01): complete PCA calibration CLI test scaffold plan
Tony Sina
committed
feat(04-02): implement 6 calibrator primitives
Tony Sina
committed
test(04-02): replace stubs with real assertions for CAL-02/CAL-03/CAL-04
Tony Sina
committed
docs(04-02): complete calibrator primitives plan
Tony Sina
committed
feat(04-03): implement run_calibration() full KV calibration pipeline
Tony Sina
committed
feat(04-03): update TestCLIDispatch with mock-based dispatch assertions
Tony Sina
committed
docs(04-03): complete run_calibration pipeline plan
Tony Sina
committed
docs(04-03): mark plan complete after human-verify approval

Tony Sina
and
claude
committed
test(04): complete UAT - 4 passed, 0 issues
Tony Sina
committed
docs(05): capture phase context
Tony Sina
committed
docs(state): record phase 5 context session
Tony Sina
committed
docs(05): research pipeline assembly phase
Tony Sina
committed
docs(05): add validation strategy
Tony Sina
committed
docs(05): create phase 5 pipeline assembly plans

Tony Sina
and
claude
committed
feat(05-01): add KVCachePipeline stub and PipelineBlob dataclass
Tony Sina
committed
test(05-01): add test_pipeline.py scaffold — RED state Wave 0
Tony Sina
committed
docs(05-01): complete pipeline scaffold plan — Wave 0 RED state
Tony Sina
committed
feat(05-02): implement KVCachePipeline compact/compress/decompress/_strip_rope
Tony Sina
committed
docs(05-02): complete pipeline implementation plan
Tony Sina
committed
feat(05-03): implement TestSlowQwen.test_qwen_round_trip
Tony Sina
committed
docs(05-03): complete pipeline end-to-end slow test plan

Tony Sina
and
claude
committed
docs(phase-05): complete phase execution
Tony Sina
committed
docs(06): capture phase context
Tony Sina
committed
docs(state): record phase 6 context session
Tony Sina
committed
docs(06): research phase cache integration domain
Tony Sina
committed
docs(phase-6): add validation strategy
Tony Sina
committed
docs(06-cache-integration): create phase 6 plan (5 plans, 3 waves)
Tony Sina
committed
fix(06): revise plans based on checker feedback
Tony Sina
committed
fix(06): revise plans based on checker feedback
Tony Sina
committed
test(06-01): add Wave 0 RED test scaffold for Phase 6 cache integration
Tony Sina
committed
feat(06-02): implement CompressionConfig dataclass with thread-safe set_enabled
Tony Sina
committed
chore(06-01): add compressed_cache_manager.py stub for import resolution
Tony Sina
committed
feat(06-02): implement CompressedPagedSSDCacheManager and CompressedTieredCacheManager

Tony Sina
and
claude
committed
docs(06-02): complete cache integration plan 02 - compressed cache managers

Tony Sina
and
claude
committed
feat(06-03): add compression_config to CacheFactory.create_full_cache_stack()
Tony Sina
committed
feat(06-03): add --compression-bundle CLI flags and wire SchedulerConfig.compression_config
Tony Sina
committed
docs(06-03): complete cache factory and CLI compression wiring plan
Tony Sina
committed
feat(06-04): add POST /admin/api/compression/config endpoint
Tony Sina
committed
docs(06-04): complete admin compression config endpoint plan
Tony Sina
committed
feat(06-05): implement TestSlowQwen.test_qwen_round_trip with real Qwen 2.5 7B
Tony Sina
committed
docs(06-05): complete cache-integration final verification plan
Tony Sina
committed
docs(06-05): complete cache-integration final verification plan

Tony Sina
and
claude
committed
docs(phase-06): complete phase execution
Tony Sina
committed
docs(phase-7): add validation strategy
Tony Sina
committed
docs(07-benchmark-suite): create phase 7 plan — 5 plans in 4 waves

Tony Sina
and
claude
committed
feat(07-01): create benchmark and evaluators stubs

Tony Sina
and
claude
committed
test(07-01): add benchmark test scaffold (RED state)

Tony Sina
and
claude
committed
docs(07-01): complete benchmark suite RED-state scaffold plan
Tony Sina
committed
feat(07-02): implement SWA detection and cosine similarity

Tony Sina
and
claude
committed
feat(07-02): implement BenchmarkRunner and seeding

Tony Sina
and
claude
committed
feat(07-02): add benchmark-compression CLI subparser

Tony Sina
and
claude
committed
test(07-02): update tests for BenchmarkRunner and evaluators

Tony Sina
and
claude
committed
docs(07-02): complete benchmark suite fast-path implementation plan

Tony Sina
and
claude
committed
docs(09-10): add gap closure phases for OBS-04 and test fixes

Tony Sina
and
claude
committed
docs: capture todo - Add compression admin card for KVTC functionality
Tony Sina
committed
docs(phase-10): complete test suite fixes and Nyquist compliance

Tony Sina
and
claude
committed
docs(phase-10): update plan completion marker after GSD phase complete
Tony Sina
committed
test(09): complete UAT - 8 passed, 0 issues
Tony Sina
committed
docs(phase-09): add Nyquist validation mapping for OBS-01/02/03

Tony Sina
and
claude
committed
docs(phase-09): update STATE.md with validation audit

Tony Sina
and
claude
committed
docs(milestone): update v1.0 audit with corrected Phase 9 status

Tony Sina
and
claude
committed
docs(phase-11): add tech debt cleanup phase for v1.0 audit gaps

Tony Sina
and
claude
committed
docs(state): update session continuity after Phase 11 tech debt cleanup

Tony Sina
and
claude
committed
fix(11-01): remove xfail from CAL-05 timing tests (run_calibration is implemented)
Tony Sina
committed
feat(11-01): add direct AM-02/AM-08 behavioral tests without diagnostics dependency
Tony Sina
committed
wip: phase-11-tech-debt-cleanup paused at task 1/7
Tony Sina
committed
chore(11-01): add Nyquist compliance frontmatter to Phase 10 VALIDATION.md
Tony Sina
committed
feat(11-02): add compression settings card to admin dashboard

Tony Sina
and
claude
committed
feat(11-02): add compression stats card to admin dashboard

Tony Sina
and
claude
committed
docs(11-02): complete Admin UI Dashboard plan - OBS-05 satisfied

Tony Sina
and
claude
committed
docs(11-01): complete Wave 0 test suite cleanup plan - CAL-05 xfail removed and AM behavioral tests added

Tony Sina
and
claude
committed
docs(11-01): complete Wave 0 test suite cleanup plan
Tony Sina
committed
docs(11-03): create gap closure plan for OBS-05 am_ratio/n_components fix
Tony Sina
committed
feat(11-03): add am_ratio and n_components to compression status payload
Tony Sina
committed
docs(11-03): complete OBS-05 gap closure plan - am_ratio and n_components fix

Tony Sina
and
claude
committed
feat(11): improve compression UI — fix am_ratio display, move stats to runtime cache table, add paper links
Tony Sina
committed
docs(phase-11): complete phase execution — all plans verified, human approval received
Tony Sina
committed
chore: remove .planning/ from repo and add to .gitignore

Tony Sina
and
claude
committed
chore: remove CLAUDE.md and add to .gitignore

Tony Sina
and
claude
committed
docs: consolidate compression docs into single KV_Cache_Compression.md

Tony Sina
and
claude
committed
fix: remove duplicate dependency-groups in pyproject.toml and fix linalg comment

Tony Sina
and
claude
committed