Commit 49b9dcf
committed
[optimizer] run linear attention on LiteHitMamba and retire the legacy indexer
Linear-attention instances were simulated by the legacy LruCacheIndexer, whose
model fuses a Full block and its Mamba checkpoint into one cache entry whose
charge doubles as the checkpoint tag. The two could therefore never be evicted
independently - exactly the case that matters, since a Full block stays warm
through prefix reuse while its checkpoint goes cold. It also built one real LRU
per capacity tier, had no facts pipeline, and inferred a checkpoint by comparing
a stored charge against a constant.
InstanceState now holds exactly one analyzer: LiteHit for full attention,
LiteHitMamba for linear attention. The legacy indexer, its factory, its TTL
wrapper and its hit-age histogram are deleted (the histogram had no proto field
and no client or dashboard consumer).
Linear attention with a group TTL is rejected at registration: the Mamba core
carries no time axis yet, and the offline runner already refused that
combination.
Two byte quantities replace the fused wording internally: a Full block always
costs full_charge_bytes and a checkpoint additionally stores mamba_charge_bytes,
instead of adding them into size_full_linear and subtracting them back out at
every use. The register response is unchanged; the fused value is composed once
at the proto boundary.
The block count in that response is now computed explicitly instead of dividing
by an "average bytes per block" that does not exist when two charges differ. It
stays exact for full attention, where it doubles as the projection slot, and is
labelled an estimate for linear attention, where hits are decided on the byte
axis and nothing consumes the estimate.1 parent 873ccde commit 49b9dcf
26 files changed
Lines changed: 517 additions & 2056 deletions
File tree
- kv_cache_manager
- optimizer
- docs
- index
- online
- liteHit
- manager
- online_runtime
- service
- metrics
- test
- protocol/protobuf
Lines changed: 11 additions & 9 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
41 | 41 | | |
42 | 42 | | |
43 | 43 | | |
44 | | - | |
| 44 | + | |
45 | 45 | | |
46 | 46 | | |
47 | 47 | | |
| |||
76 | 76 | | |
77 | 77 | | |
78 | 78 | | |
79 | | - | |
80 | | - | |
81 | | - | |
| 79 | + | |
| 80 | + | |
82 | 81 | | |
83 | 82 | | |
84 | 83 | | |
| |||
95 | 94 | | |
96 | 95 | | |
97 | 96 | | |
98 | | - | |
99 | | - | |
100 | | - | |
101 | | - | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
102 | 102 | | |
103 | 103 | | |
104 | 104 | | |
| |||
154 | 154 | | |
155 | 155 | | |
156 | 156 | | |
157 | | - | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
158 | 160 | | |
159 | 161 | | |
160 | 162 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
16 | 16 | | |
17 | 17 | | |
18 | 18 | | |
19 | | - | |
20 | | - | |
21 | | - | |
22 | | - | |
23 | | - | |
24 | | - | |
25 | | - | |
26 | | - | |
27 | | - | |
28 | | - | |
29 | | - | |
30 | | - | |
31 | | - | |
32 | | - | |
33 | | - | |
34 | | - | |
35 | | - | |
36 | | - | |
37 | | - | |
38 | | - | |
39 | | - | |
40 | | - | |
41 | | - | |
42 | | - | |
43 | | - | |
44 | | - | |
45 | | - | |
46 | | - | |
47 | | - | |
48 | | - | |
49 | | - | |
50 | | - | |
51 | | - | |
52 | | - | |
53 | | - | |
54 | | - | |
55 | | - | |
56 | | - | |
57 | | - | |
58 | | - | |
59 | | - | |
60 | | - | |
61 | | - | |
62 | | - | |
63 | | - | |
This file was deleted.
Lines changed: 0 additions & 68 deletions
This file was deleted.
Lines changed: 0 additions & 23 deletions
This file was deleted.
0 commit comments