Commit f1423da
committed
feat: Implement temporal tensor store with block-based tiered compression
Implements the block-based storage engine specified in ADR-018 through
ADR-023 with 5 new modules and 1 benchmark/test suite.
New modules:
- store.rs (1056 lines): BlockKey, BlockMeta, Tier, TieredStore with
HashMap index, per-tier data storage, CRC32 checksums, eviction,
and BlockIO/MetaLog/Clock traits
- tiering.rs (846 lines): EMA + popcount + recency scoring with LUT-based
fast_exp_neg, hysteresis, min_residency, budgeted maintenance,
MigrationCandidate selection, warm aggressive mode (7->5 bit)
- delta.rs (825 lines): Sparse delta format (u16 index + i16 value),
DeltaChain with bounded length and compaction, FactorSet for
low-rank reconstruction, encode/decode serialization
- metrics.rs (770 lines): WitnessLog (ring buffer), WitnessEvent enum
(Access, TierChange, Eviction, Maintenance, Compaction, etc.),
StoreMetrics aggregates, StoreSnapshot serialization
- store_ffi.rs (680 lines): tts_init/put/get/tick/stats/touch/evict
WASM exports with u128 split into hi/lo u64, feature-gated
Optimizations:
- 8-bit fast path in quantizer: direct byte read/write, no bit
accumulator. Dequant: 7313ns -> 1290ns (5.7x faster, 12.7 GB/s)
- 8-bit fast path in bitpack: direct copy, no accumulator.
Pack: 8484ns -> 742ns (11.4x), Unpack: 8845ns -> 396ns (22.3x)
- #[inline] on hot functions
Benchmark results (release, 16KB blocks):
Quantize 8-bit: 18.9us Dequant 8-bit: 1.3us (12.7 GB/s)
Quantize 3-bit: 22.5us Dequant 3-bit: 7.2us (2.3 GB/s)
Score compute: 10ns Single frame decode: 178ns
Segment 8-bit decode: 1.5us (11.2 GB/s)
Zipf P95 read: 48ns Tier flip rate: 0.074/block/min
Quality (all PASS):
8-bit: 0.39% max error 7-bit: 0.79% max error
5-bit: 3.33% max error 3-bit: 16.67% max error
Tests: 170 unit + 12 integration/benchmark, all passing.
https://claude.ai/code/session_01Ksy165BL5nGpVoWaAfTE7t1 parent 67ce0aa commit f1423da
9 files changed
Lines changed: 5548 additions & 3 deletions
File tree
- crates/ruvector-temporal-tensor
- src
- tests
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
6 | 6 | | |
7 | 7 | | |
8 | 8 | | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
9 | 12 | | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
10 | 19 | | |
11 | 20 | | |
12 | 21 | | |
| |||
28 | 37 | | |
29 | 38 | | |
30 | 39 | | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
31 | 43 | | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
32 | 51 | | |
33 | 52 | | |
34 | 53 | | |
| |||
0 commit comments