W-TinyLFU policy should update the frequency of the key even if it is not in the cache #849
Description
Hi.
In the mokabench result moka-rs/mokabench#20 (comment), foyer had lower hit ratio than moka in the same workload.
| Cache | Size | Num Clients | Hit Ratio (Higher is better) |
|---|---|---|---|
| Foyer in-memory | 100,000 | 1 | 9.438 |
| Moka Async | 100,000 | 1 | 10.439 |
| Foyer in-memory | 400,000 | 1 | 28.592 |
| Moka Async | 400,000 | 1 | 42.470 |
| Foyer in-memory | 800,000 | 1 | 64.728 |
| Moka Async | 800,000 | 1 | 70.331 |
I think foyer used the W-TinyLFU policy, and moka used the TinyLFU policy for the benchmark. I believe they should have similar hit ratios for this workload (ARC S3). For the expected results, see this comment moka-rs/moka#446 (comment) having the results from Caffeine Simulator.
I briefly reviewed the foyer code, and found potential errors in the W-TinyLFU policy implementation.
1. The W-TinyLFU policy does not seem to update the frequency of the key if it is not in the cache.
For a cache read, foyer should update the frequency of the key no matter if it is in the cache or not.
foyer/foyer-memory/src/eviction/lfu.rs
Lines 368 to 370 in 92522d3
The inferred type for record is &Arc<Record<Lfu<K, V>>>. This implies the closure will never be called if a Record does not exist in the cache.
2. The W-TinyLFU policy updates the frequency of the key when inserting it into the cache.
It should not update the frequency of the key for cache writes. At least, moka does not do that. The popularity of the key should only be determined when it is requested, not when it is created.
foyer/foyer-memory/src/eviction/lfu.rs
Lines 277 to 287 in 92522d3
I think fixing 1. will improve the hit ratio of foyer in the mokabench result. So please check and fix it if necessary.