fix: fix w-TinyLFU frequencies updates

[MrCroxx]

What's changed and what's your intention?

Please explain IN DETAIL what the changes are in this PR and why they are needed. :D

As @tatsuya6502 suggested in #849 , this PR:

• Skip frequencies updates on cache miss.
• Skip frequencies updates on insertions.

Thank @tatsuya6502 for help. 🥰

Checklist

• I have written the necessary rustdoc comments
• I have added the necessary unit tests and integration tests
• I have passed make all (or make fast instead if the old tests are not modified) in my local environment.

Related issues or PRs (optional)

close #849

[MrCroxx]

bench_hit_ratio

baseline (main)

zif_exp, cache_size           fifo            lru             lfu             s3fifo (0g)     s3fifo (1g)     moka            
  0.90,  0.005                16.26%          18.86%          32.35%          32.16%          31.94%          33.41%
  0.90,   0.01                22.53%          25.79%          38.52%          39.07%          38.42%          37.81%
  0.90,   0.05                41.11%          45.18%          55.42%          56.64%          55.35%          55.27%
  0.90,    0.1                51.10%          55.31%          63.85%          65.20%          63.63%          64.18%
  0.90,   0.25                66.86%          70.87%          76.26%          77.60%          75.78%          77.14%
  1.00,  0.005                26.60%          30.60%          44.12%          44.27%          43.56%          45.63%
  1.00,   0.01                34.33%          38.66%          50.60%          51.39%          50.53%          50.69%
  1.00,   0.05                54.04%          58.35%          66.83%          67.97%          66.85%          67.01%
  1.00,    0.1                63.17%          67.25%          73.96%          74.99%          73.82%          74.37%
  1.00,   0.25                76.16%          79.62%          83.61%          84.52%          83.27%          84.38%
  1.05,  0.005                32.66%          37.23%          50.27%          50.12%          49.67%          51.86%
  1.05,   0.01                40.93%          45.62%          56.75%          57.57%          56.69%          57.06%
  1.05,   0.05                60.46%          64.68%          72.08%          73.15%          72.13%          72.35%
  1.05,    0.1                68.94%          72.78%          78.50%          79.46%          78.41%          78.97%
  1.05,   0.25                80.40%          83.51%          86.82%          87.52%          86.53%          87.44%
  1.10,  0.005                39.00%          43.98%          56.26%          56.36%          55.65%          57.89%
  1.10,   0.01                47.65%          52.51%          62.64%          63.41%          62.62%          63.13%
  1.10,   0.05                66.60%          70.61%          76.92%          77.90%          76.99%          77.17%
  1.10,    0.1                74.25%          77.79%          82.58%          83.29%          82.51%          83.02%
  1.10,   0.25                84.14%          86.86%          89.56%          90.08%          89.32%          90.09%
  1.50,  0.005                81.18%          84.98%          88.90%          89.27%          88.78%          89.90%
  1.50,   0.01                86.93%          89.69%          92.26%          92.67%          92.29%          92.80%
  1.50,   0.05                94.75%          95.96%          96.95%          97.06%          96.95%          97.09%
  1.50,    0.1                96.66%          97.46%          98.07%          98.10%          98.05%          98.14%
  1.50,   0.25                98.36%          98.79%          99.04%          99.02%          99.02%          99.09%

PR

zif_exp, cache_size           fifo            lru             lfu             s3fifo (0g)     s3fifo (1g)     moka            
  0.90,  0.005                16.25%          18.86%          33.28%          32.52%          31.94%          33.52%
  0.90,   0.01                22.54%          25.80%          39.38%          39.13%          38.41%          37.86%
  0.90,   0.05                41.13%          45.20%          55.42%          56.72%          55.40%          55.25%
  0.90,    0.1                51.06%          55.27%          63.82%          65.26%          63.60%          64.09%
  0.90,   0.25                66.83%          70.83%          76.23%          77.66%          75.71%          77.18%
  1.00,  0.005                26.59%          30.58%          45.07%          44.47%          43.54%          45.63%
  1.00,   0.01                34.41%          38.74%          51.14%          51.44%          50.58%          50.69%
  1.00,   0.05                54.05%          58.35%          66.79%          67.97%          66.80%          66.99%
  1.00,    0.1                63.17%          67.25%          73.91%          74.95%          73.80%          74.32%
  1.00,   0.25                76.19%          79.65%          83.66%          84.57%          83.32%          84.39%
  1.05,  0.005                32.66%          37.23%          51.15%          50.55%          49.66%          51.89%
  1.05,   0.01                40.98%          45.66%          57.07%          57.51%          56.74%          57.12%
  1.05,   0.05                60.48%          64.70%          72.06%          73.14%          72.12%          72.35%
  1.05,    0.1                68.90%          72.74%          78.47%          79.46%          78.36%          78.94%
  1.05,   0.25                80.37%          83.48%          86.78%          87.48%          86.51%          87.41%
  1.10,  0.005                39.04%          44.03%          57.07%          56.47%          55.69%          57.86%
  1.10,   0.01                47.65%          52.52%          62.78%          63.48%          62.62%          63.15%
  1.10,   0.05                66.58%          70.60%          76.93%          77.86%          76.98%          77.27%
  1.10,    0.1                74.28%          77.79%          82.57%          83.25%          82.50%          83.01%
  1.10,   0.25                84.18%          86.89%          89.56%          90.10%          89.33%          90.08%
  1.50,  0.005                81.18%          84.99%          88.93%          89.30%          88.77%          89.92%
  1.50,   0.01                86.91%          89.68%          92.23%          92.65%          92.29%          92.79%
  1.50,   0.05                94.76%          95.96%          96.95%          97.07%          96.95%          97.07%
  1.50,    0.1                96.65%          97.46%          98.06%          98.09%          98.05%          98.15%
  1.50,   0.25                98.36%          98.80%          99.04%          99.03%          99.02%          99.10%

[MrCroxx]

This PR shows higher hit ratio when zipf_exp is under 1.0 .

Let's check the behavior with mokabench.

[tatsuya6502]

As @tatsuya6502 suggested in #849 , this PR:

* Skip frequencies updates on cache miss.
* Skip frequencies updates on insertions.

Hi. Oops. I meant the following:

* Do frequencies update on both cache miss and hit.
* Skip frequencies updates on insertions.

I thought folyer was doing frequencies update only on cache hit, but I guess I was wrong?

[MrCroxx]

I thought folyer was doing frequencies update only on cache hit, but I guess I was wrong?

My bad. I may not have been fully awake when writing the PR. 🥹

[tatsuya6502]

Hi. I ran some mokabench benchmarks comparing the hit ratio of different foyer versions and configurations.

$ cargo run -F foyer --release -- --num-clients 1

Cache	Version	Policy	Shards	Max Capacity	Hit Ratio	Duration Secs
Foyer In-memory Cache	v0.14.0	W-TinyLFU (1%)	64	100,000	9.439	9.428
Foyer In-memory Cache	xx/lfu br	W-TinyLFU (1%)	64	100,000	9.432	8.909
Foyer In-memory Cache	v0.14.0	W-TinyLFU (1%)	1	100,000	9.415	6.583
Foyer In-memory Cache	xx/lfu br	W-TinyLFU (1%)	1	100,000	9.436	5.954
Moka Sync Cache (papaya)	papaya br	TinyLFU		100,000	10.439	38.352
Foyer In-memory Cache	v0.14.0	W-TinyLFU (1%)	64	400,000	28.606	8.977
Foyer In-memory Cache	xx/lfu br	W-TinyLFU (1%)	64	400,000	28.603	9.955
Foyer In-memory Cache	v0.14.0	W-TinyLFU (1%)	1	400,000	31.647	7.482
Foyer In-memory Cache	xx/lfu br	W-TinyLFU (1%)	1	400,000	31.579	7.729
Moka Sync Cache (papaya)	papaya br	TinyLFU		400,000	42.470	12.723
Foyer In-memory Cache	v0.14.0	W-TinyLFU (1%)	64	800,000	64.727	9.369
Foyer In-memory Cache	xx/lfu br	W-TinyLFU (1%)	64	800,000	64.725	9.810
Foyer In-memory Cache	v0.14.0	W-TinyLFU (1%)	1	800,000	64.761	5.691
Foyer In-memory Cache	xx/lfu br	W-TinyLFU (1%)	1	800,000	64.833	6.816
Moka Sync Cache (papaya)	papaya br	TinyLFU		800,000	70.331	10.838

• Foyer xx/lfu branch (#ecfa539e)
• Moka papaya branch (#9066978b)

As for the change to make Foyer not to update the frequency when inserting, the above tests are not very good to show the difference. It would be better to do a test where you insert the same key multiple times without reading it. It will skew the hit ratio if the frequency is updated on inserts.

I found that making the number of shards smaller (1) makes the hit ratio better when the max capacity is around 400k. I think having all keys in the same shard makes the eviction policy more effective.

I could not find the reason for the difference in hit ratio between the foyer and moka caches. I probably need to check the implementation and configuration of the CountMin sketch.

[MrCroxx]

As for the change to make Foyer not to update the frequency when inserting, the above tests are not very good to show the difference. It would be better to do a test where you insert the same key multiple times without reading it. It will skew the hit ratio if the frequency is updated on inserts.

I found that making the number of shards smaller (1) makes the hit ratio better when the max capacity is around 400k. I think having all keys in the same shard makes the eviction policy more effective.

I could not find the reason for the difference in hit ratio between the foyer and moka caches. I probably need to check the implementation and configuration of the CountMin sketch.

Thank you for helping, @tatsuya6502 . 🥰 I'm also investigating the cache hit/miss trace for more information. However, it might be a little slow these days because of some family activities during CNY.

[codecov]

Codecov Report

Attention: Patch coverage is 16.94915% with 98 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
foyer-memory/src/eviction/lfu.rs	0.00%	47 Missing ⚠️
foyer-memory/src/eviction/mod.rs	19.04%	34 Missing ⚠️
foyer-memory/src/raw.rs	47.05%	9 Missing ⚠️
foyer-memory/src/eviction/fifo.rs	0.00%	4 Missing ⚠️
foyer-memory/src/eviction/s3fifo.rs	0.00%	4 Missing ⚠️

❗ There is a different number of reports uploaded between BASE (efa47f7) and HEAD (5196ac1). Click for more details.

HEAD has 2 uploads less than BASE

Flag	BASE (efa47f7)	HEAD (5196ac1)
	3	1

Files with missing lines	Coverage Δ
foyer-memory/src/eviction/lru.rs	69.31% <100.00%> (-24.06%)	⬇️
foyer-memory/src/eviction/fifo.rs	0.00% <0.00%> (-93.26%)	⬇️
foyer-memory/src/eviction/s3fifo.rs	0.00% <0.00%> (-91.75%)	⬇️
foyer-memory/src/raw.rs	54.34% <47.05%> (-34.86%)	⬇️
foyer-memory/src/eviction/mod.rs	23.33% <19.04%> (-76.67%)	⬇️
foyer-memory/src/eviction/lfu.rs	0.00% <0.00%> (-85.33%)	⬇️

... and 55 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.