Evaluating new cache designs

[byrnedj]

Hi everyone,

I am evaluating new cache designs specifically for hybrid systems (using DRAM as L1 and NVM as L2).
CacheLib comes pretty close to supporting our designs out of the box, except for two things (to the best of my knowledge):

1. Fixed amount of items per size class in NVM (block_cache/BigCache) - It looks like we can put items in the same size class in the same eviction pool, but it doesn't seem like we can set an upper limit in terms of bytes for a given size class.

2. Data replication (inclusive vs. exclusive) - For classes of items that tend to receive mostly reads, employing an inclusive cache design can reduce writes to NVM in a hybrid cache. When an item is brought from L2 to L1, we leave a copy of it in L2. Eventually, the L1 copy will get evicted, if the L1 copy is clean w.r.t. the L2 version, we don't need to write back the data to L2.

I'm wondering, where would be the best place to start in the code for implementing these two features?

I want to compare these against admission policies at the object level. CacheLib seems to be the perfect system to do so.

[therealgymmy]

Can you explain more what you're trying to do? And what is your high level cache design / needs? You mention you want to compare admission policies. It would be great if you could elaborate on them.

For your questions:

1. We do not support setting an upper-limit on size class in Navy. In addition, we're actually going to deprecate size-class support in the near future (all different sizes will be written to the same block-cache region in-memory and then flushed onto device). Can you explain why you need a limit on size-class?
2. CacheLib is already inclusive. When we read an item into RAM-cache, the flash-copy is kept while the ram-copy is marked as NvmClean. When the ram item is mutated, user can call an API to mark it as "unclean" and invalidate its corresponding flash copy.

[byrnedj]

Hi, thanks for the response. I have two main goals:

1. Compare designs such as inclusive/exclusive for a particular class of objects (i.e write-heavy or read-heavy). I want to measure the number of NVM writes as a result of cache design.
2. For a given amount of cache space (DRAM and NVM) how does that effect the number of NVM writes? To reference your answer to my (1), I am thinking that just partitioning DRAM may be enough and NVM should just be pooled anyway.

It would be interesting to see how (1) and (2) stack up against typical NVM admission policies.

Thanks for the clarification on my question (2), with that answer, is there a way to make the cache exclusive? i.e. delete the flash-copy when brought to DRAM

[sathyaphoenix]

@byrnedj deleting the flash copy when the object is brought into DRAM would incur an extra write to delete as well as another write to write the object back into NVM when it is then evicted from DRAM subsequently. Isn't that against what you are trying to solve. This can definitely be prototyped by tweaking the logic in NvmCache.h if you are interested in trying out.

There is some limit on storing objects by size since we have SOC(BigHash) and LOC(BlockCache) allocated a certain portion of flash up-front during initialization. In DRAM, CacheLib has an MMContainer per size class per pool and the PoolRebalancer can be used to control the memory allocated per size. Unlike DRAM, we don't have this mechanism for NVM. However, you can use the SFIFO and the reinsertion policy in Navy to admit objects with a probability associated with their size if you want to experiment with adding a size class based approach.

[pbhandar2]

Closing this discussion as it has been inactive. If this is still relevant, then please create a new discussion post and we will get back to you as soon as we can. Thank you!