RFC: Introduce Object disk cache engine #1200
Description
Based on the idea posted previously in my blog https://blog.mrcroxx.com/posts/foyer-a-hybrid-cache-in-rust-past-present-and-future/#213-disk-cache-architecture . Foyer expects to use different disk cache engines to optimize the disk cache for cache entries of different sizes. Among them, for entries ranging from several MB to GB, we expect to introduce an Object disk cache engine for caching.
As the name suggests, the "Object" engine treats each cache entry as an object and caches it directly without any merging. This is suitable for cache entries with larger sizes. Since the entries are large and fewer in total, they do not put much pressure on the metadata management of the storage device.
At the same time, we can perform cache eviction at the per-object granularity, which is more precise than the block engine's block-level granularity.
In theory, the object engine should be the easiest to implement. The reason it hasn't been implemented is that Foyer previously planned to support all engine types for all device types. For example, supporting object cache on a raw block device is almost equivalent to rewriting a user-space file system.
After discussion and consideration, I think Foyer can refer to the model of OpenDAL, where each device is similar to a service in OpenDAL and has a compatible feature list. In this way, we can iterate new features more quickly for the majority of users' use cases (using a file system directory as the disk cache device) without having to consider complex compatibility issues.
Similarly, this also brings new benefits. We can introduce devices with more complex interfaces, and even use OpenDAL directly as a type of device. Imagine: for some entries, we use disk cache; for others, we use OpenDAL to cache them directly to remote Redis; or even combine the two to implement more tiers of caching.
In general, we need to implement this step by step. The plan may continue to be optimized during the process. The first things we need to do are:
- Add compatibility attributes to the device to facilitate engine compatibility verification.
- Standardize the interface to support more types of devices.
- Add an Object disk cache engine.