How about introducing Rcoksdb BlobDB feature?

[ninsmiracle]

Feature Request

Is your feature request related to a problem? Please describe:
We have some business requirements to write relatively large kv into the Pegasus.
For example, a user wants to write hundreds of thousands of data per second, and every singal date is about 80KB.
We have made many attempts to optimize the write parameters, such as adjust the memtable number/size of rocksdb,but they are far away from our target.
I noticed a paper named WiscKey https://www.usenix.org/system/files/conference/fast16/fast16-papers-lu.pdf, this paper proposes a solution to separate the key and value of RocksDB, which helps to significantly reduce the write amplification factor.
I read the logic of Rocksdb BlobDB carefully (Inspiration comes from WiscKey):https://rocksdb.org/blog/2021/05/26/integrated-blob-db.html, I think it's the time to introduce the feature to pegasus now.

Describe the feature you'd like:
I'd like to implement this feature in Pegasus.

[acelyc111]

Good proposal! Is there any breaking changes? I guess it's an optional to enable BlobDB or not, right?

[ninsmiracle]

Good proposal! Is there any breaking changes? I guess it's an optional to enable BlobDB or not, right?

I read the source code of rocksdb. If it is a version after 8.0.0, the implementation of blobdb is relatively complete. Therefore, I think there is no need to make too many changes to introduce blobdb into the Pegasus current master branch. But specifically, I need to test this function on a test cluster.

[acelyc111]

Good proposal! Is there any breaking changes? I guess it's an optional to enable BlobDB or not, right?

I read the source code of rocksdb. If it is a version after 8.0.0, the implementation of blobdb is relatively complete. Therefore, I think there is no need to make too many changes to introduce blobdb into the Pegasus current master branch. But specifically, I need to test this function on a test cluster.

Good, look forward to your good news!