Skip to content

[BUG] compact_range miss keys in [begin, end), causing compactor_filter fail to remove outdated keys #13526

Open
@979357361

Description

@979357361

We need to filtered out some outdated keys sometimes, we suppose to achieve this with CompactRange and CompactFilter. For comapction performance reasons, we only create NonNull filter in manual_compction, (for auto compact, filter do nothing and ret diretly). The other options are left as default (exclusive_compact = false).

However, we found sometimes after filter, there still has outdated KVs, leading to data error.
We searched here and found some post mentioned that compact_range may miss keys

#11468 , https://github.com/cbi42/rocksdb/blob/796f58f42ad1bdbf49e5fcf480763f11583b790e/db/db_impl/db_impl_compaction_flush.cc#L1180C29-L1184

what sould we do to ensure all keys in [begin, end) are compacted and passed through the compact filter? We may have many CompactRange with different [begin, end) run simultaneously, if we set exclusive_compact = true for each one, will the later call cancel the ongoing prev CompactRange or waiting the prev finish? the overall execution time will also increase?

BTW, C API of ComapctRange return void with no err msg, wouldn't this C API call fail?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions