Description
We've been trying unsuccessfully to enable Bloom filter usage by configuring BlockBasedTableConfig
in several ways, we tried enabling it through ColumnFamilyOptions
and Options
at the DB level.
The outcome we see is that after opening the DB and trying to retrieve db.getDefaultColumnFamily().getDescriptor().getOptions().tableFormatConfig()
we get a null object back and when trying to retrieve the rocksdb.bloom.filter.full.positive
is -1.
We noticed a PR that might explain this or is at least similar in the outcome we are seeing: #13294.
Is it possible to get a rocksdbjni release with the fix such that we can try it out or do you maybe have any advice for us what to check? We looked at other repos and how they set up bloom filters and we haven't found anything significantly different from what other repos do so we don't believe we made a mistake but any advice is welcome.
simplified initialization code:
try (Options opts = new Options()) {
opts.setCreateIfMissing(true).setCreateMissingColumnFamilies(true);
opts.setStatistics(stats);
opts.setBytesPerSync(SizeUnit.MB);
BlockBasedTableConfig blockBasedTableConfig = opts.tableFormatConfig() instanceof BlockBasedTableConfig
? (BlockBasedTableConfig) opts.tableFormatConfig()
: new BlockBasedTableConfig();
blockBasedTableConfig.setFilterPolicy(new BloomFilter(10));
opts.setTableFormatConfig(blockBasedTableConfig);
opts.setWriteBufferSize(HumanReadableByteCount.mebibytes(16).toBytes());
opts.setLevelCompactionDynamicLevelBytes(true);
var db = checkNotNull(
OptimisticTransactionDB.open(opts, dataDirectoryString),
"unexpected null value without rocks exception");
// we changed this just to see it gets applied
checkState(db.getDefaultColumnFamily().getDescriptor().getOptions().writeBufferSize()
== HumanReadableByteCount.mebibytes(16).toBytes());
// this throws with options being null
checkState(db.getDefaultColumnFamily().getDescriptor().getOptions().tableFormatConfig() != null);
return db;
}