Skip to content

[rocksdbjni] Filter config options apparently lost after opening OptimisticTransactionDB #13519

Open
@OStevan

Description

@OStevan

We've been trying unsuccessfully to enable Bloom filter usage by configuring BlockBasedTableConfig in several ways, we tried enabling it through ColumnFamilyOptions and Options at the DB level.

The outcome we see is that after opening the DB and trying to retrieve db.getDefaultColumnFamily().getDescriptor().getOptions().tableFormatConfig() we get a null object back and when trying to retrieve the rocksdb.bloom.filter.full.positive is -1.

We noticed a PR that might explain this or is at least similar in the outcome we are seeing: #13294.

Is it possible to get a rocksdbjni release with the fix such that we can try it out or do you maybe have any advice for us what to check? We looked at other repos and how they set up bloom filters and we haven't found anything significantly different from what other repos do so we don't believe we made a mistake but any advice is welcome.

simplified initialization code:

        try (Options opts = new Options()) {
            opts.setCreateIfMissing(true).setCreateMissingColumnFamilies(true);
            opts.setStatistics(stats);
            opts.setBytesPerSync(SizeUnit.MB);
            BlockBasedTableConfig blockBasedTableConfig = opts.tableFormatConfig() instanceof BlockBasedTableConfig
                    ? (BlockBasedTableConfig) opts.tableFormatConfig()
                    : new BlockBasedTableConfig();
            blockBasedTableConfig.setFilterPolicy(new BloomFilter(10));
            opts.setTableFormatConfig(blockBasedTableConfig);
            opts.setWriteBufferSize(HumanReadableByteCount.mebibytes(16).toBytes());
            opts.setLevelCompactionDynamicLevelBytes(true);
            var db = checkNotNull(
                    OptimisticTransactionDB.open(opts, dataDirectoryString),
                    "unexpected null value without rocks exception");

            // we changed this just to see it gets applied
            checkState(db.getDefaultColumnFamily().getDescriptor().getOptions().writeBufferSize()
                    == HumanReadableByteCount.mebibytes(16).toBytes());
            // this throws with options being null
            checkState(db.getDefaultColumnFamily().getDescriptor().getOptions().tableFormatConfig() != null);
            return db;
        }

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions