Skip to content

Include ksqldb-rocksdb-config-setter jar in distributed artifacts #6838

@colinhicks

Description

@colinhicks

The How to Limit Memory Usage in ksqlDB blog post details the use of KsqlBoundedMemoryRocksDBConfigSetter, but the containing jar is not on the classpath – nor included at all in releases, including the standalone Docker image.

This yields the following when the custom config setter is otherwise correctly configured:

org.apache.kafka.common.config.ConfigException: Invalid value io.confluent.ksql.rocksdb.KsqlBoundedMemoryRocksDBConfigSetter for configuration rocksdb.config.setter: Class io.confluent.ksql.rocksdb.KsqlBoundedMemoryRocksDBConfigSetter could not be found.
	at org.apache.kafka.common.config.ConfigDef.parseType(ConfigDef.java:729)
	at io.confluent.ksql.config.ConfigItem$Resolved.parseValue(ConfigItem.java:125)
	at io.confluent.ksql.util.KsqlConfig.lambda$resolveStreamsConfig$2(KsqlConfig.java:902)
	at java.base/java.util.Optional.map(Optional.java:265)

At minimum, we should ensure that the ksqldb-rocksdb-config-setter jar is included in distributed artifacts. We should consider altering ksql-run-class to automatically add it to the classpath.

As a workaround, users of the standalone Docker image would need to build the ksqldb-rocksdb-config-setter jar at the release version, mount it in the container, and ensure it is added to the classpath.

Metadata

Metadata

Assignees

Labels

P1Slightly lower priority to P0 ;)core-infraIssues owned by the ksqlDB Core Infra teamenhancementfix-it-week

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions