Skip to content

Include ksqldb-rocksdb-config-setter jar in distributed artifacts #6838

@colinhicks

Description

The How to Limit Memory Usage in ksqlDB blog post details the use of KsqlBoundedMemoryRocksDBConfigSetter, but the containing jar is not on the classpath – nor included at all in releases, including the standalone Docker image.

This yields the following when the custom config setter is otherwise correctly configured:

org.apache.kafka.common.config.ConfigException: Invalid value io.confluent.ksql.rocksdb.KsqlBoundedMemoryRocksDBConfigSetter for configuration rocksdb.config.setter: Class io.confluent.ksql.rocksdb.KsqlBoundedMemoryRocksDBConfigSetter could not be found.
	at org.apache.kafka.common.config.ConfigDef.parseType(ConfigDef.java:729)
	at io.confluent.ksql.config.ConfigItem$Resolved.parseValue(ConfigItem.java:125)
	at io.confluent.ksql.util.KsqlConfig.lambda$resolveStreamsConfig$2(KsqlConfig.java:902)
	at java.base/java.util.Optional.map(Optional.java:265)

At minimum, we should ensure that the ksqldb-rocksdb-config-setter jar is included in distributed artifacts. We should consider altering ksql-run-class to automatically add it to the classpath.

As a workaround, users of the standalone Docker image would need to build the ksqldb-rocksdb-config-setter jar at the release version, mount it in the container, and ensure it is added to the classpath.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Assignees

Labels

P1Slightly lower priority to P0 ;)core-infraIssues owned by the ksqlDB Core Infra teamenhancementfix-it-week

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions