Is the values.heapOptions in cp-ksql-server doing anything?

[dberardo-com]

I have set the value: heapOptions: "-Xms64m -Xmx512m in the ksqldb (v0.26+) chart, and i cannot see it having an effect on the JVM. I see the values being passed to the ksqldb process inside the container, but when i run it in production i see its RAM consumption level at a very stable value which is higher than 512MB.

In particular it seems that ksqldb allocates memory w.r.t to the number of tables/streams it has to deal with. in fact i have 3 namespaces with 3 independent ksqldb instances (i.e. they share same kafka cluster + schema reg. but have different service ids) and i see these memory level:

• namespace 1: ksqldb instance has no tables, no streams, no queries, and memory usage of 461MB (but it is doing nothing ...)
• namespace 2: ksqldb instance with 7 tables, 3 streams, 2 pers. queries (joins) --> mem usage: 1.1 GB
• namespace 3: exactly the same setup and schemas as namespace2 (but on different topics) --> mem usage: 1.13 GB

so i wonder ... is it possible to control the RAM usage of ksqldb via the heap options or is it a must to use RockDB cache configs? and even if not, what are heapOptions for then ?

[OneCricketeer]

How are you measuring usage? RocksDB isn't stored in the JVM heap, nor can it be configured there.

[dberardo-com]

i am using the default prometheus helm chart, which monitors the ksqldb service, i assume that the total value is JVM + rocksdb cache

[OneCricketeer]

I don't think it does. The JMX exporter will only track JVM heap.

Try something like this to get full container memory usage - https://stackoverflow.com/questions/55143656/prometheus-queries-to-get-cpu-and-memory-usage-in-kubernetes-pods

[dberardo-com]

thanks for sharing, i will have a look and post my results in here. I really want to get memory management under control, so i will try to set up proper monitoring first