Consumer pause/Resume funtionality leasks memory

[jyberg]

It looks like consumer pause/resume functionality leaks memory, at least in situation when consumed partitions has all the time lot of data available.
I have tested this with only consuming data from kafka. Without Pause/Resume, With adaptive Pause/Resume and Aggressive Pause/Resume.
Pod memory utilization shows clearly that memory consumption is not healthy. From consumer statistics I can't find any reason for memory issue.

In attachment pictures from memory usages, and producer configuration parameters

consumer confic.txt

[mensfeld]

FYI I was not able to replicate this with the ruby bindings running for several minutes. With 10 000 pause/resume cycles the memory increase was close to 0.

[jyberg]

In our system, during the test application received 120k reports/second and producer produced little more to keep back pressure. In aggressive case had hundred or thousand pause/resumes on second. That is not normal case it was only to test and get enough cycles.

[jyberg]

Hi, here is still one picture that shows relation of pause calls and pod memory usage pause count graph has wrong y axis label it is not Bytes it is just count...