Viewing hazelcast map causes memory overflow

[sandiegoe]

Hello, we are using your open source hazelcast-monitor and encountered an OutOfMemory error. We have a very large IMap cache with a data volume of about 10 million. In hazelcast we have configured a maximum capacity of 2 million, but when querying the map data through hazelcast-monitor, it will cause memory overflow. .
After reading the code, I found that all the data was loaded during the query. I want to confirm whether this is the reason.

[daniloarcidiacono]

Hello,
yes the monitor will indeed pull each entry of the Map and apply the filtering predicate.
In your case I suspect that there is no filter so all 2 million entries are added in the resulting list, causing the OutOfMemoryError.

At the time of writing, I was using Hazelcast 3.9.3 which didn't have any facility to do this type of things, so I chose to implement this manual filter. However, even with enough memory and/or a restrictive filter, I would expect very poor performance on such a large Map, let alone the fact that the query is executed periodically.

A possible improvement would be to upgrade Hazelcast to at least version 3.12 which introduces paginated distributed queries with SQL syntax (see Hazelcast Docs: Distributed Query, and Hazelcast Docs: Querying in Collections and Arrays), which I expect to perform much better.

Last but not least, please keep in mind that hazelcast-monitor is intended as an aid during development rather than a battle-tested tool for production monitoring.

[sandiegoe]

Thank you very much for your answers！

The monitor tool uses true by default to filter data. It is best to add a limit to the number of existing implementations to avoid possible problems