Please check the domain decomposition structure!

[christina-rice]

I received the following warning and I am not sure how to adjust. I increased the number of processors and decreased the hematocrit but neither seemed to be the solution. Any suggestions?

"(HemoCell) WARNING: The number of atomic neighbours is suspiciously high: 92 Usually it should be < 30 ! Please check the domain decomposition structure!"

[gzavo]

Hi @christina-rice,

This means that at least some atomic blocks (subdomains computed by single CPU cores) are very small. When they are smaller than the communication envelope (typically ~25 lattice units), then these domains need to communicate not just the next-neighbours, but also next-next-neighbours. This can create a large communication overhead, and can result in bad performance.
This can happen if you started a relatively small overall domain on many CPU cores, because then the domain gets decomposed to very small atomic blocks (<25 lattice units).
It is not necessarily bad, but it signals that adding more CPUs for the same domain size might not increase the performance anymore.

[christina-rice]

Thank you!

I was increasing the processors because of the message "(Voxelizer) (Warning) Running with 1440 CPU's, consider running with 1548 CPU's for optimal efficiency," but I guess there might be competing divisions that cause some processes to want more CPUs and others to want fewer CPUs.