Omega on pm-cpu hanging and crashing #396
Description
I am seeing hanging and crashing in various places almost every time I run the omega_pr suite on pm-cpu. It is in different places (though often in the higher-res runs that take longer).
It is my hope that updating Polaris' modules (and spack environments) to match E3SM will help. This will have to wait a few more days until E3SM-Unified gets deployed, both so I have sufficient time to do it and because an updated version of mache is needed that will not be released until E3SM-Unified is also released.
It would be best to wait until E3SM-Project/E3SM#7872 gets merged into E3SM and then Omega