Conserve mapping with ESMF_RegridWeightGen

[alexolinhager]

ESMF_RegridWeightGen contains an undetermined bug that was introduced after compass version 1.8.0-alpha.2. This bug only seems to exist when using the 'conserve' mapping method to interpolate at high resolution. The bug was discovered using conservative mapping to interpolate a 150 m data product onto a 1-10km MALI mesh. The bug was not triggered interpolating the same 150 m data product onto a 4-40 km MALI mesh of the same domain. 'Bilinear' and 'patch' mapping methods worked successfully with the 1-10km mesh, so the issue seems to be solely with the 'conserve' mapping option.

The bug occurs when using the command: srun -n 128 ESMF_RegridWeightGen --source BedMachineGreenland-v5.scrip.nc --destination UummannaqDisko_scrip.nc --weight gridded_to_MPAS_weights.nc --method conserve --netcdf4 --dst_regional --src_regional --ignore_unmapped

When run as part of a bash script, ESMF_RegridWeightGen gets stuck indefinitely and will not produce any output or progress for the duration of a 12 hour job. If run in isolation on an interactive node, ESMF_RegridWeightGen produces a general segmentation fault.

This issue does not occur with compass version 1.8.0-alpha.2 (tested at 52b6a6c), but does occur with compass 1.9.0-alpha.1. It seems likely the bug was introduced with the update of ESMF here: f995c42

[xylar]

This would be something to report to ESMF developers if you can provide a small reproducer that works v8.8.1 and fails with v8.9.0 (which I believe are the versions in compass 1.8.0-alpha.x and 1.9.0-alpha.y, respectively)

[xylar]

I would recommend to the MALI team more broadly to consider transitioning to using mbtempest for emapping where possible (especially conservative remapping) because it is faster and, in my experience, more reliable in parallel than ESMF.

[jonbob]

@xylar -- I'm trying to move to mbtempest for my workflow as well. I spent some time with Vijay at the all-hands meeting and we came up with settings and a version that worked especially well for a very tough regridding between two large meshes. But there are still a couple of issues before I can use it for all the mapping files I make. Still, I think it's a great idea to start transitioning