Skip to content

Use serial ESMF in xESMF data tests#788

Open
jder wants to merge 1 commit into
mainfrom
codex/stabilize-xesmf-data-tests
Open

Use serial ESMF in xESMF data tests#788
jder wants to merge 1 commit into
mainfrom
codex/stabilize-xesmf-data-tests

Conversation

@jder

@jder jder commented Jul 9, 2026

Copy link
Copy Markdown
Member

Force the conda-forge nompi ESMF build in the serial xESMF data-test workflow.

We are getting intermittent failures for these tests which show up as SIGTERMs. I don't believe it's an out-of-memory issue (the test data is tiny) and can't reproduce these locally. Running under Valgrind Memcheck in docker environments produced many errors during UCX TCP reachability discovery in:

uct_tcp_iface_is_reachable_v2
ucp_wireup_select_transport
MPIDI_UCX_init_local
MPI_Init_thread
ESMCI::VMK::init
ESMF_Initialize

As a control, the same MPI initialization under Valgrind with UCX_TLS=self reported 0 errors. That isolates the finding to unnecessary network transport discovery.

Python/glibc allocator-debug stress using PYTHONMALLOC=debug, MALLOC_CHECK_=3, and MALLOC_PERTURB_ also completed 500 regrids on both builds without detecting heap corruption.

All together, this is not really proof that the SIGTERM on GH is caused by this (since we can't repro locally to get more info) but seems worth trying.

@jder jder requested a review from alxmrs July 10, 2026 16:22
@jder jder marked this pull request as ready for review July 10, 2026 16:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant