Regression-tests don't link inside "make check" but appear to work ok outside of it.

[cponder]

The system is an ARM64 cluster running Rocky Linux 9.3.
The native GCC is 11.4.1 but I'm using a custom build of 14.2.0, and believe the problem is collisions between my built libraries and the ones installed under /usr.
When I run this

make -i -k V=1 check

ALL of the cases report failures like this:

FAIL: nctst
===========
/work/01768/ponderc/vista/SHARE/Utils/PNetCDF/1.14.0/GCC-14.2.0_OpenMPI-5.0.7/distro/test/CXX/.libs/nctst: symbol lookup error: /work/01768/ponderc/vista/SHARE/Utils/OpenMPI/5.0.7/GCC-14.2.0_CUDA-12.8.0.0_570.86.10_UCX-1.18.0_HWLoc-2.12.0_PMIx-5.0.6_ZLib-1.3.1/lib/libopen-pal.so.80: undefined symbol: pmix_framework_names
FAIL nctst (exit status: 127)

I believe my environment is set up correctly, though -- the PNetCDF doesn't complain about the build, and if I run the command

distro/test/CXX/.libs/nctst

standalone, it looks like it did work

*** TESTING C++ nctst for APIs with different netCDF formats       ------ pass

and this file gets created

-rw-r--r-- 1 ponderc G-821262 1692 Feb 25 07:35 testfile.nc

My suspicion is that the harness is shuffling the $LD_LIBRARY_PATH, but don't see an easy way to check this.
I tried putting /usr/lib64 at the front but got a different failure about the GLIBCXX version.

[cponder]

My HDF5 build passes most of its regression-tests but a few cases give the same error as above.
I expect that different parts of their make check harness are coded different ways, so it wouldn't be surprising that they behave differently.

Regressions for the PIO library all pass.
NetCDF-F regressions all fail.
Some of the NetCDF-C regressions appear to pass.

[wkliao]

This looks like a autoconf issue.
Could you try the master branch which has recently been updated with fixes
and now requires libtool version 2.5.4.

Also, a quick googling of the error message lands me to this OpenMPI issue. Not sure if it is relevant.

[cponder]

You mean the Master Branch of PNetCDF? When do you think you'll spin a new release?

The nctest, if I run outside the harness, should I use mpirun with some number of procs?

The OpenMPI issue is pretty old, also mentioned was that it had been caused by a sloppy build.

[wkliao]

Yes, PnetCDF's master branch.

Command 'make check' will run test/CXX/nctst on one MPI process.
Command 'make ptest' will run it on 4 processes.

When using OpenMPI, please add '--oversubscribe' to the mpirun command line.

[wkliao]

I just now used gcc 14.2.1 to build OpenMPI 5.0.7 and PnetCDF 1.14.0.
Command 'make check' ran fine without errors.

If you are using a customized GCC 14.2.0, maybe you want to rebuild everything
with the same GCC, including OpenMPI.

[wkliao]

libopen-pal.so.80: undefined symbol: pmix_framework_names

Such error message may be due to the use of different versions of GCC.

It appears that your OpenMPI was built with a CUDA version of GCC
(GCC-14.2.0_CUDA), but a different GCC (GCC-14.2.0_OpenMPI-5.0.7)
is used to build PnetCDF. If that is the case, can you try the same GCC?

[wkliao]

Hi @cponder
I wonder if PR #164 resolves this issue for you.
If you can try the latest master branch and let me know, that will be great.

[cponder]

I tried the zip'd version of the latest "master" branch, but it didn't contain a configure script.
Can you generate a 1.15.rc1 release-candidate that I can try?

[wkliao]

Here is the tar ball.

pnetcdf-1.14.1.rc1.tar.gz