Failed to build and use HDF5 with existing openMPI

[adamdarx]

Because I haven't installed Parallel HDF5 on my WSL system, I decided to use make script provided in KHARMA but I received error messages as follows. Actually, when I use Parthenon isolatedly I encounter this issue too.

adamdarx@adamdarx:~/kharma$ ./make.sh clean cuda hdf5
Configuring HDF5...
configure: error: unable to link a simple MPI-IO C program
Building HDF5 (probably 30s-2min)
Built HDF5
/home/adamdarx/kharma
-- The C compiler identification is GNU 12.4.0
-- The CXX compiler identification is GNU 12.4.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /home/adamdarx/kharma/bin/nvcc_wrapper - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found MPI_C: /usr/local/openmpi/lib/libmpi.so (found version "3.1")
-- Found MPI_CXX: /usr/local/openmpi/lib/libmpi_cxx.so (found version "3.1")
-- Found MPI: TRUE (found version "3.1")
CMake Warning at external/parthenon/cmake/MachineCfg.cmake:61 (message):
  Not using any machine configuration.  Consider creating a configuration
  file following the examples in /home/adamdarx/kharma/cmake/machine_cfgs/
  and then point the MACHINE_CFG cmake or environment variable to your custom
  file.Note, that the machine file can be placed in any directory (also
  outside the repo).
Call Stack (most recent call first):
  external/parthenon/CMakeLists.txt:21 (include)


CMake Warning at external/parthenon/cmake/Format.cmake:53 (message):
  Couldn't determine clang-format version.  clang-format >=11.0 is required -
  results on previous versions may not be stable
Call Stack (most recent call first):
  external/parthenon/CMakeLists.txt:61 (include)


-- Looking for C++ include filesystem
-- Looking for C++ include filesystem - found
-- Performing Test CXX_FILESYSTEM_NO_LINK_NEEDED
-- Performing Test CXX_FILESYSTEM_NO_LINK_NEEDED - Success
-- Found MPI: TRUE (found version "3.1") found components: CXX
-- Found HDF5: hdf5::hdf5-static (found version "2.0.0") found components: C
CMake Error at external/parthenon/CMakeLists.txt:146 (message):
  Both MPI and HDF5 are enabled but only a serial version of HDF5 was found.
  Please install a parallel version of HDF5 (or point CMake to it by adding
  its path to the CMAKE_PREFIX_PATH environment variable or by setting the
  HDF5_ROOT variable), or disable either MPI or HDF5 by rerunning CMake with
  -DPARTHENON_DISABLE_MPI=ON or -DPARTHENON_DISABLE_HDF5=ON


-- Configuring incomplete, errors occurred!
make: *** No targets specified and no makefile found.  Stop.
cp: cannot stat 'kharma/kharma.*': No such file or directory

I have tried manually compiled HDF5 and OpenMPI from source code but I failed again and again.

[c-prather]

Hi! It looks like you're using HDF5 2.0 (hdf5::hdf5-static (found version "2.0.0")), at least in the logged compile, is this correct? I believe Parthenon autodetects parallel vs serial hdf5 incorrectly for version 2.0+, though it's otherwise compatible.

Maybe try applying external/patches/mac-parthenon-no-complain-hdf5.patch (which just comments out this error) and see if anything is actually wrong.

If things still don't work, that means whatever you're doing is compiling serial HDF5 rather than parallel. You could compile KHARMA with the nompi argument to make.sh to disable MPI altogether, which would at least get you started on single GPUs or CPUs (OpenMP shared-memory parallelization is supported, so the performance difference with/without MPI on a single node is minimal). In the long term, failing to compile parallel HDF5 is almost certainly an issue with MPI on the machine or cluster you're compiling on.