libmesh build error on INL cluster

[aruybalid]

Hi,

I am a LANL-postdoc developing surrogate models in MOOSE (working with Laurent Capolungo, Topher Matthews and Andrea Rovinelli). I am relatively new to working with MOOSE on the INL cluster and experiencing problems installing libmesh on the INL cluster (sawtooth) through update_and_rebuild_libmesh.sh. I have just cloned MOOSE, compiled PETSC successfully (update_and_rebuild_petsc.sh), and get an error during libmesh build (see first few lines of error copied below). How can I resolve this to compile libmesh?

Warm regards,
Andre Ruybalid

../src/base/libmesh.C: In constructor 'libMesh::LibMeshInit::LibMeshInit(int, const char* const*, TIMPI::communicator, int)':
../src/base/libmesh.C:441:38: error: 'using_threads' was not declared in this scope; did you mean 'n_threads'?
441 | new TIMPI::TIMPIInit(argc, argv, using_threads,
| ^~~~~~~~~~~~~
| n_threads
../src/base/libmesh.C:442:26: error: 'handle_mpi_errors' was not declared in this scope
442 | handle_mpi_errors);
| ^~~~~~~~~~~~~~~~~

[GiudGiud]

Hello

Which modules did you load for compilers? Especially for MPI
For next time, there's a pre-built PETSC module you can load on hpc, saves one step.

Guillaume

[GiudGiud]

what does echo $CC $CXX return ?
You may need to set CC=mpicc CXX=mpicxx when running this script

[aruybalid]

echo $CC $CXX returns:

/apps/local/spack/software/gcc-4.8.5/gcc-9.3.0-twlsdabweaw75pj2f2hq7k7vo3vilvmi/bin/gcc
/apps/local/spack/software/gcc-4.8.5/gcc-9.3.0-twlsdabweaw75pj2f2hq7k7vo3vilvmi/bin/g++

It should be the following (I had added it to .bashrc, but now am unsure if I sourced it)?

export CC=mpicc
export CXX=mpicxx

[aruybalid]

Actually, I added this (in this order) to .bashrc:

module purge
export CC=mpicc
export CXX=mpicxx
export F90=mpif90
export F77=mpif77
export FC=mpif90
export PETSC_DIR=/home/ruybandr/projects/moose/petsc
export PETSC_ARCH=arch-moose all
module load openmpi
module load python
module load cmake

It seems loading openmpi after export CC=mpicc and export CXX=mpicxx changes the CC and CXX selection. I am now attempting to rebuild libmesh with CC=mpicc and CXX=mpicxx.

I'll keep you posted..

[GiudGiud]

ok hope it works!
mvapich2/2.3.3-gcc-9.2.0 is the one everyone uses on sawtooth I think btw. Openmpi should work too
With the pre-built petsc module, you ll have to make sure to load the relevant compiler too

[aruybalid]

It worked. So first:

module load openmpi

and subsequently set

export CC=mpicc
export CXX=mpicxx
etc.

Thanks for your help.

Andre