Issues with compiling HIT on hpc

[TheGreatCid]

Check these boxes if you have followed the posting rules.

• Q&A General is the most appropriate section for my question
• I have consulted the posting Guidelines on the Discussions front page
• I have searched the Discussions forum and my question has not been asked before
• I have searched the MOOSE website and the documentation does not answer my question
• I have formatted my post following the posting guidelines (screenshots as a last resort, triple back quotes around pasted text)

Question

Hi all,

I have been running moose on a specific hpc for a while. I went and changed some files in MOOSE and went to recompile but now HIT seems to be having issues. I tried a fresh install as well but I see the same problem.
I use the libmesh and petsc script for installation and then having my own installation of python which has worked in the past.

Here is the compile error I am seeing:

Building and linking /home/detorre/raccoon_blade/raccoon/moose/framework/contrib/capabilities/capabilities.so...
/home/detorre/raccoon_blade/raccoon/moose/framework/contrib/capabilities/capabilities.C:11:10: fatal error: Python.h: No such file or directory
   11 | #include <Python.h>
      |          ^~~~~~~~~~
compilation terminated.
Linking Library /home/detorre/raccoon_blade/raccoon/moose/framework/contrib/pcre/libpcre-opt.la...
Rebuilding symlinks in /home/detorre/raccoon_blade/raccoon/moose/framework/build/header_symlinks
Creating Unity Directory /home/detorre/raccoon_blade/raccoon/moose/framework/build/unity_src
Compiling C++ (in opt mode) /home/detorre/raccoon_blade/raccoon/moose/framework/contrib/hit/src/hit/parse.cc...
Compiling C++ (in opt mode) /home/detorre/raccoon_blade/raccoon/moose/framework/contrib/hit/src/hit/lex.cc...
Compiling C++ (in opt mode) /home/detorre/raccoon_blade/raccoon/moose/framework/contrib/hit/src/hit/braceexpr.cc...
Building and linking /home/detorre/raccoon_blade/raccoon/moose/framework/contrib/hit/hit.so...
Compiling C++ (in opt mode) /home/detorre/raccoon_blade/raccoon/moose/framework/contrib/exodiff/ED_SystemInterface.C...
Compiling C++ (in opt mode) /home/detorre/raccoon_blade/raccoon/moose/framework/contrib/exodiff/FileInfo.C...
Compiling C++ (in opt mode) /home/detorre/raccoon_blade/raccoon/moose/framework/contrib/exodiff/GetLongOpt.C...
Compiling C++ (in opt mode) /home/detorre/raccoon_blade/raccoon/moose/framework/contrib/exodiff/SL_tokenize.C...
Compiling C++ (in opt mode) /home/detorre/raccoon_blade/raccoon/moose/framework/contrib/exodiff/STRINGLIB_tokenize.C...
Compiling C++ (in opt mode) /home/detorre/raccoon_blade/raccoon/moose/framework/contrib/exodiff/Tolerance.C...
Compiling C++ (in opt mode) /home/detorre/raccoon_blade/raccoon/moose/framework/contrib/exodiff/add_to_log.C...
Compiling C++ (in opt mode) /home/detorre/raccoon_blade/raccoon/moose/framework/contrib/exodiff/check.C...
Compiling C++ (in opt mode) /home/detorre/raccoon_blade/raccoon/moose/framework/contrib/exodiff/create_file.C...
Compiling C++ (in opt mode) /home/detorre/raccoon_blade/raccoon/moose/framework/contrib/exodiff/exoII_read.C...
Compiling C++ (in opt mode) /home/detorre/raccoon_blade/raccoon/moose/framework/contrib/exodiff/exo_block.C...
Compiling C++ (in opt mode) /home/detorre/raccoon_blade/raccoon/moose/framework/contrib/exodiff/exo_entity.C...
Compiling C++ (in opt mode) /home/detorre/raccoon_blade/raccoon/moose/framework/contrib/exodiff/exodiff.C...
Compiling C++ (in opt mode) /home/detorre/raccoon_blade/raccoon/moose/framework/contrib/exodiff/iqsort.C...
/home/detorre/raccoon_blade/raccoon/moose/framework/contrib/hit/src/hit/hit.cpp:16:10: fatal error: Python.h: No such file or directory
   16 | #include "Python.h"

Here is my diagnostic script:

#############################################################################################
Influential Environment Variables

CC=/projects/aue/hpc/deploy/b306e83c/linux-rhel8-x86_64/gcc-12.1.0/openmpi-4.1.6-fa5w34i/bin/mpicc
CONDA_CHANNEL=https://conda.software.inl.gov/public
CONDA_EXE=/home/detorre/miniforge/bin/conda
CONDA_PYTHON_EXE=/home/detorre/miniforge/bin/python
CONDA_SHLVL=0
CURL_CA_BUNDLE=
CXX=/projects/aue/hpc/deploy/b306e83c/linux-rhel8-x86_64/gcc-12.1.0/openmpi-4.1.6-fa5w34i/bin/mpicxx
F77=/projects/aue/hpc/deploy/b306e83c/linux-rhel8-x86_64/gcc-12.1.0/openmpi-4.1.6-fa5w34i/bin/mpif77
F90=/projects/aue/hpc/deploy/b306e83c/linux-rhel8-x86_64/gcc-12.1.0/openmpi-4.1.6-fa5w34i/bin/mpif90
FC=/projects/aue/hpc/deploy/b306e83c/linux-rhel8-x86_64/gcc-12.1.0/openmpi-4.1.6-fa5w34i/bin/mpif90
http_proxy=http://proxy.sandia.gov:80
https_proxy=http://proxy.sandia.gov:80
HTTPS_PROXY=http://proxy.sandia.gov:80
MODULES_CMD=/usr/share/Modules/libexec/modulecmd.tcl
MODULESHOME=/usr/share/Modules
MODULES_LMNOTUASKED=aue/binutils/2.43.1
MODULES_LMNOTUASKED_modshare=aue/binutils/2.43.1:1
MODULES_LMPREREQ=aue/gcc/12.1.0&aue/binutils
MODULES_LMPREREQ_modshare=aue/gcc/12.1.0&aue/binutils:1
MODULES_RUN_QUARANTINE=LD_LIBRARY_PATH LD_PRELOAD
MOOSE_JOBS=40
PATH_modshare=/ascldap/users/detorre/bin:1:/home/detorre/miniforge/condabin:1:/usr/sbin:1:/projects/aue/cee/builds/x86_64/rhel8/3b40855e/openmpi-4.1.6-gcc-12.1.0/install/linux-rhel8-x86_64/gcc-12.1.0/openmpi-4.1.6-cwth5vp/bin:1:/usr/bin:1:/projects/aue/cee/builds/x86_64/rhel8/ae2aa8f4/gcc-12.1.0/install/linux-rhel8-x86_64/gcc-10.3.0/gcc-12.1.0-xpyjv5s/bin:1:/projects/aue/cee/deploy/200a9402/linux-rhel8-x86_64/gcc-12.3.0/binutils-2.43.1-t7nqn3l/bin:1:/usr/local/sbin:1:/usr/share/Modules/bin:1:/usr/local/bin:1
PATH=/projects/aue/cee/builds/x86_64/rhel8/3b40855e/openmpi-4.1.6-gcc-12.1.0/install/linux-rhel8-x86_64/gcc-12.1.0/openmpi-4.1.6-cwth5vp/bin:/projects/aue/cee/builds/x86_64/rhel8/ae2aa8f4/gcc-12.1.0/install/linux-rhel8-x86_64/gcc-10.3.0/gcc-12.1.0-xpyjv5s/bin:/projects/aue/cee/deploy/200a9402/linux-rhel8-x86_64/gcc-12.3.0/binutils-2.43.1-t7nqn3l/bin:/ascldap/users/detorre/bin:/ascldap/users/detorre/bin:/home/detorre/miniforge/condabin:/usr/share/Modules/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin
REQUESTS_CA_BUNDLE=
SSL_CERT_FILE=
#############################################################################################
Compiler(s) (CC CXX FC F77 F90):

which $CC; /projects/aue/hpc/deploy/b306e83c/linux-rhel8-x86_64/gcc-12.1.0/openmpi-4.1.6-fa5w34i/bin/mpicc
$CC --version; gcc (Spack GCC) 12.1.0
$CC -show; /projects/aue/hpc/builds/x86_64/rhel8/ba17d7f2/gcc-12.1.0/install/linux-rhel8-x86_64/gcc-10.3.0/gcc-12.1.0-xpyjv5s/bin/gcc -I////////projects/aue/hpc/deploy/b306e83c/linux-rhel8-x86_64/gcc-12.1.0/openmpi-4.1.6-fa5w34i/include -pthread -L////////projects/aue/hpc/deploy/b306e83c/linux-rhel8-x86_64/gcc-12.1.0/openmpi-4.1.6-fa5w34i/lib -L/projects/aue/hpc/deploy/b306e83c/linux-rhel8-x86_64/gcc-12.1.0/hwloc-2.9.3-ylujs7h/lib -L/projects/aue/hpc/deploy/b306e83c/linux-rhel8-x86_64/gcc-12.1.0/libevent-2.1.12-oiai2u4/lib -Wl,-rpath,/projects/aue/hpc/builds/x86_64/rhel8/ba17d7f2/gcc-12.1.0/install/linux-rhel8-x86_64/gcc-10.3.0/gcc-12.1.0-xpyjv5s/lib/gcc/x86_64-pc-linux-gnu/12.1.0 -Wl,-rpath,/projects/aue/hpc/builds/x86_64/rhel8/ba17d7f2/gcc-12.1.0/install/linux-rhel8-x86_64/gcc-10.3.0/gcc-12.1.0-xpyjv5s/lib64 -Wl,-rpath -Wl,////////projects/aue/hpc/deploy/b306e83c/linux-rhel8-x86_64/gcc-12.1.0/openmpi-4.1.6-fa5w34i/lib -Wl,-rpath -Wl,/projects/aue/hpc/deploy/b306e83c/linux-rhel8-x86_64/gcc-12.1.0/hwloc-2.9.3-ylujs7h/lib -Wl,-rpath -Wl,/projects/aue/hpc/deploy/b306e83c/linux-rhel8-x86_64/gcc-12.1.0/libevent-2.1.12-oiai2u4/lib -lmpi

which $CXX; /projects/aue/hpc/deploy/b306e83c/linux-rhel8-x86_64/gcc-12.1.0/openmpi-4.1.6-fa5w34i/bin/mpicxx
$CXX --version; g++ (Spack GCC) 12.1.0
$CXX -show; /projects/aue/hpc/builds/x86_64/rhel8/ba17d7f2/gcc-12.1.0/install/linux-rhel8-x86_64/gcc-10.3.0/gcc-12.1.0-xpyjv5s/bin/g++ -I////////projects/aue/hpc/deploy/b306e83c/linux-rhel8-x86_64/gcc-12.1.0/openmpi-4.1.6-fa5w34i/include -pthread -L////////projects/aue/hpc/deploy/b306e83c/linux-rhel8-x86_64/gcc-12.1.0/openmpi-4.1.6-fa5w34i/lib -L/projects/aue/hpc/deploy/b306e83c/linux-rhel8-x86_64/gcc-12.1.0/hwloc-2.9.3-ylujs7h/lib -L/projects/aue/hpc/deploy/b306e83c/linux-rhel8-x86_64/gcc-12.1.0/libevent-2.1.12-oiai2u4/lib -Wl,-rpath,/projects/aue/hpc/builds/x86_64/rhel8/ba17d7f2/gcc-12.1.0/install/linux-rhel8-x86_64/gcc-10.3.0/gcc-12.1.0-xpyjv5s/lib/gcc/x86_64-pc-linux-gnu/12.1.0 -Wl,-rpath,/projects/aue/hpc/builds/x86_64/rhel8/ba17d7f2/gcc-12.1.0/install/linux-rhel8-x86_64/gcc-10.3.0/gcc-12.1.0-xpyjv5s/lib64 -Wl,-rpath -Wl,////////projects/aue/hpc/deploy/b306e83c/linux-rhel8-x86_64/gcc-12.1.0/openmpi-4.1.6-fa5w34i/lib -Wl,-rpath -Wl,/projects/aue/hpc/deploy/b306e83c/linux-rhel8-x86_64/gcc-12.1.0/hwloc-2.9.3-ylujs7h/lib -Wl,-rpath -Wl,/projects/aue/hpc/deploy/b306e83c/linux-rhel8-x86_64/gcc-12.1.0/libevent-2.1.12-oiai2u4/lib -lmpi

which $FC; /projects/aue/hpc/deploy/b306e83c/linux-rhel8-x86_64/gcc-12.1.0/openmpi-4.1.6-fa5w34i/bin/mpif90
$FC --version; GNU Fortran (Spack GCC) 12.1.0
$FC -show; /projects/aue/hpc/builds/x86_64/rhel8/ba17d7f2/gcc-12.1.0/install/linux-rhel8-x86_64/gcc-10.3.0/gcc-12.1.0-xpyjv5s/bin/gfortran -I////////projects/aue/hpc/deploy/b306e83c/linux-rhel8-x86_64/gcc-12.1.0/openmpi-4.1.6-fa5w34i/include -pthread -I////////projects/aue/hpc/deploy/b306e83c/linux-rhel8-x86_64/gcc-12.1.0/openmpi-4.1.6-fa5w34i/lib -L////////projects/aue/hpc/deploy/b306e83c/linux-rhel8-x86_64/gcc-12.1.0/openmpi-4.1.6-fa5w34i/lib -L/projects/aue/hpc/deploy/b306e83c/linux-rhel8-x86_64/gcc-12.1.0/hwloc-2.9.3-ylujs7h/lib -L/projects/aue/hpc/deploy/b306e83c/linux-rhel8-x86_64/gcc-12.1.0/libevent-2.1.12-oiai2u4/lib -Wl,-rpath,/projects/aue/hpc/builds/x86_64/rhel8/ba17d7f2/gcc-12.1.0/install/linux-rhel8-x86_64/gcc-10.3.0/gcc-12.1.0-xpyjv5s/lib/gcc/x86_64-pc-linux-gnu/12.1.0 -Wl,-rpath,/projects/aue/hpc/builds/x86_64/rhel8/ba17d7f2/gcc-12.1.0/install/linux-rhel8-x86_64/gcc-10.3.0/gcc-12.1.0-xpyjv5s/lib64 -Wl,-rpath -Wl,////////projects/aue/hpc/deploy/b306e83c/linux-rhel8-x86_64/gcc-12.1.0/openmpi-4.1.6-fa5w34i/lib -Wl,-rpath -Wl,/projects/aue/hpc/deploy/b306e83c/linux-rhel8-x86_64/gcc-12.1.0/hwloc-2.9.3-ylujs7h/lib -Wl,-rpath -Wl,/projects/aue/hpc/deploy/b306e83c/linux-rhel8-x86_64/gcc-12.1.0/libevent-2.1.12-oiai2u4/lib -lmpi_usempif08 -lmpi_usempi_ignore_tkr -lmpi_mpifh -lmpi

which $F77; /projects/aue/hpc/deploy/b306e83c/linux-rhel8-x86_64/gcc-12.1.0/openmpi-4.1.6-fa5w34i/bin/mpif77
$F77 --version; GNU Fortran (Spack GCC) 12.1.0
$F77 -show; /projects/aue/hpc/builds/x86_64/rhel8/ba17d7f2/gcc-12.1.0/install/linux-rhel8-x86_64/gcc-10.3.0/gcc-12.1.0-xpyjv5s/bin/gfortran -I////////projects/aue/hpc/deploy/b306e83c/linux-rhel8-x86_64/gcc-12.1.0/openmpi-4.1.6-fa5w34i/include -pthread -I////////projects/aue/hpc/deploy/b306e83c/linux-rhel8-x86_64/gcc-12.1.0/openmpi-4.1.6-fa5w34i/lib -L////////projects/aue/hpc/deploy/b306e83c/linux-rhel8-x86_64/gcc-12.1.0/openmpi-4.1.6-fa5w34i/lib -L/projects/aue/hpc/deploy/b306e83c/linux-rhel8-x86_64/gcc-12.1.0/hwloc-2.9.3-ylujs7h/lib -L/projects/aue/hpc/deploy/b306e83c/linux-rhel8-x86_64/gcc-12.1.0/libevent-2.1.12-oiai2u4/lib -Wl,-rpath,/projects/aue/hpc/builds/x86_64/rhel8/ba17d7f2/gcc-12.1.0/install/linux-rhel8-x86_64/gcc-10.3.0/gcc-12.1.0-xpyjv5s/lib/gcc/x86_64-pc-linux-gnu/12.1.0 -Wl,-rpath,/projects/aue/hpc/builds/x86_64/rhel8/ba17d7f2/gcc-12.1.0/install/linux-rhel8-x86_64/gcc-10.3.0/gcc-12.1.0-xpyjv5s/lib64 -Wl,-rpath -Wl,////////projects/aue/hpc/deploy/b306e83c/linux-rhel8-x86_64/gcc-12.1.0/openmpi-4.1.6-fa5w34i/lib -Wl,-rpath -Wl,/projects/aue/hpc/deploy/b306e83c/linux-rhel8-x86_64/gcc-12.1.0/hwloc-2.9.3-ylujs7h/lib -Wl,-rpath -Wl,/projects/aue/hpc/deploy/b306e83c/linux-rhel8-x86_64/gcc-12.1.0/libevent-2.1.12-oiai2u4/lib -lmpi_usempif08 -lmpi_usempi_ignore_tkr -lmpi_mpifh -lmpi

which $F90; /projects/aue/hpc/deploy/b306e83c/linux-rhel8-x86_64/gcc-12.1.0/openmpi-4.1.6-fa5w34i/bin/mpif90
$F90 --version; GNU Fortran (Spack GCC) 12.1.0
$F90 -show; /projects/aue/hpc/builds/x86_64/rhel8/ba17d7f2/gcc-12.1.0/install/linux-rhel8-x86_64/gcc-10.3.0/gcc-12.1.0-xpyjv5s/bin/gfortran -I////////projects/aue/hpc/deploy/b306e83c/linux-rhel8-x86_64/gcc-12.1.0/openmpi-4.1.6-fa5w34i/include -pthread -I////////projects/aue/hpc/deploy/b306e83c/linux-rhel8-x86_64/gcc-12.1.0/openmpi-4.1.6-fa5w34i/lib -L////////projects/aue/hpc/deploy/b306e83c/linux-rhel8-x86_64/gcc-12.1.0/openmpi-4.1.6-fa5w34i/lib -L/projects/aue/hpc/deploy/b306e83c/linux-rhel8-x86_64/gcc-12.1.0/hwloc-2.9.3-ylujs7h/lib -L/projects/aue/hpc/deploy/b306e83c/linux-rhel8-x86_64/gcc-12.1.0/libevent-2.1.12-oiai2u4/lib -Wl,-rpath,/projects/aue/hpc/builds/x86_64/rhel8/ba17d7f2/gcc-12.1.0/install/linux-rhel8-x86_64/gcc-10.3.0/gcc-12.1.0-xpyjv5s/lib/gcc/x86_64-pc-linux-gnu/12.1.0 -Wl,-rpath,/projects/aue/hpc/builds/x86_64/rhel8/ba17d7f2/gcc-12.1.0/install/linux-rhel8-x86_64/gcc-10.3.0/gcc-12.1.0-xpyjv5s/lib64 -Wl,-rpath -Wl,////////projects/aue/hpc/deploy/b306e83c/linux-rhel8-x86_64/gcc-12.1.0/openmpi-4.1.6-fa5w34i/lib -Wl,-rpath -Wl,/projects/aue/hpc/deploy/b306e83c/linux-rhel8-x86_64/gcc-12.1.0/hwloc-2.9.3-ylujs7h/lib -Wl,-rpath -Wl,/projects/aue/hpc/deploy/b306e83c/linux-rhel8-x86_64/gcc-12.1.0/libevent-2.1.12-oiai2u4/lib -lmpi_usempif08 -lmpi_usempi_ignore_tkr -lmpi_mpifh -lmpi

OK
#############################################################################################
Python Sanity Checks

/usr/bin/env python3 --version; (reporting as: Python 3.11.8) matches
which python3 python;

/ascldap/users/detorre/bin/python3 --version; == Python 3.11.8
/ascldap/users/detorre/bin/python --version; == Python 3.11.8

OK
#############################################################################################
Python Modules (TestHarness, run-ability)

OK


checks PASSED

The diagnostic script finds python fine.

It's possible I need to link the python libraries in the make file, but I would be surprised if so. I have successfully compiled this before without needing to do that.

[GiudGiud]

Hello

You do need the python development headers.
You need to load an additional module on HPC to provide those

[TheGreatCid]

I'm exporting to PATH
I've also tried directly linking with ADDITIONAL_CPP_FLAGS

I'm not on an INL hpc, so I don't have access to all those nice modules.

[TheGreatCid]

Should I be exporting to PYTHON_PATH instead?

[GiudGiud]

No it s the C++ compiler struggling
PYTHONPATH is for loading modules from python

Something like this, pointing to the right python should help
export CPPFLAGS="-I/usr/include/python3.11"

Do you have access to INL HPC?
I would log in there, load the modules, then run env to see all that got done

[TheGreatCid]

I was able to get it working the cppflags route you suggested.
Thanks for the help again!

and I don't have access to the INL HPC

[hugary1995]

It would be nice to decouple the capabilities python bindings from the C++ implementation. I guess another way to view this problem is to treat "pycapabilities" as a capability -- things should be allowed to compile/run without python development headers. @dschwen What do you think? It would be nice to allow users to skip compiling python bindings for capabilities. I believe that's currently the only thing that depends on python dev headers.