Failed to run update_and_rebuild_petsc.sh on SAWTOOTH #25623

wangcj05 · 2023-09-28T19:36:01Z

wangcj05
Sep 28, 2023

I tried to build moose on sawtooth with a clean clone. The following are the steps that I performed, but the it failed at the stage of update_and_rebuild_petsc.sh. I have included the screen output. Could anyone help me on this issue?

[wangc@sawtooth2 moose]$ module list

Currently Loaded Modules:
  1) use.moose                  4) patchelf/0.9-8.5.0          7) moose-dev/mvapich2-2.3.5
  2) gcc/8.4.0-gcc-4.8.5-jacd   5) git/2.25.0-gcc-9.2.0-v4mv
  3) mvapich2/2.3.5-gcc-8.4.0   6) mambaforge3

MOOSE:

commit e876aed746d1f399da8111abb29673de11df08d9 (HEAD -> next, origin/next, origin/HEAD)
Merge: 02d970f4fc fc1018dc11
Author: Casey Icenhour <casey.icenhour@inl.gov>
Date:   Thu Sep 28 08:35:51 2023 -0600

    Merge pull request #25607 from GiudGiud/PR_links

    Add links to moosedocs tagging

[wangc@sawtooth2 bison]$ module load use.moose moose-dev
[wangc@sawtooth2 bison]$ export MOOSE_JOBS=6 METHODS=opt
[wangc@sawtooth2 bison]$ cd ../moose/
[wangc@sawtooth2 moose]$ ls
aclocal.m4  configure        COPYRIGHT  gui          m4       README.md  tutorials
apptainer   configure.ac     docker_ci  large_media  modules  scripts    unit
CODEOWNERS  conf_vars.mk.in  examples   libmesh      petsc    stork
conda       COPYING          framework  LICENSE      python   test
[wangc@sawtooth2 moose]$ ./scripts/update_and_rebuild_petsc.sh
/home/wangc/projects/moose/scripts
Submodule 'petsc' (https://gitlab.com/petsc/petsc.git) registered for path 'petsc'
Cloning into '/home/wangc/projects/moose/petsc'...
Submodule path 'petsc': checked out '08c1f94318ee3b3ca1edb31fd14572b8083ef302'
INFO: Checking for HDF5...
INFO: HDF5 library not detected, opting to download via PETSc...
=============================================================================================
                         Configuring PETSc to compile on your system
=============================================================================================
=============================================================================================
                                     ***** WARNING *****
  Found environment variable: CC=mpicc. Ignoring it! Use "./configure CC=$CC" if you really
  want to use this value
=============================================================================================
=============================================================================================
                                     ***** WARNING *****
  Found environment variable: CXX=mpicxx. Ignoring it! Use "./configure CXX=$CXX" if you
  really want to use this value
=============================================================================================
=============================================================================================
                                     ***** WARNING *****
  Found environment variable: FC=mpif90. Ignoring it! Use "./configure FC=$FC" if you
  really want to use this value
=============================================================================================
=============================================================================================
                                     ***** WARNING *****
  Found environment variable: F77=mpif77. Ignoring it! Use "./configure F77=$F77" if you
  really want to use this value
=============================================================================================
=============================================================================================
                                     ***** WARNING *****
  Found environment variable: F90=mpif90. Ignoring it! Use "./configure F90=$F90" if you
  really want to use this value
=============================================================================================
=============================================================================================
                                     ***** WARNING *****
  Using default optimization C flags "-g -O". You might consider manually setting optimal
  optimization flags for your system with COPTFLAGS="optimization flags" see
  config/examples/arch-*-opt.py for examples
=============================================================================================
=============================================================================================
                                     ***** WARNING *****
  Using default Cxx optimization flags "-g -O". You might consider manually setting optimal
  optimization flags for your system with CXXOPTFLAGS="optimization flags" see
  config/examples/arch-*-opt.py for examples
=============================================================================================
=============================================================================================
                                     ***** WARNING *****
  Using default FORTRAN optimization flags "-g -O". You might consider manually setting
  optimal optimization flags for your system with FOPTFLAGS="optimization flags" see
  config/examples/arch-*-opt.py for examples
=============================================================================================
=============================================================================================
                                     ***** WARNING *****
  You have a version of GNU make older than 4.0. It will work, but may not support all the
  parallel testing options. You can install the latest GNU make with your package manager,
  such as Brew or MacPorts, or use the --download-make option to get the latest GNU make
=============================================================================================
=============================================================================================
  Trying to download
  https://support.hdfgroup.org/ftp/HDF5/releases/hdf5-1.12/hdf5-1.12.2/src/hdf5-1.12.2.tar.bz2
  for HDF5
=============================================================================================
=============================================================================================
                   Running configure on HDF5; this may take several minutes
=============================================================================================
=============================================================================================
                     Running make on HDF5; this may take several minutes
=============================================================================================
=============================================================================================
                 Running make install on HDF5; this may take several minutes
=============================================================================================
=============================================================================================
        Trying to download https://bitbucket.org/petsc/pkg-fblaslapack for FBLASLAPACK
=============================================================================================
=============================================================================================
                     Compiling FBLASLAPACK; this may take several minutes
=============================================================================================
=============================================================================================
            Trying to download https://bitbucket.org/petsc/pkg-metis.git for METIS
=============================================================================================
=============================================================================================
                 Configuring METIS with CMake; this may take several minutes
=============================================================================================
=============================================================================================
                Compiling and installing METIS; this may take several minutes
=============================================================================================
=============================================================================================
         Trying to download https://bitbucket.org/petsc/pkg-parmetis.git for PARMETIS
=============================================================================================
=============================================================================================
                Configuring PARMETIS with CMake; this may take several minutes
=============================================================================================
=============================================================================================
               Compiling and installing PARMETIS; this may take several minutes
=============================================================================================
=============================================================================================
          Trying to download https://gitlab.inria.fr/scotch/scotch.git for PTSCOTCH
=============================================================================================
=============================================================================================
                      Compiling PTScotch; this may take several minutes
=============================================================================================
=============================================================================================
                      Installing PTScotch; this may take several minutes
=============================================================================================
=============================================================================================
         Trying to download https://github.com/xiaoyeli/superlu_dist for SUPERLU_DIST
=============================================================================================
=============================================================================================
              Configuring SUPERLU_DIST with CMake; this may take several minutes
=============================================================================================
=============================================================================================
             Compiling and installing SUPERLU_DIST; this may take several minutes
=============================================================================================
=============================================================================================
      Trying to download https://github.com/Reference-ScaLAPACK/scalapack for SCALAPACK
=============================================================================================
=============================================================================================
               Configuring SCALAPACK with CMake; this may take several minutes
=============================================================================================
=============================================================================================
              Compiling and installing SCALAPACK; this may take several minutes
=============================================================================================
=============================================================================================
            Trying to download https://github.com/pghysels/STRUMPACK for STRUMPACK
=============================================================================================
=============================================================================================
               Configuring STRUMPACK with CMake; this may take several minutes
=============================================================================================
=============================================================================================
              Compiling and installing STRUMPACK; this may take several minutes
=============================================================================================

*********************************************************************************************
           UNABLE to CONFIGURE with GIVEN OPTIONS (see configure.log for details):
---------------------------------------------------------------------------------------------
                               Error running make on  STRUMPACK
*********************************************************************************************

There was an error. Exiting...

GiudGiud · 2023-09-28T19:39:43Z

GiudGiud
Sep 28, 2023
Collaborator

Hello

what does which make return ? seems you dont have a cmake module loaded.
the configure.log file it made will have more information

28 replies

wangcj05 Oct 5, 2023
Author

@milljm That is the test for the latest moose, there is only CSVDIFF for test "vectorpostprocessors/work_balance.work_balance/replicated", and there is no additional information that I can find.

wangcj05 Oct 5, 2023
Author

In addition, I have tested BISON with the current moose version, and the following are some error messages for BISON tests:

tools/inputwizard/tests.parse_normal_good: Working Directory: /home/wangc/projects/bison/tools/inputwizard/tests
tools/inputwizard/tests.parse_normal_good: Running command: ./create_parsed.py normal_good normal_parsed_good.i
tools/inputwizard/tests.parse_normal_good: Traceback (most recent call last):
tools/inputwizard/tests.parse_normal_good:   File "/home/wangc/projects/bison/tools/inputwizard/tests/./create_parsed.py", line 16, in <module>
tools/inputwizard/tests.parse_normal_good:     parsed_str = subprocess.check_output(["node", "parser.js", input_file]).decode()
tools/inputwizard/tests.parse_normal_good:   File "/apps/moose/stack/mamba3/lib/python3.10/subprocess.py", line 421, in check_output
tools/inputwizard/tests.parse_normal_good:     return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
tools/inputwizard/tests.parse_normal_good:   File "/apps/moose/stack/mamba3/lib/python3.10/subprocess.py", line 503, in run
tools/inputwizard/tests.parse_normal_good:     with Popen(*popenargs, **kwargs) as process:
tools/inputwizard/tests.parse_normal_good:   File "/apps/moose/stack/mamba3/lib/python3.10/subprocess.py", line 971, in __init__
tools/inputwizard/tests.parse_normal_good:     self._execute_child(args, executable, preexec_fn, close_fds,
tools/inputwizard/tests.parse_normal_good:   File "/apps/moose/stack/mamba3/lib/python3.10/subprocess.py", line 1847, in _execute_child
tools/inputwizard/tests.parse_normal_good:     raise child_exception_type(errno_num, err_msg, err_filename)
tools/inputwizard/tests.parse_normal_good: FileNotFoundError: [Errno 2] No such file or directory: 'node'
tools/inputwizard/tests.parse_normal_good:
tools/inputwizard/tests.parse_normal_good: ################################################################################
tools/inputwizard/tests.parse_normal_good: Tester failed, reason: CODE 1
tools/inputwizard/tests.parse_normal_good:
tools/inputwizard/tests.parse_normal_good .................................................... FAILED (CODE 1)
tools/inputwizard/tests.check_normal_good_parsed ................................... [skipped dependency] SKIP
test:nuclearMaterialUN.ad_nitride_action .................................................................. OK
tools/inputwizard/tests.parse_normal_good: Working Directory: /home/wangc/projects/bison/tools/inputwizard/tests
tools/inputwizard/tests.parse_normal_good: Running command: ./create_parsed.py normal_good normal_parsed_good.i
tools/inputwizard/tests.parse_normal_good: Traceback (most recent call last):
tools/inputwizard/tests.parse_normal_good:   File "/home/wangc/projects/bison/tools/inputwizard/tests/./create_parsed.py", line 16, in <module>
tools/inputwizard/tests.parse_normal_good:     parsed_str = subprocess.check_output(["node", "parser.js", input_file]).decode()
tools/inputwizard/tests.parse_normal_good:   File "/apps/moose/stack/mamba3/lib/python3.10/subprocess.py", line 421, in check_output
tools/inputwizard/tests.parse_normal_good:     return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
tools/inputwizard/tests.parse_normal_good:   File "/apps/moose/stack/mamba3/lib/python3.10/subprocess.py", line 503, in run
tools/inputwizard/tests.parse_normal_good:     with Popen(*popenargs, **kwargs) as process:
tools/inputwizard/tests.parse_normal_good:   File "/apps/moose/stack/mamba3/lib/python3.10/subprocess.py", line 971, in __init__
tools/inputwizard/tests.parse_normal_good:     self._execute_child(args, executable, preexec_fn, close_fds,
tools/inputwizard/tests.parse_normal_good:   File "/apps/moose/stack/mamba3/lib/python3.10/subprocess.py", line 1847, in _execute_child
tools/inputwizard/tests.parse_normal_good:     raise child_exception_type(errno_num, err_msg, err_filename)
tools/inputwizard/tests.parse_normal_good: FileNotFoundError: [Errno 2] No such file or directory: 'node'
tools/inputwizard/tests.parse_normal_good:
tools/inputwizard/tests.parse_normal_good: ################################################################################
tools/inputwizard/tests.parse_normal_good: Tester failed, reason: CODE 1
tools/inputwizard/tests.parse_normal_good:
tools/inputwizard/tests.parse_normal_good .................................................... FAILED (CODE 1)

milljm Oct 5, 2023

I'll have to see what ./create_parsed.py does. It most likely is trying to import another python package that I need to add.

Er... that python script doesn't do anything abnormal. I'll have to pull BISON master, see whats up. I did this earlier and all the tests passed for me. But I was operating on a different version at the time.

GiudGiud Oct 6, 2023
Collaborator

I think this is trying to run some javascript.
mamba install nodejs will install the package you need to run this test.

I ll make a PR to bison to require it

milljm Oct 9, 2023

@GiudGiud thanks!

Okay... installed. On both Sawtooth and Lemhi...

wangcj05 · 2023-10-16T18:58:26Z

wangcj05
Oct 16, 2023
Author

@milljm I have tried to rebuild moose on Lemhi, but I got the following error message:

module load use.moose moose-dev
(base) [wangc@lemhi1 bison]$ module list

Currently Loaded Modules:
  1) flex/2.6.4-gcc-11.2.0-wvjp    4) mvapich2/2.3.7-1                     7) use.moose
  2) bison/3.7.6-gcc-11.2.0-aysl   5) cmake/3.21.3-gcc-9.4.0-d22d          8) mamba3
  3) gcc/9.4.0-gcc-8.4.1-57pg      6) patchelf/0.13-oneapi-2021.4.0-uxhq   9) moose-dev/2023-10-05

run the following without error:

cd ~/cluster_name/projects/moose/scripts
export MOOSE_JOBS=6 METHODS=opt
./update_and_rebuild_petsc.sh
./update_and_rebuild_libmesh.sh
./update_and_rebuild_wasp.sh

cd test
make -j 6

Then got the following error message:

Linking Library /home/wangc/projects/moose/framework/libmoose-opt.la...
/usr/bin/grep: /apps/local/mvapich2/2.3.7-1-gcc-8.4.0/lib/libmpicxx.la: No such file or directory
/usr/bin/sed: can't read /apps/local/mvapich2/2.3.7-1-gcc-8.4.0/lib/libmpicxx.la: No such file or directory
libtool:   error: '/apps/local/mvapich2/2.3.7-1-gcc-8.4.0/lib/libmpicxx.la' is not a valid libtool archive
make: *** [/home/wangc/projects/moose/framework/moose.mk:394: /home/wangc/projects/moose/framework/libmoose-opt.la] Error 1
make: *** Waiting for unfinished jobs....

@milljm Do you have any suggest how to fix this issue?

2 replies

milljm Oct 16, 2023

I think you have a stale build:

$ ls /apps/local/mvapich2/2.3.7-1-gcc-8.4.0/lib/libmpicxx.la:
ls: cannot access '/apps/local/mvapich2/2.3.7-1-gcc-8.4.0/lib/libmpicxx.la:': No such file or directory

That directory does not (if ever) exists. Can you try a clean build?

wangcj05 Oct 16, 2023
Author

Thanks, it works. @milljm

Failed to run update_and_rebuild_petsc.sh on SAWTOOTH #25623

Uh oh!

wangcj05 Sep 28, 2023

Replies: 2 comments · 30 replies

Uh oh!

GiudGiud Sep 28, 2023 Collaborator

Uh oh!

wangcj05 Oct 5, 2023 Author

Uh oh!

wangcj05 Oct 5, 2023 Author

Uh oh!

Uh oh!

milljm Oct 5, 2023

Uh oh!

GiudGiud Oct 6, 2023 Collaborator

Uh oh!

milljm Oct 9, 2023

Uh oh!

wangcj05 Oct 16, 2023 Author

Uh oh!

milljm Oct 16, 2023

Uh oh!

wangcj05 Oct 16, 2023 Author

wangcj05
Sep 28, 2023

Replies: 2 comments 30 replies

GiudGiud
Sep 28, 2023
Collaborator

wangcj05 Oct 5, 2023
Author

wangcj05 Oct 5, 2023
Author

GiudGiud Oct 6, 2023
Collaborator

wangcj05
Oct 16, 2023
Author

wangcj05 Oct 16, 2023
Author