JOSS article review #2

[sthavishtha]

Please find my comments regarding installation of the package here. I was attempting to install the package on the bridges2 HPC cluster (https://www.psc.edu/resources/bridges-2/user-guide/).

The installation instructions need to be significantly improved. I had to tweak the instructions to install the package (see my comments and files below). From a user's perspective, if I struggle to install a package for several hours, I would not use the package at all.

1. As suggested earlier, a containerized solution with all dependencies should be provided which would remove all installation issues.
2. Whenever there was an error during the Spack installation of the modules, I had to follow the steps below. Please update the instructions to reflect this.

cd /spack/var/spack/environments/vertex/
rm spack.lock
make changes to the spack.yaml file here (this one is a copy of the yaml file placed elsewhere), and repeat the installation

3. I had to modify the yaml file spack-trilinos.yaml to install the spack modules successfully. The installation instructions should be user-friendly to reflect the version of the spack modules.

4. Please update the instructions to direct users to https://code.ornl.gov/mxd/vertex-cfd.git for the regression tests and examples folders. Please direct the user to mxd/update branch of this repository, as other branches have some missing files and throw installation errors.

5. Installation (CPU) using make -j install (just before testing) gives the error below. Looks like some files are missing in the source code. My configurations scripts are vertex-config.sh and vertex-env-cpu.sh

[sbhopala@r218 build]$ make -j install
[  0%] Building CXX object src/utils/CMakeFiles/Utils.dir/VertexCFD_Utils_BCStrategy_Defines.cpp.o
[  0%] Building CXX object src/test_harness/unit_test/CMakeFiles/VertexCFD_KokkosMPI_test_SERIAL.dir/tstKokkosMPI.cpp.o
[  0%] Building CXX object src/utils/CMakeFiles/Utils.dir/VertexCFD_Utils_MagneticDim.cpp.o
[  1%] Building CXX object src/test_harness/unit_test/CMakeFiles/VertexCFD_EvaluatorTestHarness_test_SERIAL.dir/tstEvaluatorTestHarness.cpp.o
[  1%] Building CXX object src/utils/CMakeFiles/Utils.dir/VertexCFD_Utils_MagneticLayout.cpp.o
[  1%] Building CXX object src/utils/CMakeFiles/Utils.dir/VertexCFD_Utils_PhaseIndex.cpp.o
[  1%] Building CXX object src/test_harness/unit_test/CMakeFiles/VertexCFD_EvaluatorTestHarness_test_SERIAL.dir/__/test_main.cpp.o
[  1%] Building CXX object src/utils/CMakeFiles/Utils.dir/VertexCFD_Utils_Version.cpp.o
[  1%] Building CXX object src/utils/CMakeFiles/Utils.dir/VertexCFD_Utils_PhaseLayout.cpp.o
[  1%] Building CXX object src/utils/CMakeFiles/Utils.dir/VertexCFD_Utils_VelocityDim.cpp.o
[  1%] Building CXX object src/utils/CMakeFiles/Utils.dir/VertexCFD_Utils_VelocityLayout.cpp.o
[  1%] Building CXX object src/test_harness/unit_test/CMakeFiles/VertexCFD_KokkosMPI_test_SERIAL.dir/__/test_main.cpp.o

/ocean/projects/ees250012p/sbhopala/joss/trilinos/VERTEX-CFD/VERTEX-CFD/src/utils/VertexCFD_Utils_BCStrategy_Defines.cpp:4:10: fatal error: 'Panzer_BC.hpp' file not found
#include <Panzer_BC.hpp>
         ^~~~~~~~~~~~~~~

In file included from /ocean/projects/ees250012p/sbhopala/joss/trilinos/VERTEX-CFD/VERTEX-CFD/src/utils/VertexCFD_Utils_PhaseIndex.cpp:1:
/ocean/projects/ees250012p/sbhopala/joss/trilinos/VERTEX-CFD/VERTEX-CFD/src/utils/VertexCFD_Utils_PhaseIndex.hpp:4:10: fatal error: 'Phalanx_ExtentTraits.hpp' file not found
#include <Phalanx_ExtentTraits.hpp>
         ^~~~~~~~~~~~~~~~~~~~~~~~~~

In file included from /ocean/projects/ees250012p/sbhopala/joss/trilinos/VERTEX-CFD/VERTEX-CFD/src/utils/VertexCFD_Utils_VelocityDim.cpp:1:
/ocean/projects/ees250012p/sbhopala/joss/trilinos/VERTEX-CFD/VERTEX-CFD/src/utils/VertexCFD_Utils_VelocityDim.hpp:4:10: fatal error: 'Phalanx_ExtentTraits.hpp' file not found
#include <Phalanx_ExtentTraits.hpp>
         ^~~~~~~~~~~~~~~~~~~~~~~~~~

In file included from /ocean/projects/ees250012p/sbhopala/joss/trilinos/VERTEX-CFD/VERTEX-CFD/src/utils/VertexCFD_Utils_PhaseLayout.cpp:1:
/ocean/projects/ees250012p/sbhopala/joss/trilinos/VERTEX-CFD/VERTEX-CFD/src/utils/VertexCFD_Utils_PhaseLayout.hpp:4:10: fatal error: 'Phalanx_DataLayout.hpp' file not found
#include <Phalanx_DataLayout.hpp>
         ^~~~~~~~~~~~~~~~~~~~~~~~

In file included from /ocean/projects/ees250012p/sbhopala/joss/trilinos/VERTEX-CFD/VERTEX-CFD/src/utils/VertexCFD_Utils_MagneticLayout.cpp:1:
/ocean/projects/ees250012p/sbhopala/joss/trilinos/VERTEX-CFD/VERTEX-CFD/src/utils/VertexCFD_Utils_MagneticLayout.hpp:4:10: fatal error: 'Phalanx_DataLayout.hpp' file not found
#include <Phalanx_DataLayout.hpp>
         ^~~~~~~~~~~~~~~~~~~~~~~~

In file included from /ocean/projects/ees250012p/sbhopala/joss/trilinos/VERTEX-CFD/VERTEX-CFD/src/utils/VertexCFD_Utils_MagneticDim.cpp:1:
/ocean/projects/ees250012p/sbhopala/joss/trilinos/VERTEX-CFD/VERTEX-CFD/src/utils/VertexCFD_Utils_MagneticDim.hpp:4:10: fatal error: 'Phalanx_ExtentTraits.hpp' file not found
#include <Phalanx_ExtentTraits.hpp>
         ^~~~~~~~~~~~~~~~~~~~~~~~~~

make*** [src/utils/CMakeFiles/Utils.dir/build.make:149: src/utils/CMakeFiles/Utils.dir/VertexCFD_Utils_VelocityDim.cpp.o] Error 1
make*** Waiting for unfinished jobs....
make*** [src/utils/CMakeFiles/Utils.dir/build.make:121: src/utils/CMakeFiles/Utils.dir/VertexCFD_Utils_PhaseIndex.cpp.o] Error 1
make*** [src/utils/CMakeFiles/Utils.dir/build.make:93: src/utils/CMakeFiles/Utils.dir/VertexCFD_Utils_MagneticDim.cpp.o] Error 1

/ocean/projects/ees250012p/sbhopala/joss/trilinos/VERTEX-CFD/VERTEX-CFD/src/utils/VertexCFD_Utils_VelocityLayout.cpp:1:10: fatal error: 'Phalanx_DataLayout.hpp' file not found
#include <Phalanx_DataLayout.hpp>
         ^~~~~~~~~~~~~~~~~~~~~~~~

/ocean/projects/ees250012p/sbhopala/joss/trilinos/VERTEX-CFD/VERTEX-CFD/src/test_harness/unit_test/tstKokkosMPI.cpp:1:10: fatal error: 'Phalanx_KokkosDeviceTypes.hpp' file not found
#include <Phalanx_KokkosDeviceTypes.hpp>
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

/ocean/projects/ees250012p/sbhopala/joss/trilinos/VERTEX-CFD/VERTEX-CFD/src/test_harness/unit_test/tstEvaluatorTestHarness.cpp:1:
fatal error: 'Panzer_BasisIRLayout.hpp' file not found
#include <Panzer_BasisIRLayout.hpp>
         ^~~~~~~~~~~~~~~~~~~~~~~~~~

make*** [CMakeFiles/Makefile2:2662: src/utils/CMakeFiles/Utils.dir/all] Error 2
make*** [CMakeFiles/Makefile2:2630: src/test_harness/unit_test/CMakeFiles/VertexCFD_EvaluatorTestHarness_test_SERIAL.dir/all] Error 2
make*** [CMakeFiles/Makefile2:2598: src/test_harness/unit_test/CMakeFiles/VertexCFD_KokkosMPI_test_SERIAL.dir/all] Error 2
make: *** [Makefile:146: all] Error 2