Before running the validation tests, make sure you can run the unit tests and the parallel unit tests.
The script here can be used to run test problems in folders within ../../examples/ and ../../examples/nthmp/ and ../../examples/dispersive/ via:
Rscript run_validations.R
This takes about 70 min to run on my desktop (as of October 2022). It searches the test problem directories for scripts named run_model.sh, and runs them inside their parent directories.
Most tests use openmp parallelism alone, but a significant minority also use MPI. The commands controlling openmp and MPI (i.e. number of threads, ranks, etc) are determined by the script ../../src/test_run_commands.sh. For easier portability between machines the latter script usually just sources some machine specific script, such as this example.
For each problem the run_model.sh scripts print multiple PASS (or FAIL) statements and, in most cases, make figures in the test problem directories. FAIL statements indicate the test criteria were not satisfied.
Aside from genuine bugs, FAIL statements may reflect the ad-hoc nature of the test criteria. For example, a test problem might specify an error tolerance of 0.01, but depending on the case, somewhat larger errors might also be acceptable. Typically there is no objective error tolerance. For this reason FAIL statements may arise when testing new flow algorithms, or when porting the code to a different compiler, even if there are not genuine problems. Thus user judgement is required to interpret FAILs -- are they genuine problems, or simply unimportant violations of the test criteria? The test problem figures will likely help to judge this.
The outputs should be similar to below, but may be interspersed with messages from the compiler and/or R, and nowadays include a few extra test problems. Model run files and figures are created in the individual problem directories.
The time taken includes (for each problem) compilation, running the model, and post-processing. It can vary greatly depending on your machine and compiler. This example used the ifort compiler on a half-node of NCI's Gadi machine. While ifort is a good compiler the compilation process is much slower than gfortran, and for the smaller test problems compilation can dominates the recorded time.
gxd547@gadi-cpu-clx-0792 validation_tests]$ Rscript run_validations.R
[1] ""
[1] "Testing ../../examples/circular_island/run_model.sh"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "Time taken (build, run, plot) : 2.1107 mins"
[1] ""
[1] "Testing ../../examples/dambreak/run_model.sh"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "Time taken (build, run, plot) : 1.591035 mins"
[1] ""
[1] "Testing ../../examples/generic_example/run_model.sh"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "Time taken (build, run, plot) : 3.801955 mins"
[1] ""
[1] "Testing ../../examples/isolated_building/run_model.sh"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "Time taken (build, run, plot) : 1.720887 mins"
[1] ""
[1] "Testing ../../examples/landslide_tsunami/run_model.sh"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "Time taken (build, run, plot) : 2.221726 mins"
[1] ""
[1] "Testing ../../examples/merewether/run_model.sh"
[1] "PASS"
[1] "PASS"
[1] "Time taken (build, run, plot) : 1.569787 mins"
[1] ""
[1] "Testing ../../examples/nesting_plane_wave/run_model.sh"
[1] "PASS" "0.00846102363336937"
[1] "PASS" "0.00834473169304869"
[1] "PASS" "0.000713169705934801"
[1] "PASS" "0.000713264826770574"
[1] "Time taken (build, run, plot) : 1.895894 mins"
[1] ""
[1] "Testing ../../examples/overbank_flow/run_model.sh"
[1] "PASS"
[1] "PASS"
[1] "Time taken (build, run, plot) : 1.722191 mins"
[1] ""
[1] "Testing ../../examples/parabolic_canal/run_model.sh"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "Time taken (build, run, plot) : 1.604347 mins"
[1] ""
[1] "Testing ../../examples/paraboloid_bowl/run_model.sh"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "Time taken (build, run, plot) : 3.337309 mins"
[1] ""
[1] "Testing ../../examples/periodic_convergence/run_model.sh"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "Time taken (build, run, plot) : 1.934422 mins"
[1] ""
[1] "Testing ../../examples/radial_dam_break/run_model.sh"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "Time taken (build, run, plot) : 1.39817 mins"
[1] ""
[1] "Testing ../../examples/spherical_box/run_model.sh"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "Time taken (build, run, plot) : 1.580632 mins"
[1] ""
[1] "Testing ../../examples/uniform_channel/run_model.sh"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "Time taken (build, run, plot) : 1.644815 mins"
[1] ""
[1] "Testing ../../examples/uniform_slope/run_model.sh"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "Time taken (build, run, plot) : 1.81154 mins"
[1] ""
[1] "Testing ../../examples/uniform_slope_shallow/run_model.sh"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "Time taken (build, run, plot) : 1.392999 mins"
[1] ""
[1] "Testing ../../examples/nthmp/BP01/run_model.sh"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "Time taken (build, run, plot) : 2.298866 mins"
[1] ""
[1] "Testing ../../examples/nthmp/BP02_and_BP05/run_model.sh"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "Time taken (build, run, plot) : 4.344378 mins"
[1] ""
[1] "Testing ../../examples/nthmp/BP04/run_model.sh"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "Time taken (build, run, plot) : 4.650797 mins"
[1] ""
[1] "Testing ../../examples/nthmp/BP06/run_model.sh"
[1] "PASS" "0.16645523261498"
[1] "PASS" "0.0157782348437547"
[1] "PASS" "0.0913552253347327"
[1] "PASS" "0.0639461467714567"
[1] "Time taken (build, run, plot) : 2.140898 mins"
[1] ""
[1] "Testing ../../examples/nthmp/BP07/run_model.sh"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "Time taken (build, run, plot) : 1.93509 mins"
[1] ""
[1] "Testing ../../examples/nthmp/BP09/run_model.sh"
[1] "PASS" "26.2152252197266"
[1] "PASS" "4.372615814209"
[1] "PASS" "26.2152252197266"
[1] "PASS" "4.382375717163"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS" "26.1336212158203"
[1] "PASS" "4.054662704468"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "Time taken (build, run, plot) : 12.32333 mins"
[1] ""
[1] "Testing ../../examples/nthmp/Hilo_Tohoku_tsunami/run_model.sh"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "Time taken (build, run, plot) : 5.820813 mins"
[1] ""
[1] "Testing ../../examples/nthmp/Seaside_OSU_model/run_model.sh"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "Time taken (build, run, plot) : 2.19928 mins"
[1] ""
[1] "Testing ../../examples/nthmp/Submerged_Island_Lab/run_model.sh"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "Time taken (build, run, plot) : 2.897971 mins"
[1] ""
[1] "Testing ../../examples/nthmp/Tauranga_harbour_Tohoku_tsunami/run_model.sh"
[1] "PASS" "0.00617253174644793"
[1] "PASS" "0.0236844158991954"
[1] "PASS" "0.0257962528028354"
[1] "PASS" "0.0489072626700319"
[1] "PASS" "0.00617253174644793"
[1] "PASS" "0.0236844158991954"
[1] "PASS" "0.0257962528028354"
[1] "PASS" "0.0489072626700319"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "PASS"
[1] "Time taken (build, run, plot) : 4.997875 mins"