Bcs imp b4 b

[kestonsmith-noaa]

Pull Request Summary

Communication is added after boundary conditions are applied in PDLIB_JACOBI_GAUSS_SEIDEL_BLOCK. Change is needed for reproducability in simulations with boundary conditions.

Description

Communication is added after boundary conditions are applied in PDLIB_JACOBI_GAUSS_SEIDEL_BLOCK. This PR fixes a bug affecting reproducability across different numbers of MPI processes when boundary conditions are applied in the unstructured mesh implicit time stepping algorithm.

Additionally a test case is added to ww3_tp2.17 to test with ice forcing in addition to wind, currents, water level, and boundary forcing. This test uses a new water level file, regtests/ww3_tp2.17/input_ice/levelNoNaN.nc, rather than regtests/ww3_tp2.17/input_ice/level.nc which has NaN values within bay. The presence of NaN in simulations using regtests/ww3_tp2.17/input_ice/level.nc appears to cause non reproducability when different number of MPI processes are used.

Issue(s) addressed

#1537 (non bit for bit reproducability)
#1533 (ice forcing added to regtests/ww3_tp2.17 )

Commit Message

Communication is added after boundary conditions are applied in PDLIB_JACOBI_GAUSS_SEIDEL_BLOCK to enable bit for bit reproducability across different numbers of MPI processes.

Check list

• Branch is up to date with the authoritative repository (NOAA-EMC) develop branch.
• Checked the checklist for a developer submitting to develop.
• If a version number update is required, checked the updating version number checklist.
• If a new feature was added, a regression test for testing the new feature is added.

Testing

• How were these changes tested?
• regresion tests and modified version of regtest ww3_tp2.17
• Are the changes covered by regression tests? (If not, why? Do new tests need to be added?)
• Have the matrix regression tests been run (if yes, please note HPC and compiler)?
• yes, ursa intel
• Please indicate the expected changes in the regression test output, (Note the list of known non-identical tests.)
• It is possible that there will be changes in regtest ww3_tp2.17 due to bug fix
  Changes may occur in regtest ww3_tp2.17 due to added communication of boundary condition.
• Please provide the summary output of matrix.comp (matrix.Diff.txt, matrixCompFull.txt and matrixCompSummary.txt):

matrixCompSummary.txt

[mingchen-NOAA]

@kestonsmith-noaa Thank you! I will review it.