Fix potential divide by zero in ww3; Add timers to WW3 cap (was #2086); Allow MOM6 to write non-interval restarts (was #2099)

[DeniseWorthen]

Commit Queue Requirements:

• Fill out all sections of this template.
• All sub component pull requests have been reviewed by their code managers.
• Run the full RT suite (compared to current baselines) on either Hera/Derecho/Hercules AND have committed the log to my PR branch.
• Add list of all failed regression tests in "Regression Tests" section.

PR Information

• closes Resolve division by 0 when meshcap is run with WW3-CICE coupling; Add timers and overwrite capability for mod_def timesteps to meshcap  NOAA-EMC/WW3#1163
• closes Add non-interval restart capability NOAA-EMC/MOM6#126
• needed by Scalability of the Coupled Model #1367

Description

Prevent divide-by-zero when using WAV-ICE coupling in UWM

Update WW3 to enable run timers in the cap. As with the CICE and MOM6 caps, the feature is enabled by setting
RunTimeLog = true in the ALLCOMP_attributes.

The timer log will be log.ww3.timer which will show

20210322   60000   21600 WW3 InitializeRealize time: 0.7798858
20210322   60000   21600 WW3 ModelAdvance time since last step:  4.489980
20210322   61200   22320 WW3 ModelAdvance time:  2.570183
20210322   61200   22320 WW3 ModelAdvance time since last step:  8.023816
20210322   62400   23040 WW3 ModelAdvance time:  1.164186
20210322   62400   23040 WW3 ModelAdvance time since last step: 0.1100351E-01

Also adds the capability of reading a WW3 configuration attribute to allow WW3 timesteps to over-write timesteps saved in pre-created mod_def file. This allows the same mod_def file to be used for either inner or outer loop coupling. To test the feature, a version of mod_def.mx100 was created with the "outer loop" timesteps:

$ if outerloop: 3600 900 1800 30
 720 360 360 30

This mod_def was then used for the cpld_control RTs, but adding the WAV attribute:

# WAV #
WAV_model:                      ww3
...(snip)
  dt_in = 720,360,360,30
::

All tests passed.

See also #2099 for description for non-interval restarts in MOM6.

Commit Message

fix potential divide by zero in WW3 appendtail routine
add timers to WW3 mesh cap
add non-interval restart capability to MOM6

Priority

• Critical Bugfix (This PR contains a critical bug fix and should be prioritized.)
• High (This PR contains a feature or fix needed for a time-sensitive project (eg, retrospectives, implementations))
• Normal Required by PR Add regression testing for wave-sea ice coupling #2072

Blocking Dependencies

Git Issues Fixed By This PR

• fixes Allow timer log functionality in the meshcap NOAA-EMC/WW3#1162
• fixes Appendtail (called by w3fld1 and w3fld2) can get NaNs from variable "avg" NOAA-EMC/WW3#1159
• fixes Enable controlling MOM6 ocean restart files with a more flexible approach #1976

Changes

Subcomponent (with links)

• AQM
• CDEPS
• CICE
• CMEPS
• CMakeModules
• FV3
• GOCART
• HYCOM
• MOM6
• NOAHMP
• WW3
• stochastic_physics
• none

Input data

• No changes are expected to input data.
• Changes are expected to input data:
  • New input data.
  • Updated input data.

Regression Tests:

• No changes are expected to any regression test.
• Changes are expected to the following tests:

FAILED REGRESSION TESTS

Note: Prior to the cpld GNU tests being disabled on Hercules, the debug tests using ww3 failed using that later GNU compiler.

Libraries

• Not Needed
• Needed
  • Create separate issue in JCSDA/spack-stack asking for update to library. Include library name, library version.
  • Add issue link from JCSDA/spack-stack following this item

Testing Log:

• RDHPCS
  • Hera
  • Orion
  • Hercules
  • Jet
  • Gaea
  • Derecho
• WCOSS2
  • Dogwood/Cactus
  • Acorn
• CI
  • Completed
• opnReqTest
  • N/A
  • Log attached to comment

[DeniseWorthen]

@JessicaMeixner-NOAA I know there was an email (?) about combining the two WW3 PRs but I can't find it now. I want to be sure it is oK to combine NOAA-EMC/WW3#1163 and NOAA-EMC/WW3#1164.

[JessicaMeixner-NOAA]

@DeniseWorthen Yes - feel free to combine!

[jiandewang]

@DeniseWorthen that's totally fine. I will push your work back to main repo. once this is done.

[DeniseWorthen]

@jkbk2004 We also have the option of combining in the MOM6 restart PR #2099 since it also does not change baselines.

[jkbk2004]

@DeniseWorthen we need more time on #2066 and #2059. Please, feel free to combine with #2099 to this pr. Whenever ready, we can start working on this pr. @zach1221 @FernandoAndrade-NOAA @BrianCurtis-NOAA FYI

[FernandoAndrade-NOAA]

@jkbk2004 @zach1221 FYI I've updated Jet's rocoto version in rt.sh to 1.3.6 after the default was upgraded following the recent maintenance. Otherwise a not found error would be triggered.

[zach1221]

Ok, regression is should be complete, since we're skipping Gaea and WCOSS. We can begin the merge process with WW3 #1163 and MOM6 #126 .

[BrianCurtis-NOAA]

Ok, regression is should be complete, since we're skipping Gaea and WCOSS. We can begin the merge process with WW3 #1163 and MOM6 #126 .

@zach1221 WCOSS2 is almost done. It opened back up this AM and i'll have those logs in a little bit.

[zach1221]

Ok, regression is should be complete, since we're skipping Gaea and WCOSS. We can begin the merge process with WW3 #1163 and MOM6 #126 .

@zach1221 WCOSS2 is almost done. It opened back up this AM and i'll have those logs in a little bit.

Oh, ok. No problem. Thanks, Brian.

[BrianCurtis-NOAA]

@zach1221 all good to start merge process

[zach1221]

@DeniseWorthen WW3 sub-pr has been merged already. Hash: NOAA-EMC/WW3@4ffc47e

[zach1221]

MOM6 is merged as well. Hash: NOAA-EMC/MOM6@c8a7325