NPP_BURNEDOFF doesn't seem to be restarting correctly

[ekluzek]

Erik Kluzek < erik > - 2016-03-02 15:28:18 -0700
Bugzilla Id: 2292
Bugzilla CC: dlawren, mvertens, rfisher, sacks,

This issue doesn't show up in CLM standalone testing (possibly just by random chance), but Mariana found a case where NPP_BURNEDOFF is different for a B1850 case. Talking this over with Rosie, this is a kludge that we don't expect to happen often. It does pass balance checks, and as noted it passes the standard offline CLM testing. She's not sure why it passes balance checks, but not a restart test. And it shouldn't have a large effect on the code. So running with this issue is probably OK.

Here is Mariana's email on this regarding what she did specifically...

I am not finding any problems in testing restarts. As a sanity check I did the following:

1. checkout out the same code base as Cecile
2. created a test using the following
   ./create_test -testname ERS_Ld7.f09_g16.B1850.yellowstone_intel.allactive-defaultio -testid hannay
3. modified user_nl_clm, and user_nl_cpl and user_nl_cpl as in Cecile's case
4. modified env_run.xml as in Cecile's case

The test built and ran successfuly with no problem. The output is below.

DONE ERS_Ld7.f09_g16.B1850.yellowstone_intel.allactive-defaultio : (test finished, successful coupler log)
--- Test Functionality ---:
PASS ERS_Ld7.f09_g16.B1850.yellowstone_intel.allactive-defaultio.cam.h0.nc : test compare cam.h0 (.base and .rest files)
PASS ERS_Ld7.f09_g16.B1850.yellowstone_intel.allactive-defaultio.cice.h.nc : test compare cice.h (.base and .rest files)
FAIL ERS_Ld7.f09_g16.B1850.yellowstone_intel.allactive-defaultio.clm2.h0.nc : test compare clm2.h0 (.base and .rest files)
PASS ERS_Ld7.f09_g16.B1850.yellowstone_intel.allactive-defaultio.pop.h.nc : test compare pop.h (.base and .rest files)
PASS ERS_Ld7.f09_g16.B1850.yellowstone_intel.allactive-defaultio.cism.h.nc : test compare cism.h (.base and .rest files)
PASS ERS_Ld7.f09_g16.B1850.yellowstone_intel.allactive-defaultio.cpl.hi.nc : test compare cpl.hi (.base and .rest files)
FAIL ERS_Ld7.f09_g16.B1850.yellowstone_intel.allactive-defaultio : test functionality summary (ERS_test)
PASS ERS_Ld7.f09_g16.B1850.yellowstone_intel.allactive-defaultio.memleak

However, that said - clm did not pass the restart history file test for only the following field:
RMS NPP_BURNEDOFF 2.7888E-08 NORMALIZED 1.2109E-01

My testdir is:
/glade/u/home/mvertens/src/cesm1_5_alpha06_cntlexp05/cime/scripts/ERS_Ld7.f09_g16.B1850.yellowstone_intel.allactive-defaultio.hannay

My executable dir is:
/glade/scratch/mvertens/ERS_Ld7.f09_g16.B1850.yellowstone_intel.allactive-defaultio.hannay/run

[ekluzek]

Erik Kluzek < erik > - 2016-03-18 12:20:42 -0600

I'm also seeing differences in NPP_BURNEDOFF in the difference between respmods_n12_clm4_5_7_r169 and respmods_n13_clm4_5_7_r172. I'm going to ignore that, because I think this is related to this issue where we know there are issues with how NPP_BURNDEDOFF is handled

For example here is a case that shows the issue...

grep RMS ERP_P15x2_Lm25.f10_f10.ICLM50BGCDVCROP.yellowstone_intel.clm-monthly.GC.160318-012401/TestStatus.log
RMS NPP_BURNEDOFF 3.1330E-09 NORMALIZED 2.5670E-02
RMS NPP_BURNEDOFF 5.9461E-08 NORMALIZED 4.7349E-01

[ekluzek]

Bill Sacks < sacks > - 2016-04-13 14:24:49 -0600

For now I'm making NPP_BURNEDOFF inactive. That should be changed back once this bug is fixed.

[ekluzek]

NPP_BURNEDOFF is still marked inactive. Likely this bug is still happening. This is set in CNFUNMod.

[samsrabin]

It looks like the issue is just that npp_burnedoff_patch isn't on the restart file. Should be an easy fix. Make sure to add a testmod and aux_clm test exercising this.