@ekluzek, since this is ready to go and going to b4b dev, can you just go ahead a merge this when you're ready. I've put it to the top of the b4b upcoming tags that are in progress.

Actually, I've been running into issues here, so it isn't ready to go.

One issue is that the CISM tag changes answers, so we'll bring it in seperately.

These tests showed a change in answers because of CISM:

ERI_D.ne30pg3_t232.I1850Clm60BgcCropG.derecho_intel.clm-clm60cam7LndTuningModeLDust
ERP_D_Ld5.f10_f10_mg37.I1850Clm50BgcCropG.derecho_gnu.clm-glcMEC_changeFlags
ERP_D_Ld9.ne30pg3_t232.I1850Clm60BgcCropG.derecho_intel.clm-clm60cam6LndTuningMode
ERP_D_Ld9.ne30pg3_t232.I1850Clm60BgcCropG.derecho_intel.clm-clm60cam7LndTuningModeLDust
ERP_D_Ld9.ne30pg3_t232.IHistClm60BgcCropG.derecho_intel.clm-clm60cam7LndTuningModeLDust
ERP_P256x2_D_Ld5.f19_g17_gris4.I1850Clm50BgcCropG.derecho_intel.clm-glcMEC_increase
ERP_P64x2_D_Ld10.f10_f10_mg37.IHistClm50SpG.derecho_intel.clm-glcMEC_decrease--clm-nofireemis
ERP_P64x2_Ld396.f10_f10_mg37.IHistClm60Bgc.derecho_intel.clm-monthly--clm-matrixcnOn_ignore_warnings             EXPECTED POSSIBILITY
ERS_D_Ld12.f10_f10_mg37.I1850Clm50BgcCropG.derecho_intel.clm-glcMEC_spunup_inc_dec_bgc
ERS_D_Ld9.ne30pg3_t232.I1850Clm60BgcCropG.derecho_intel.clm-clm60cam7LndTuningMode
ERS_Ly3_P64x2.f10_f10_mg37.IHistClm50BgcCropG.derecho_intel.clm-cropMonthOutput
SMS_Lm1.ne30pg3_t232.I1850Clm60BgcCropCrujraG.derecho_intel.clm-clm60cam7LndTuningMode
SMS_Lm1.ne30pg3_t232.I1850Clm60BgcCropG.derecho_intel.clm-clm60cam7LndTuningMode
SMS_Ly3.f10_f10_mg37.I1850Clm50SpG.derecho_intel.clm-glcMEC_long--clm-nofireemis

And one failing test that's unexpected:

ERR_Ld7.f10_f10_mg37.IHistClm60BgcCrop.derecho_gnu.drv-interim_restart (NLCOMP RUN)

This is also updated to a cime based off of cime6.1.137, as updated to cime6.1.142 had cases die in setup, which look like is due to the namelist case change that comes in with cime6.1.138.

This is also updated to a cime based off of cime6.1.137, as updated to cime6.1.142 had cases die in setup, which look like is due to the namelist case change that comes in with cime6.1.138.

@ekluzek - can you tell me what failures you're seeing that appear due to cime6.1.138? That was a CIME tag that I made, so if you're seeing issues with it, I'd like to understand these and look into them. (I'm wondering if there's some externals difference between what you're testing and what I tested with, or if there's some issue that turns up in some test cases that I didn't see in my more limited testing of that change?)

@ekluzek - can you tell me what failures you're seeing that appear due to cime6.1.138? That was a CIME tag that I made, so if you're seeing issues with it, I'd like to understand these and look into them.

Thanks for asking @billsacks . My comment came early enough that I hadn't narrowed down exactly what was going on. But from looking at cime tags I was most suspicious of that one. And tried the one before it which has also been in CESM testing and found it to work. I also found I needed a cime change on a branch I had -- but I thought my testing problems also had something else going on besides that. But I'm going back a little more carefully I updated my branch to 6.1.137 and found it to work. And it worked with an update to 6.1.143 as well. At least for the one test I did.

So bottom line is that 6.1.137 looks to be fine right now. But I might learn something new after I run all the testing.

Sorry for laying blame on your tag when it was something else. I did that before I had absolutely verified it, but it was my guess at the time

@ekluzek - okay, thanks for the follow-up. I'm glad to hear that it looks like cime6.1.138 is probably okay: that turned out to be a trickier set of changes than I had originally anticipated, so I'm really hoping that these changes haven't caused some new subtle issue that would add even more trickiness! Do let me know if you are seeing some issues with it in full testing.

There are 22 tests on Derecho that are still in the queue. And several unexpected fails to look into:

ERI_D_Ld9.f45_f45_mg37.I2000Clm60FatesSpCruRsGs.derecho_intel.clm-FatesColdSatPhenCamLndTuningMode      (NLCOMP RUN)
(COMPARE_base_hybrid)
ERP_D_Ld5.f10_f10_mg37.I1850Clm50BgcCropG.derecho_gnu.clm-glcMEC_changeFlags    (NLCOMP RUN)
ERP_P256x2_D_Ld5.f19_g17_gris4.I1850Clm50BgcCropG.derecho_intel.clm-glcMEC_increase     (NLCOMP RUN)
ERP_P64x2_D_Ld10.f10_f10_mg37.IHistClm50SpG.derecho_intel.clm-glcMEC_decrease--clm-nofireemis   (NLCOMP RUN)
ERR_Ld7.f10_f10_mg37.IHistClm60BgcCrop.derecho_gnu.drv-interim_restart  (NLCOMP RUN)
ERS_D_Ld12.f10_f10_mg37.I1850Clm50BgcCropG.derecho_intel.clm-glcMEC_spunup_inc_dec_bgc  (NLCOMP RUN)
ERS_Ly3_P64x2.f10_f10_mg37.IHistClm50BgcCropG.derecho_intel.clm-cropMonthOutput (NLCOMP RUN)
LII2FINIDATAREAS_D_P256x2_Ld1.f09_t232.I1850Clm60BgcCrop.derecho_intel.clm-default--clm-matrixcnOn_ignore_warnings  (NLCOMP RUN)
PEM_D_Ld20.5x5_amazon.I2000Clm50FatesRs.derecho_intel.clm-FatesColdSeedDisp     (NLCOMP COMPARE_base_modpes TPUTCOMP)
REP_P64x2_Ld396.f10_f10_mg37.IHistClm60Bgc.derecho_intel.clm-monthly--clm-matrixcnOn_ignore_warnings    (NLCOMP COMPARE_base_rep2
SMS_Ly3.f10_f10_mg37.I1850Clm50SpG.derecho_intel.clm-glcMEC_long--clm-nofireemis        (NLCOMP RUN)

One Izumi, all of the tests failed at runtime, so that needs to be figured out as well.

Most of the fails are due to a dependency between the latest CISM and ccs_config. So I need to back up ccs_config a little bit to 1.0.61 in order to work with the older cism.

The ERR test fails because it's looking for the wrong cpl rpointer file. The LIIFINIDATAREAS test fails as it can't find bare soil, which is wierd. FATES SeedDisp test fails with a difference in two seed fields. The top ERI test fails with a floating invalid in taking the difference between mlai2t(:,1) vs. mlai2t(:,2) at the end or readMonthlyVegetation.

So the ERR test has to do with the cmeps update for rpointer file handling. I'm not sure about the other two.

From the CSEG meeting yesterday there are some status readability changes for gitr-fleximod in v1.1.0 and a fix to a problem from v1.0.1 in v1.1.1 so we should also update it. We currently have v1.0.2 so we especially want to bring that fix in. The problem will overwrite changes in submodules with modified code, so an especially bad one.

A follow-up thought:
Do we allow submodule updates merged to b4b-dev? Does testing confirm that this is b4b?

A follow-up thought:
Do we allow submodule updates merged to b4b-dev? Does testing confirm that this is b4b?

Good questions. We have allowed submodule updates on b4b-dev, but with extra testing. The changes are extensive enough that we could make this a tag on master though. So far my testing has shown that this is b4b.

Also the changes to git-fleximod dominate -- but that's just because git-fleximod is a subtree. It's actually just updating it to v1.1.1 so not really something for us to look at the differences in detail.

@ekluzek I could approve, but I am not familiar with these codes:

• seems like git-fleximod stuff mostly
• also do you need a new line in .git-blame-ignore-revs, because I sense that many of the mods are due to reformatting by black or lint

The first one I talk about above. So neither of us need to know much about git-fleximod.

We don't need new lines in the git blame ignore because the black format commits are only for git-fleximod and don't apply to general CTSM code where we do want to track the black commits.

I've sent all the testing for Derecho/Izumi: aux_clm, fates, ctsm_sci. So far it looks fine, but a few more need to finish

One test failed, but I think this is expected:

ERI_Ld9.f09_t232.I2000Clm60BgcCropCrujra.derecho_intel.clm-default

It looks like the ERI test is just a reappearance of this problem that was dealt with on the alpha ctsm5.4 branch, because the IC file is from ctsm5.4.

#3576 (comment)

So I'm calling this handled for the purpose of this PR.