post5.4.000: Use the CMIP7 Carbon Isotope streams files by default

[ekluzek]

Description of changes

By default read in the cmip7 c13/c14 stream files rather than the previous cmip6 ones. Read in the cmip6 ones if CLM_CMIP_ERA=cmip6 and use_c13/use_c14 are on.

Specific notes

Contributors other than yourself, if any:

CTSM Issues Fixed (include github issue #):

Are answers expected to change (and if so in what way)? Yes (only when use_c13/use_c14 is on)

Any User Interface Changes (namelist or namelist defaults changes)?
By default read in cmip7 streams files for use_c13/use_c14
Only read in older CMIP6 timeseries files for use_c13/use_c14 if CLM_CMIP_ERA=cmip6

Does this create a need to change or add documentation? Did you do so? No No

Testing performed, if any: Running standard testing

[ekluzek]

Fails I see in testing on Derecho:

Fail at runtime (in the C14 stream Advance, having trouble finding the current VM:

ERP_D_Ld10_P64x2.f10_f10_mg37.IHistClm60BgcCrop.derecho_intel.clm-ciso_decStart (NLCOMP RUN)
ERP_D_Ld10_P64x2.f10_f10_mg37.IHistClm60BgcCrop.derecho_intel.clm-ciso_decStart--clm-matrixcnOn_ignore_warnings (NLCOMP RUN)
ERP_P64x2_D_Ld5.f10_f10_mg37.I1850Clm45BgcCru.derecho_intel.clm-ciso    (NLCOMP RUN)
ERP_P64x2_D_Ld5.f10_f10_mg37.I1850Clm50Bgc.derecho_intel.clm-ciso       (NLCOMP RUN)
ERP_P64x2_D_Ld5.f10_f10_mg37.I1850Clm50Bgc.derecho_intel.clm-ciso--clm-matrixcnOn_ignore_warnings       (NLCOMP RUN)

Fail with the restart comparison: (for C13 variables)

ERP_D_Ld9.ne30pg3_t232.IHistClm60BgcCropG.derecho_intel.clm-clm60cam7LndTuningModeLDust (NLCOMP COMPARE_base_rest BASELINE)
ERS_D_Ld5.f10_f10_mg37.IHistClm50BgcQian.derecho_intel.clm-ciso_bombspike1963DecStart   (NLCOMP COMPARE_base_rest BASELINE)
ERS_D_Ld5.f10_f10_mg37.IHistClm50BgcQian.derecho_intel.clm-ciso_bombspike1963DecStart--clm-matrixcnOn_ignore_warnings   (NLCOMP COMPARE_base_rest BASELINE)

I think the first group may be a problem in needing more memory? Since these are ERP tests it will do one run at P32x1 and that might be too few tasks. I'll try these same tests at P128x1 and P128x2 (which I suspect won't work because of too many processors). If the last doesn't work I'll try something like P200x2.

On the second set, it's a clue that it's only failing for C13. So there must be something different between C13 and C14. I also wonder about the order that things are done including initialization, and possibly there needs to be an adjustment as to when these things happen. I can compare to other streams to get some clues.

[ekluzek]

Actually I see a note in clm_driver, that streams aren't threadsafe. So likely what needs to happen is to lift the streams handling out of Photosynthesis to a higher level in clm_driver. This could be done on b4b-dev.

[ekluzek]

OK, moving the C13/C14Streams calls to the clm_driver outside of openMP thread loops fixes one problem. And there was a missing Advance in C13Streams that fixes the other. I'll bring these in on b4b-dev and then merge it in here to resolve the final issues here.