rpointer.cpl bug causes ctsm restart to fail

[slevis-lmwg]

General bug information

CTSM version you are using:
branch_tags/tmp-241219.n01.ctsm5.3.016
Would have been ctsm master tag ctsm5.3.017 if ctsm's master branch were not "locked" while we wait for a new cesm tag.

Does this bug cause significantly incorrect results in the model's science? [Yes / No]
I'm guessing not.

Configurations affected: [Fill this in if known.]
CONTINUE = .true.

Important details of your setup / configuration so we can reproduce the bug

Case /glade/u/home/slevis/cases_LMWG_dev/ctsm53017_f19_BNF_AD documented in NCAR/LMWG_dev#88.

I have worked around the problem, so the simulation is in progress again. Here's a complete sequence of events:

1. I started this AD spinup as a cold start.
2. STOP_N was 40 and RESUBMIT was 6.
3. The model resubmitted successfully after year 40.
4. The model failed to resubmit after year 80.
5. I didn't find an error, so I tried to submit manually and immediately got
   ERROR: CONTINUE_RUN is true but this case does not appear to have restart files staged in /glade/derecho/scratch/slevis/ctsm53017_f19_BNF_AD/run rpointer.cpl
6. I got past the error with
   mv rpointer.cpl.0081-01-01-00000 rpointer.cpl
   but the run now failed while looking for rpointer.cpl.0081-01-01-00000
7. I got past both errors with
   cp rpointer.cpl.0081-01-01-00000 rpointer.cpl
   and the run is in progress now. I changed STOP_N to 210 hoping that I will not have to deal with this problem again in this simulation.

[ekluzek]

@slevis-lmwg I saw Jim commit a change to cime that might cover this. Can you try with cime6.1.56 and see if that fixes it?

This is also something that our SSP tests might have caught. But we let them go through failing.

[jedwards4b]

Please try with cime6.1.56 and let me know if it is still a problem.

[slevis-lmwg]

cime6.1.56 took care of it, thanks.

[ekluzek]

@slevis-lmwg let's leave this open until we have a tag on master that has the cime update in place.