Add filenames to cpl.input_data_list, more error messaging, set drv_restart_pointer default to none unless needed #528

ekluzek · 2025-01-18T23:02:40Z

Description of changes

Correct the input groups with filenames to be added to cpl.input_data_list. Add more error messaging to the log files around rpointer errors. Set the drv_restart_pointer by default to none, unless it's needed, and make it a relative path to RUNDIR so will be added to cpl.input_data_list and checked that it exists.

Specific notes

Contributors other than yourself, if any:

CMEPS Issues Fixed (include github issue #):
Fixes #524
Fixes #525
Fixes #526

Are changes expected to change answers? bfb

Any User Interface Changes (namelist or namelist defaults changes)? yes
drv_restart_pointer gets set to "none" except for cases that need it like branch

Testing performed.

currently only tested with ctsm5.3.020 which uses: ccs_config_cesm1.0.16, and cime from ESMCI/cime#4739
and for a simple case with a startup and branch type

will do more testing, and will update the PR and report about what's done.
Plan to run aux_clm testlist as well as aux_cmeps.

…st file to check for file existence, fixing ESCOMP#526

… allow continue_run as well), add it as a file that will be put into cpl.input_data_list for existence checking, fixing ESCOMP#525

…e drv_restart_pointer is set in that case, also move it and rundir to the default_settings group so won't be output

…utdata named none -- but when UNSET is used it doesn't add it to cpl.input_data_list and so works

ekluzek · 2025-01-24T23:50:55Z

#532 brings in some additional nice changes beyond what is in here. And after that comes in I'll update to it, and likely change a few things here that won't be needed anymore.

ekluzek · 2025-01-24T23:52:22Z

Also update the expected fails file to resolve Expected fail in aux_cmeps test for nvhcp compile due to fail in compile of CAM file #529

cesm/driver/esm_time_mod.F90

billsacks

Thanks a lot for your work fixing this issue, @ekluzek !

I have a few questions / possible suggestions here. I'm not familiar enough with this to give a confident review, but I can try to review more carefully if needed.

cesm/driver/esm_time_mod.F90

billsacks · 2025-01-28T19:26:37Z

cime_config/buildnml

    # --------------------------------
    namelist_file = os.path.join(confdir, "drv_in")
-    drv_namelist_groups = ["papi_inparm", "prof_inparm", "debug_inparm"]
+    drv_namelist_groups = ["DRIVER_attributes", "MED_attributes", "ALLCOMP_attributes", "ROF_attributes", "WAV_attributes"]


Can you explain this change - why you removed the old set and how you chose this new set of groups?

The old ones no longer exist. And the new ones are valid namelist groups that weren't included.

Checking in the latest code, papi_inparm and prof_inparm are here and in the namelist definitation -- but not references in the Fortran code. So they can be removed.

It looks like debug_inparm now has one namelist item, so should be added back in.

I rechecked the others, and it looks like ROF_attributes and WAV_attributes are no longer used in the Fortran source, but the others are.

Here's an example check of namelist items that are in the namelist definition, but NOT in the Fortran source:

git grep -i WAV_attributes | more cime_config/namelist_definition_drv.xml: <entry id="Verbosity@wav_attributes" modify_via_xml="ESMF_VERBOSITY_LEVEL"> cime_config/namelist_definition_drv.xml: <group>WAV_attributes</group> cime_config/namelist_definition_drv.xml: <group>WAV_attributes</group>

While here is what I see for something that is in the Fortran code:

cesm/driver/ensemble_driver.F90: call ReadAttributes(driver, config, "DRIVER_attributes::", rc=rc) cesm/driver/esm.F90: call ReadAttributes(driver, config, "DRIVER_attributes::", formatprint=.true., rc=rc) cesm/driver/esm.F90: character(len=*), parameter :: subname = '(driver_attributes_check) ' cime_config/namelist_definition_drv.xml: <entry id="Verbosity@driver_attributes" modify_via_xml="ESMF_VERBOSITY_LEVEL"> cime_config/namelist_definition_drv.xml: <group>DRIVER_attributes</group> cime_config/namelist_definition_drv.xml: <group>DRIVER_attributes</group> cime_config/namelist_definition_drv.xml: <group>DRIVER_attributes</group> . . . cime_config/namelist_definition_drv.xml: <group>DRIVER_attributes</group>

These attributes are read and used by ESMF/NUOPC

So... partly for my own tracking of this... my understanding from this conversation is that some updates are still needed to drv_namelist_groups, like adding debug_inparm back in? And I'm wondering if papi_inparm and prof_inparm should either be added back here or removed from the namelist_definition?? It seems confusing to me to have buildnml be inconsistent with the namelist_definition: this seems prone to confusing someone later. But I'm not sure I'm following this conversation correctly, so am happy to be corrected on this.

billsacks · 2025-01-28T19:34:12Z

cime_config/namelist_definition_drv.xml

+  <entry id="rundir" modify_via_xml="RUNDIR">
+    <type>char</type>
+    <category>nuopc</category>
+    <!-- NOTE: This won't be output only used as an attribute in this file -->
+    <group>default_settings</group>
+    <values>
+      <value>$RUNDIR</value>
+    </values>
+  </entry>
+
+  <entry id="continue_run" modify_via_xml="CONTINUE_RUN">
+    <type>logical</type>
+    <category>nuopc</category>
+    <!-- NOTE: This won't be output only used as an attribute in this file -->
+    <group>default_settings</group>
+    <values>
+      <value>$CONTINUE_RUN</value>
+    </values>
+  </entry>


I'm not super-familiar with this, but I'm thinking that you can accomplish this by setting rundir and continue_run in the config settings in buildnml, similarly to what's currently done for many other variables in _create_drv_namelists.

Ahh, I'm pretty sure you are right, and that this bit could probably be removed.

Just for my own tracking of remaining to-dos... this one is still a to-do.

cime_config/config_component.xml

billsacks

Thanks a lot for these fixes, @ekluzek ! A few comments / questions below.

billsacks · 2025-11-12T00:16:01Z

cime_config/buildnml

    # --------------------------------
    namelist_file = os.path.join(confdir, "drv_in")
-    drv_namelist_groups = ["papi_inparm", "prof_inparm", "debug_inparm"]
+    drv_namelist_groups = ["DRIVER_attributes", "MED_attributes", "ALLCOMP_attributes", "ROF_attributes", "WAV_attributes"]


So... partly for my own tracking of this... my understanding from this conversation is that some updates are still needed to drv_namelist_groups, like adding debug_inparm back in? And I'm wondering if papi_inparm and prof_inparm should either be added back here or removed from the namelist_definition?? It seems confusing to me to have buildnml be inconsistent with the namelist_definition: this seems prone to confusing someone later. But I'm not sure I'm following this conversation correctly, so am happy to be corrected on this.

cime_config/config_component.xml

billsacks · 2025-11-12T00:22:57Z

cime_config/config_component.xml

+    <values>
+       <value>rpointer.cpl.${RUN_STARTDATE}-${RUN_REFTOD}</value>
+    </values>


This setting of the value seems redundant with the default_value above. Do you actually need both?

billsacks · 2025-11-12T00:23:51Z

cime_config/namelist_definition_drv.xml

+  <entry id="rundir" modify_via_xml="RUNDIR">
+    <type>char</type>
+    <category>nuopc</category>
+    <!-- NOTE: This won't be output only used as an attribute in this file -->
+    <group>default_settings</group>
+    <values>
+      <value>$RUNDIR</value>
+    </values>
+  </entry>
+
+  <entry id="continue_run" modify_via_xml="CONTINUE_RUN">
+    <type>logical</type>
+    <category>nuopc</category>
+    <!-- NOTE: This won't be output only used as an attribute in this file -->
+    <group>default_settings</group>
+    <values>
+      <value>$CONTINUE_RUN</value>
+    </values>
+  </entry>


Just for my own tracking of remaining to-dos... this one is still a to-do.

billsacks · 2025-11-12T00:26:49Z

cime_config/namelist_definition_drv.xml

  <entry id="drv_restart_pointer" modify_via_xml="DRV_RESTART_POINTER">
    <type>char</type>
    <category>expdef</category>
+    <default_value>UNSET</default_value>


This default_value seems redundant with the value settings below and can probably be removed.

ekluzek added 3 commits January 18, 2025 14:37

Add more error messaging fixing ESCOMP#524

b742505

Change the list of groups added for writing out the cpl.input_data_li…

ee2677f

…st file to check for file existence, fixing ESCOMP#526

Only set drv_restart_pointer for branch type (NOTE: will also need to…

c1dcacb

… allow continue_run as well), add it as a file that will be put into cpl.input_data_list for existence checking, fixing ESCOMP#525

ekluzek added enhancement New feature or request Responsibility: CTSM Responsibility to manage and accomplish this issue is the CTSM Software group labels Jan 18, 2025

ekluzek requested review from billsacks, fischer-ncar and jedwards4b January 18, 2025 23:02

ekluzek added 2 commits January 18, 2025 16:34

Add continue_run to namelist definition so it can be used to make sur…

f114657

…e drv_restart_pointer is set in that case, also move it and rundir to the default_settings group so won't be output

Setting drv_restart_pointer to none tries to pull a file from svn inp…

6ca33f9

…utdata named none -- but when UNSET is used it doesn't add it to cpl.input_data_list and so works

billsacks reviewed Jan 28, 2025

View reviewed changes

cesm/driver/esm_time_mod.F90 Show resolved Hide resolved

billsacks reviewed Jan 28, 2025

View reviewed changes

jedwards4b reviewed Nov 11, 2025

View reviewed changes

cime_config/config_component.xml Show resolved Hide resolved

jedwards4b approved these changes Nov 11, 2025

View reviewed changes

ekluzek changed the title ~~[WIP]: Add filenames to cpl.input_data_list, more error messaging, set drv_restart_pointer default to none unless needed~~ Add filenames to cpl.input_data_list, more error messaging, set drv_restart_pointer default to none unless needed Nov 11, 2025

ekluzek mentioned this pull request Nov 11, 2025

Update submodules to cesm3_0_alpha07h level ESCOMP/CTSM#3583

Open

billsacks requested changes Nov 12, 2025

View reviewed changes

Add filenames to cpl.input_data_list, more error messaging, set drv_restart_pointer default to none unless needed #528

Are you sure you want to change the base?

Add filenames to cpl.input_data_list, more error messaging, set drv_restart_pointer default to none unless needed #528

Conversation

ekluzek commented Jan 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description of changes

Specific notes

Testing performed.

Uh oh!

ekluzek commented Jan 24, 2025

Uh oh!

ekluzek commented Jan 24, 2025

Uh oh!

Uh oh!

billsacks left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

billsacks left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

ekluzek commented Jan 18, 2025 •

edited

Loading