Have the FATES parameter file created at runtime in the buildnml

[ekluzek]

This was an idea talked about here:

NGEET/fates#1073

We think a better way to handle FATES parameter files is to have the buildnml use the FATES tools to create them from the CDL files in FATES. A shorter way to do this would be for that to happen in a testmod in the shell_commands file. But, longer term we probably want it as part of the configure process for a CESM case. This also gets people in the habit of using the tools to make FATES parameter files themselves which sets them up to configure their files for their situation from the get go. That's a more FATES like way to run, and sets people up for doing things like PPE work, or site specific changes.

Definition of done: FATES parameter files are created from the CDL file stored in FATES

NOTE: The longer term solution that is desired is to store the default parameter file in FATES as XML. With this python tools that modify the file will just have standard python packages and NOT also require an extended Python environment using conda and activating ctsm_pylib as we require now.

Steps to get there:

• Add some testmods for FATES to test some different FATES parameter file options #2151
• Testing cleanup from the cesm3_0_beta03/04 "chill" tags for cesm3_0_beta05 #2888 get the shell_commands working with error checking and in one place
• ctsm5.3.014 -- brings in a testmod include dir for graceful management of ctsm_pylib for tests
• Move switches to namelist ctsm5.3.027: Migrate FATES parameter file switches to the namelist #2904
• Get approval from CESM for the preferred approach (they just don't want custom conda environments for CESM testing)
• FATES JSON parameter files #3570 move FATES to a JSON file
• Finalize document https://docs.google.com/document/d/1XUjOGcBVbW8dY_0F2kmK8QJaB2nJn1th_474lpQPS_Q/edit?tab=t.0#heading=h.9ifrjra9jr1h
• Implement just the new XML variable, and the DEFAULT and COPY option for it (DEFAULT is the default option until the step further down) COPY , copies the parameter file to the case directory, CaseDocs, and RUNDIR (this goes in buildnml)
• - [ ] Make the COPY option the default, but still allow the DEFAULT option
• Have that testmod include directory used for ALL FATES tests?
• fates parameter file auto-build for all tests #2336?
• Add the MODIFY_PATCH option that runs the modify_params script
• Add the BATCH_PATCH option, that runs the COPY option and then runs batch_patch on top of it. This will require checking the python environment.
• Convert the tests to use the BATCH_PATCH option

[ekluzek]

There's several considerations for this that we need to think about:

• Will this take enough time that we should add a switch for it so it's not always done (note one way to do that is to skip the namelist generation step, which CESM has several ways of doing already)?
• Should there be a way to do this for multi-instance cases?
• Should there be a way to put a CDL file in your case directory to be used in place of the default one in FATES?
• Should there be any other control variables (namelist or XML) to control this process?
• Should we still store FATES parameter files in svn inputdata -- or always rely on this process?
• If we do store FATES parameter files in svn inputdata how often do we do that?
• Are there any robustness concerns with this process, or python package or versioning issue concerns?

@adrifoster @glemieux @rgknox @billsacks

[ekluzek]

I think the main script for this is here:

https://github.com/NGEET/fates/blob/main/tools/UpdateParamAPI.py

Any scripts used would be in that directory.

Looking at dependencies, I see that scipy and numpy are being used, so that's pretty basic, but beyond the current barebones requirements that cime currently has.

[rgknox]

@ekluzek and all:

When I'm running FATES and making perturbations to parameters in the FATES parameter file, there are two things I typically do, depending on the scenario.

Scenario 1: I'm running a simulation where I've already calibrated a set of parameters, and this set of parameters is not encapsulated in the default file. This set of parameters could be associated with the default PFT set, or a subset, or a completely new set of PFTs.

1. run BatchPatchParams.py , which will interpret an xml file that describes this set of parameter file modifications to the default. That xml file is kind of like a patch or diff file for a base parameterization.
2. copy the new binary to my case directory and point to it in user_nl_clm

Scenario 2: I want to do a "one-off" perturbation to a parameter, or multiple parameters. The basis of my parameter set may be the default, or a different file perhaps generated from Scenario 1:

1. if not done already, copy the binary form of the fates parameter file (default or other) to my case directory and point to it in user_nl_clm
2. run the python module: modify_fates_paramfile.py for each specific parameter I want modified. This will act on the binary form of the parameter file, and runs in an over-write fashion

---

I could picture both of these methods being used in a testing workflow, or normal build and run workflow. The combination of them would certainly give us better testing coverage of FATES. The BatchPatchParams.py could potentially over-complicate a testing workflow, but might come in handy. The most direct method to modifying fates parameter for testing, might be calling modify_fates_paramfile.py in some type of shell or existing execution script.

Either way, both could be worked into a system where we build the file at run-time.

[ekluzek]

@rgknox that's very helpful. Yes, we should incorporate both scenarios into our testing workflow and make sure they are options that can be done from buildnml. As I think about this I think there should be some XML variables that control this operation. I'd like to talk about this more at tomorrow's software meeting.

I have one more idea I'm going to put in another comment...

[ekluzek]

Since, there are python module dependencies that are beyond what cime normally requires (which right now is only the python version) we should add some error checking for those modules and python version dependencies.

This should also eventually go into the tool in FATES, but to start with we should have a graceful fail in the CTSM side. Here's the issue to put it into FATES...

NGEET/fates#1076

As a name for an XML field I'm thinking of something like:

CLM_FATES_PARAMFILE_BUILD = DEFAULT,NCGEN,BATCH_PATCH

The BATCH_PATH option will also have to assume a name for the input file it'll use we suggest: user_nl_FATES_param_changes.xml below.

[ekluzek]

We had some discussion on this today. I modified the above proposal a bit based on that. It does seem clear that transitioning to always building the parameter file would be the best way to separate the HLM's from needing to update whenever new parameters are added to the FATES parameter file. But, I think we should do this as a transition, so start allowing it, and then transition to one of the parameter file creation options is the only choice.

I tried to see if there might be a way to have the HLM still work with an updated version of FATES that has new parameters on the FATES parameter file. But, that doesn't seem workable. So working towards always creating the FATES parameter file at buildnml time is a better long term solution.

I also realized that an option to use straight up ncgen should be one of the options. The nice thing about that is that's part of NetCDF so will be loaded in the modules to run the model. So it's not an new requirement.

[slevis-lmwg]

Copied from google meet chat:

• user_nl_FATES_param_changes.xml
• maybe we should add a Unit Tests section to the Design Doc

[glemieux]

From today's meeting on parameter files we discussed how a future PR could utilize this infrastructure to provide a testmod for the fates seed dispersal code, since it needs reasonable values based on the resolution type. As requested, I've attached a potential xml file (adding a txt suffix since github doesn't like xml).

seed-dispersal-f45.xml.txt

Seed dispersal PR for future reference: NGEET/fates#1005 and #2077

[ekluzek]

I looked up in cime, and we need to have a "user_nl_" prefix for it to be handled correctly. So we should have the name...

user_nl_FATES_param_changes.xml