FATES JSON parameter files #3570

rgknox · 2025-10-27T18:25:55Z

Description of changes

These changes accommodate a JSON format to the FATES parameter file. This set of changes is synchronized with FATES 1493.

This is a work in progress.

Specific notes

CTSM/CIME is in charge of identifying the parameter file that is used by fates
CTSM finds an available unit number and passes that unit number and the parameter file name to FATES
The complicated coordination between CTSM/FATES that was necessary because FATES did not have access to the netcdf libraries is no longer required.
Parameter file reads in FATES previous occurred in two seperate instances (for pft variables and for everything else), which was weird, that is changed here.
This PR should contain updates to the testing code to handle a parameter file manipulation

Contributors other than yourself, if any:

CTSM Issues Fixed (include github issue #):
Work on #2126

Are answers expected to change (and if so in what way)? No

Any User Interface Changes (namelist or namelist defaults changes)? Change FATES parameter file to JSON format

Does this create a need to change or add documentation? Did you do so? No

Testing performed, if any: individual testing, will run aux_clm and fates on Derecho and Izumi

ekluzek · 2025-11-19T18:54:13Z

@rgknox can you add the fates branch you are pointing to here in .gitmodules?

I also wanted to know what the python version and module environment requirements will be for the new

$FATESDIR/tools/modify_fates_paramfile.py

script? If it's just bare python, that will allow us to run these tests in CESM testing which would be a nice thing to have for when this comes in. I just found the FATES PR and it looks like the one requirement is numpy, which doesn't come in by default for bare python. So CESM tests wouldn't be able to run modify_fates_paramfile.py, unless we do something about removing that requirement. Hmmm.

… use that root

…also updated shell_commands to use updated file type and scripting

Merge b4b-dev to master PR ESCOMP#3609

rgknox · 2025-11-26T19:22:12Z

This is passing fates test suit on derecho, B4B with base

rgknox · 2025-11-26T19:24:07Z

@ekluzek , the python scripts for fates parameter file modifications, including $FATESDIR/tools/modify_fates_paramfile.py, just use the core libraries now. Derecho had no trouble with modify_fates_paramfile.py. Lets see how izumi does..

ekluzek

Cool. I have some comments and questions from looking at this on my own. But, I'm looking forward to getting the walkthrough to understand in more depth.

ekluzek · 2025-12-04T07:29:25Z

bld/CLMBuildNamelist.pm

     Create the namelist for CLM
 REQUIRED OPTIONS
     -cimeroot "directory"    Path to cime directory
+     -landroot "directory"    Path to the land models parent directory, ie <>/ctsm


We probably don't need to send landroot in, as we should be able to get it from information already in CLMBuildNamelist.pm

ekluzek · 2025-12-04T07:33:50Z

bld/namelist_files/namelist_defaults_ctsm.xml


-<fates_paramfile>lnd/clm2/paramdata/fates_params_api.41.0.0_14pft_c250813.nc</fates_paramfile>
-
+<fates_paramfile>src/fates/parameter_files/fates_params_default.json</fates_paramfile>


Hmmm. Interesting. Currently we assume that anything with a relative path is under $DIN_LOC_ROOT. But, here we need to point out that it's under LANDROOT. So we should probably add an attribute to say that here.

ekluzek · 2025-12-04T07:34:44Z

bld/namelist_files/namelist_definition_ctsm.xml


 <entry id="fates_paramfile" type="char*512" category="datasets"
-       input_pathname="abs" group="clm_inparm" valid_values="" >
+       input_pathname="landroot" group="clm_inparm" valid_values="" >


Ahhh, here is where you are pointing out that it's under landroot.

ekluzek · 2025-12-04T07:39:41Z

src/utils/clmfates_interfaceMod.F90

+
+     ! It is FATES' job to open and close the file
+     ! This step is simply to provide a unit number to use
+     fates_paramfile_unit = shr_file_getUnit()


shr_file_getUnit is actually deprecated now that Fortran has built-in file unit management. When you use open to open a file you can use newunit= to get a free unit. So it simplifies how this is done.

ekluzek · 2025-12-04T07:41:39Z

src/utils/clmfates_paraminterfaceMod.F90

@@ -1,235 +0,0 @@
-module CLMFatesParamInterfaceMod
-  ! NOTE(bja, 2017-01) this code can not go into the main clm-fates
-  ! interface module because of circular dependancies with pftvarcon.


Wow, you were able to remove this entire file?! That's amazing.

So basically this file that's HLM specific gets replaced by FatesReportParameters which is in FATES so it's reused by all of the HLM's, which is a great direction to go.

ekluzek · 2025-12-04T07:52:17Z

cime_config/testdefs/testmods_dirs/clm/FatesColdPRT2/shell_commands


+$FATESDIR/tools/modify_fates_paramfile.py --overwrite --fin $FATESPARAMFILE --param fates_cnp_prescribed_puptake --values 1.0 --indices all
+
+echo "fates_paramfile = '$FATESPARAMFILE'" >> $CASEDIR/user_nl_clm


I'm a bit concerned about this bit about adding it to the end of the user_nl_clm. A small thing is that each time shell_commands is run it'll append it to the end again. It still works -- but looks a bit strange.

But, putting it here could conflict with what the user adds to a test case. Since, it is a test case that seems unlikely -- but I wonder about unintended consequences. This is a part that can be removed when we can move forward on the plan to move fates parameter file handling into buildnml. So it's likely fine to do as a first step.

ekluzek · 2025-12-04T07:53:26Z

cime_config/testdefs/ExpectedTestFails.xml

    </phase>
  </test>
-
-  <test name="ERI_D_Ld9.f45_f45_mg37.I2000Clm60FatesSpCruRsGs.derecho_gnu.clm-FatesColdSatPhenCamLndTuningMode">


This fixes these two tests? Or is this something that was fixed with the nextsw change, that just didn't get these removed from ExpectedFails?

initial work transitioning to a json FATES parameter file

71e5d91

rgknox added code health improving internal code structure to make easier to maintain (sustainability) FATES API update Changes to the FATES version that also REQUIRE an API change in CTSM labels Oct 27, 2025

rgknox mentioned this pull request Oct 27, 2025

JSON based parameters NGEET/fates#1493

Open

17 tasks

rgknox added 2 commits October 28, 2025 13:11

removed deprecated fates parameter interface

4f88906

fates json parameter file overhaul.

4a59e01

ekluzek mentioned this pull request Nov 17, 2025

Have the FATES parameter file created at runtime in the buildnml #2126

Open

14 tasks

ekluzek mentioned this pull request Nov 19, 2025

fates parameter file auto-build for all tests #2336

Draft

rgknox added 4 commits November 21, 2025 20:57

passing landroot to CLMBuildNamelist, setting fates parameter file to…

96459bc

… use that root

Updating build scripts to accommodate the fates json parameter file, …

632c9bc

…also updated shell_commands to use updated file type and scripting

Merge tag 'ctsm5.3.085' into fates-json-params

4dbe613

Merge b4b-dev to master PR ESCOMP#3609

json updates, fixes to shell commands during testing

1942849

ekluzek added this to CTSM: Upcoming tags Dec 1, 2025

ekluzek moved this to In progress - master in CTSM: Upcoming tags Dec 1, 2025

glemieux changed the title ~~[WIP] FATES JSON parameter files~~ FATES JSON parameter files Dec 1, 2025

glemieux requested a review from ekluzek December 1, 2025 20:07

rgknox mentioned this pull request Dec 1, 2025

Add FATES test that uses stochastic promotion/demotion #3639

Open

ekluzek reviewed Dec 4, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

FATES JSON parameter files #3570

FATES JSON parameter files #3570

rgknox commented Oct 27, 2025 •

edited by ekluzek

Loading

Uh oh!

ekluzek commented Nov 19, 2025 •

edited

Loading

Uh oh!

rgknox commented Nov 26, 2025

Uh oh!

rgknox commented Nov 26, 2025

Uh oh!

ekluzek left a comment

Uh oh!

ekluzek Dec 4, 2025

Uh oh!

ekluzek Dec 4, 2025

Uh oh!

ekluzek Dec 4, 2025

Uh oh!

ekluzek Dec 4, 2025

Uh oh!

ekluzek Dec 4, 2025

Uh oh!

ekluzek Dec 4, 2025

Uh oh!

ekluzek Dec 4, 2025

Uh oh!

ekluzek Dec 4, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants


		<fates_paramfile>lnd/clm2/paramdata/fates_params_api.41.0.0_14pft_c250813.nc</fates_paramfile>

		<fates_paramfile>src/fates/parameter_files/fates_params_default.json</fates_paramfile>


		$FATESDIR/tools/modify_fates_paramfile.py --overwrite --fin $FATESPARAMFILE --param fates_cnp_prescribed_puptake --values 1.0 --indices all

		echo "fates_paramfile = '$FATESPARAMFILE'" >> $CASEDIR/user_nl_clm

FATES JSON parameter files #3570

Are you sure you want to change the base?

FATES JSON parameter files #3570

Conversation

rgknox commented Oct 27, 2025 • edited by ekluzek Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description of changes

Specific notes

Uh oh!

ekluzek commented Nov 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rgknox commented Nov 26, 2025

Uh oh!

rgknox commented Nov 26, 2025

Uh oh!

ekluzek left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

rgknox commented Oct 27, 2025 •

edited by ekluzek

Loading

ekluzek commented Nov 19, 2025 •

edited

Loading