Skip to content

fre.cmor: adjust current variable list usage to map modeler variable names to MIP ones #845

@ilaflott

Description

@ilaflott

fully describe the way fre.cmor is using variable lists, look at the unit test test_fre_cmor_cli.py::test_cli_fre_cmor_run_case2():

def test_cli_fre_cmor_run_case2():
    ''' fre cmor run, test-use case '''

    # explicit inputs to tool
    indir = f'{ROOTDIR}/ocean_sos_var_file'
    varlist = f'{ROOTDIR}/varlist_local_target_vars_differ'
    table_config = f'{ROOTDIR}/cmip6-cmor-tables/Tables/CMIP6_Omon.json'
    exp_config = f'{ROOTDIR}/CMOR_input_example.json'
    outdir = f'{ROOTDIR}/outdir'
    grid_label = 'gr'
    grid_desc = 'FOO_BAR_PLACEHOLD'
    nom_res = '10000 km'
    calendar='julian'
...
...
...

the corresponding variable list:

 $] cat fre/tests/test_files/varlist_local_target_vars_differ 
{
    "sosV2": "sos",
    "sos": "sosTYPO"
}

if one consults the help menu of the fre cmor CLI tool for... e.g. run, we see the following description for it's varlist argument:

 $] module load fre/test
 $] fre cmor run --help
Usage: fre cmor run [OPTIONS]

  Rewrite climate model output files with CMIP-compliant metadata for down-
  stream publishing

Options:
  -d, --indir TEXT         directory containing netCDF files. keys specified
                           in json_var_list are local variable names used for
                           targeting specific files in this directory
                           [required]
  -l, --varlist TEXT       path pointing to a json file containing directory
                           of key/value pairs. the keys are the 'local' names
                           used in the filename, and the values pointed to by
                           those keys are strings representing the name of the
                           variable contained in targeted files. the key and
                           value are often the same, but it is not required.
                           [required]
...
...
...

which clarifies what the different key/value pairs are in varlist_local_target_vars_differ. The first pair is "sosV2": "sos",, this implies that sosV2 will be used to find files, and sos is the variable name expected to be in any files found using sosV2.

this behavior should be adjusted- it's quite rare to see a postprocessed file with a file name pointing to a variable name differing to what is attached to the array contained in the file.

behavior changes for fre.cmor:

  • if the variable name in the filename is not equivalent to the name of the array within the file, fre.cmor will error with a helpful error message
  • the variable lists will still have entries with key/value pairs
  • the key is still the one used to find target files
  • the value will now correspond to a variable entry in e.g. cmip7-cmor-tables after stripping off the brand info.
  • to summarize what key/value pairs will become: "modelers_varname_choice":"mip_table_varname"
  • we will still have many cases where the key and value are identical

todo:

  • adjust variable list making behavior to match above description
  • adjust fre cmor run to work with the new variable list
  • adjust fre cmor yaml to work with the new variable list
  • update tests for the new behavior, both successful scenarios and failure scenarios

Metadata

Metadata

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions