Trouble with test that runs case.setup in shell_commands

[samsrabin]

(CIME tag cime6.0.100.)

I've designed a test that needs to do some pre-processing before submission if a certain CLM variable, flanduse_timeseries, is set in CaseDocs/lnd_in. The script that checks for the flanduse_timeseries value (and then does the pre-processing if needed) is called during shell_commands. However, it appears that shell_commands is called before case.setup, so the check for flanduse_timeseries happens too soon—it will never be set at that point.

Calling case.setup at the top of shell_commands results in the correct flanduse_timeseries value being read, but the test then fails during its SETUP step. Here's the create_test output, with my test specification abbreviated testspec for readability:

Testnames: ['testspec']
create_test will do up to 1 tasks simultaneously
create_test will use up to 45 cores simultaneously
Creating test directory /glade/scratch/samrabin/testspec.20230404_174150_pdkb4z
RUNNING TESTS:
  testspec
Starting CREATE_NEWCASE for test testspec with 1 procs
Finished CREATE_NEWCASE for test testspec in 58.607306 seconds (PASS)
Starting XML for test testspec with 1 procs
Finished XML for test testspec in 0.523778 seconds (PASS)
Starting SETUP for test testspec with 1 procs
Finished SETUP for test testspec in 0.603841 seconds (FAIL). [COMPLETED 1 of 1]
    Case dir: /glade/scratch/samrabin/testspec.20230404_174150_pdkb4z
    Errors were:
        ERROR: inconsistent test name in parse_test_status: 'testspec' != 'testspec.20230404_174150_pdkb4z'

I tried adding these to the bottom of shell_commands, but they didn't help:

• ./case.setup --clean (no apparent effect)
• ./xmlchange TEST_ARGV="" (errors I think because the XML file containing TEST_ARGV, env_test.xml, doesn't exist during shell_commands)
• mkdir tmp_namelists; ./case.cmpgen_namelists -g --baseline-root ./tmp_namelists (case.cmpgen_namelists does nothing without case.setup being called first)

Is there any way to work around this? For example:

• A way to build the namelists without calling case.setup?
• A way to fully "undo" the results of case.setup, or at least to undo whatever it did to cause the "inconsistent test name" error?

Thanks in advance!

[rljacob]

"case.setup --reset" will undo the action of "case.setup"

[samsrabin]

Unfortunately --reset gives the same result as --clean, which makes sense—it's basically just a shortcut for case.setup --clean; case.setup, right?

[jedwards4b]

right

[billsacks]

@samsrabin - without having looked carefully at the details, I think the easiest and most robust way to achieve what you want to do here is to make a custom test type, which lets you do custom actions at various stages in a python framework. This is probably easiest to show in person or by zoom call... would you like to schedule a time to go over this? Or if you want to look at some examples yourself, you can look in https://github.com/ESMCI/cime/tree/master/CIME/SystemTests and https://github.com/ESCOMP/CTSM/tree/master/cime_config/SystemTests - the latter for CTSM-specific tests, which I think is where yours would belong. See also https://github.com/ESCOMP/CTSM/blob/master/cime_config/config_tests.xml.

By the way: thank you very much for working on adding a new test for the functionality you're adding!

[samsrabin]

Thanks, Bill! I'll see what I can figure out from where you pointed me, and if I need help I'll email you.