Could we allow components the ability to define testmods?

[glemieux]

In our on going efforts to reduce the need to coordinate interface updates between FATES and ELM/CLM, one of the issues that we often have to deal with is coordinating an update to the fates testmods when we make changes to the FATES history variables. Ideally we wouldn't have to update the testmods under the host land model whenever we make a change that is isolated to FATES. Would it be possible to allow components the ability to define and own testmods in the external component repos?

[glemieux]

Note that one of the things that contributes to this issue is that in the FATES testmods, we often clear the history tapes to remove the HLM history and add specific FATES history output.

[ekluzek]

I think this just mainly means editing the files in for example here for CESM:

CIME/data/config/cesm/config_files.xml

With something similar for E3SM, but it's in the E3SM cime_config side. I assume the format is similar, but may be different and obviously would have different rules on what components beneath the top level could do.

The sticky thing might be on how to define FATES as a component. Since, component currently refers to the top level component -- so it might require something like having both "clm" and "clmfates" as components in cime? However, it seems like in terms of finding testmod files -- just having a "fates" prefix in the testmod name (as in fates-fates_specific_testmod in a test definition instead of clm-clm_specific_testmod) would be sufficient.

In that same file there are other things that FATES could take advantage of in terms of a testlist, or SystemTests that would be just for FATES.

I don't see any downsides of doing this for CESM. It just means that the FATES checkout will manage these things. So as long as FATES doesn't break anything for CESM or CTSM testing -- it should be fine. CTSM is what's responsible for bringing in a new FATES tag -- so that just means it on us in CTSM to make sure we don't break anything for CESM. And effectively this wouldn't be any different than what we do now. Also in my reading of "find_test_mods" it looks like it allows any name for a "component" so putting "fates" here should be fine.

So @glemieux I think you should try doing this for fates testmods and just see if it works or not.

But, @jedwards4b @billsacks @fischer-ncar or others what do you think about this idea? Is it ok to add a fates specific testmod directory that's listed in config_files.xml for cesm?

[billsacks]

Before thinking about the feature addition that @ekluzek suggests, I wonder if you can achieve what you want through the include_user_mods mechanism?: Could you keep a testmod directory with the given name under CTSM (e.g., cime_config/testdefs/testmods_dirs/clm/Fates) but put an include_user_mods there with a path that pulls in things from somewhere inside the fates directory? A possible advantage of that is that it would also provide an automatic way to blend settings from both CTSM and FATES.

How far would that get you towards what you want? How much of a pain do you feel it would be to have directories defined at these two levels? And is the possible advantage I mentioned above actually an advantage, or not really useful in practice?

I haven't thought enough about @ekluzek 's suggestion to weigh in on it yet, but just wanted to explore this possible alternative that I think you could do without any code changes.

[samsrabin]

We could even potentially just do softlinks!

[ekluzek]

Yes, that would be a simpler thing to start with as it only requires a change in CTSM and ELM. @glemieux maybe you should try both out and see if you get a feel of what's preferred.

And @billsacks maybe this is something we should discuss in CSEG (for the potential CIME change)? I could see where other components with lots of subcomponts (like CAM) might want/need something like this.

[ekluzek]

@samsrabin softlinks are allowed in git, but we haven't done it. It makes me slightly nervous -- but I can't think of any reason to not try it out.

@glemieux I suppose that would be a third option you could try.