--update-filetype traces in the experiment

[mandresm]

@alxthor asked the following question via email and personal meeting:

Hi Miguel,

I looked in the specific experiments for the flag

update-filetypes

in the experiment/config folder with all the finished_config.yamls for every year.

This flag appears nowhere.

Yet, there are multiple binaries in the experiment/bin/fesom folder and, in one of the two experiments I'm looking at, binaries in the remaining run/work folders change.

So can you please check if adding this flag is then actually tracable?

Thanks in advance,
Alex

The question is whether ESM-Tools keeps a trace in the finished_config.yaml of runs that have been submitted with the esm_runscripts flag --update-filetypes. For example, esm_runscripts --update-filetypes bin. His idea is that, if it is traceable he can understand in an old long experiment whether new binaries were introduced with this flag during resubmissions.

[mandresm]

Hi @alxthor,

I can confirm that the flag --update-filetypes leaves a trace in the finished_config.yaml file of the run that was resubmitted with that flag. Here the explanation of my tests:

So I run a regular awicm3-v3.1 experiment (/work/ab0995/a270152/alex) with runs:
run_20000105-20000105
run_20000106-20000106
run_20000107-20000107
run_20000108-20000108
run_20000109-20000109

That experiment used the binaries in /work/ab0995/a270152/model_codes/awicm3-v3.1/bin

Then I moved the fesom.x to a fesom.x_old, add a comment to the fesom source code and recompile. Now I have two different fesom binary versions, the fesom.x with the extra comment and fesom.x_old that was used on the first part of the test.

For the second part of the test I ran run_20000110-20000110 without --update-filtypes. As expected run_20000110-20000110/work/fesom is not different from model_codes/awicm3-v3.1/bin/fesom.x_old but it's different from fesom.x.

In the third part of the test I am running run_20000111-20000111 and run_20000112-20000112 (extended the final date until 20000113) with --update-filetypes bin.

Again, the expected behavior: now the fesom binaries in the 2 work folders (run_20000111-20000111/work/fesom and run_20000112-20000112/work/fesom) correspond to the new model_codes/awicm3-v3.1/bin/fesom.x.

If you do grep -r update-filetypes /work/ab0995/a270152/alex/config you'll find the trace here alex_finished_config.yaml_20000111-20000111, and only on that one, but not on the following one (i.e. it tells you exactly in which run did you copy the new binary).

I will improve the documentation for the use of the flag: for example to update more than one file type (bin and config) one needs to do --update-filetype bin config

Kind regards,
Miguel

[alxthor]

Thanks for checking this!
For completeness: /work/ab0995/a270152/alex/bin/fesom contains two binaries:

• fesom_20000110-20000110 with the timestamp of the identical binary /work/ab0995/a270152/model_codes/awicm3-v3.1/bin/fesom.x_old (14.12.23, before experiment was run). It was used up to and including run_20000110-20000110.
• fesom_20000111-20000111 identical to /work/ab0995/a270152/model_codes/awicm3-v3.1/bin/fesom.x again with their timestamps being identical as well (14.02.24, 09:35). Used since run_20000111-20000111

and also the link fesom to fesom_20000111-20000111 with a timestamp 14.02.24, 09:51 which is when run_20000112-20000112was run.