How can I sync ILAMB Data?

[forsyth2]

ILAMB data can be synced using the following process:

On Chrysalis:

# Load Unified environment
$ cd /lcrc/group/e3sm/public_html/diagnostics/ # [sync > public_diags] on https://github.com/E3SM-Project/mache/blob/main/mache/machines/chrysalis.cfg
$ ilamb-fetch # Updates the `ilamb_data` directory
$ mache sync diags from chrysalis -u <username> # Syncs to `/lcrc/group/e3sm/diagnostics` [diagnostics > base_path] on https://github.com/E3SM-Project/mache/blob/main/mache/machines/chrysalis.cfg

Then, once the data has been fetch and synced on Chrysalis, it is easy to sync to other machines.
Perlmutter:

# Load Unified environment
$ mache sync diags from chrysalis -u <Chrysalis username> # Syncs to `/global/cfs/cdirs/e3sm/diagnostics` [diagnostics > base_path] on https://github.com/E3SM-Project/mache/blob/main/mache/machines/pm-cpu.cfg

Compy:

# Load Unified environment
$ mache sync diags from chrysalis -u <Chrysalis username> # Syncs to `/compyfs/diagnostics` [diagnostics > base_path] on https://github.com/E3SM-Project/mache/blob/main/mache/machines/compy.cfg

[forsyth2]

This issue came up in #538 -- we needed to run ilamb-fetch to use the latest ILAMB data.

[chengzhuzhang]

On permultter I saw:

chengzhu@perlmutter:login22:/global/cfs/cdirs/e3sm/diagnostics> ll
total 43
dr-xrwxr-x  7 sbaldwin e3sm 16384 Aug  9  2018 amwg
-rwxrwxr-x  1 xylar    e3sm  2439 Aug 28  2018 updatescript
drwxrwxr-x  4 milena   e3sm  4096 Mar 28  2019 a-prime
drwxrwxr-x  3 xylar    e3sm  4096 Apr  8  2020 test_output
drwxrwxr-x  3 xylar    e3sm  4096 Apr 23  2022 ilamb_data
drwxrwxr-x  9 milena   e3sm  4096 Apr 23  2022 observations
drwxrwxr-x  7 xylar    e3sm  4096 Apr 23  2022 cmip6-cmor-tables
drwxrwxr-x  5 xylar    e3sm  4096 May  5  2023 e3sm_to_cmip_data
drwxrwxr-x  4 xylar    e3sm  4096 Nov 15 10:17 diagnostics
drwxrwxr-x  2 xylar    e3sm  4096 Nov 15 10:24 maps
drwxrwxr-x  6 xylar    e3sm  4096 Nov 15 10:24 mpas_analysis
drwxrwxr-x 39 forsyth  e3sm  4096 Jan 10 12:11 DATA

The DATA directory somehow in the wrong place?? I'm running mache sync diags from chrysalis again.

[forsyth2]

Ok, thanks. What directory are you running the mache sync diags from chrysalis command from? Does that matter?

[chengzhuzhang]

I'm at /global/cfs/cdirs/e3sm/diagnostics, but I don't think directory matters, when running mache sycn