How can I run zppy with more recent versions of the packages I'm using?

[forsyth2]

zppy is most often used within the latest E3SM Unified environment. However, you may find yourself needing a version of a tool that hasn't been released via E3SM Unified yet. In these cases, zppy has some parameters that can help.

Task-package alignment

First, which tasks use which packages?

task	packages used
[climo]	NCO
[ts]	NCO, optionally e3sm_to_cmip (#467 would give e3sm_to_cmip its own task)
[tc_analysis]	misc. functions defined in E3SM Unified
[e3sm_diags]	e3sm_diags
[mpas_analysis]	MPAS-Analysis
[global_time_series]	This is a built-in collection of scripts rather than a separate package (#398 would make global_time_series its own package)
[ilamb]	ILAMB

Loading newer versions

Now, here are ways to load newer versions of certain packages:

e3sm_diags package:

# `cd` to e3sm_diags directory (`git clone` it if you don't have this directory)
$ git checkout main
$ git fetch upstream
$ git reset --hard upstream/main
$ git log # Should match https://github.com/E3SM-Project/e3sm_diags/commits/main
$ mamba clean --all
$ mamba env create -f conda-env/dev.yml -n e3sm_diags_<date>
$ conda activate e3sm_diags_<date>
$ pip install .

Then, set environment_commands="source <conda_path>; conda activate e3sm_diags_<date>" under the [e3sm_diags] task in the zppy cfg. (Example <conda_path>: /home/ac.forsyth2/miniconda3/etc/profile.d/conda.sh)

e3sm_to_cmip package:

# `cd` to e3sm_to_cmip directory (`git clone` it if you don't have this directory)
$ git checkout master
$ git fetch upstream
$ git reset --hard upstream/master
$ git log # Should match https://github.com/E3SM-Project/e3sm_to_cmip/commits/master
$ mamba clean --all
$ mamba env create -f conda-env/dev.yml -n e3sm_to_cmip_<date>
$ conda activate e3sm_to_cmip_<date>
$ pip install .

Then, set e3sm_to_cmip_environment_commands="source <conda_path>; conda activate e3sm_to_cmip_<date>" under the [ts] task in the zppy cfg. (Example <conda_path>: /home/ac.forsyth2/miniconda3/etc/profile.d/conda.sh). (The reason this package needs its own specific environment_commands parameter is because most of the ts task doesn't need it -- and instead needs a different package, NCO).

NCO package:

$ mamba clean --all
$ mamba env create -f conda/dev.yml -n zppy_dev_with_nco
$ conda activate zppy_dev_with_nco
$ pip install .
$ mamba install -y nco=<version number>
$ ncks --version

Then, set environment_commands="source <conda_path>; conda activate zppy_dev_with_nco" under the [climo] and/or [ts] task in the zppy cfg. (Example <conda_path>: /home/ac.forsyth2/miniconda3/etc/profile.d/conda.sh).

Further reading

See #400 (comment) and #400 (comment) for more on this topic.