Skip to content

Add v2.1 data #50

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Nov 19, 2024
Merged

Add v2.1 data #50

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
2212473
Add v2.1 data
forsyth2 Nov 13, 2024
d672c3c
Updates on Perlmutter
forsyth2 Nov 13, 2024
63226f1
Address comments
forsyth2 Nov 19, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions docs/source/index.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,10 +16,12 @@ simulations.
:caption: Contents:

v2/index
v2.1/index
SCREAMv0/index
SCREAMv1/index



.. Indices and tables
.. ==================
..
Expand Down
16 changes: 16 additions & 0 deletions docs/source/v2.1/WaterCycle/index.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
E3SMv2.1 (Water Cycle)
====================

The simulation compaign for E3SMv2 (Water Cycle) was performed with two
Copy link
Collaborator

@chengzhuzhang chengzhuzhang Nov 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This needs to be modified as The simulation campaign for E3SMv2.1 was performed with Low Resolution (LR) configuration

separate model configurations:

* **v2_1.LR** (lower resolution)

If you use data from this simulation campaign, please cite the relevant overview
manuscripts.
Copy link
Collaborator

@chengzhuzhang chengzhuzhang Nov 18, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please cite Smith et al. 2024, GMD as the overview reference paper
https://gmd.copernicus.org/preprints/gmd-2024-149/


.. toctree::
:maxdepth: 2
:caption: Contents:

simulation_data/index
56 changes: 56 additions & 0 deletions docs/source/v2.1/WaterCycle/simulation_data/index.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
***************
Simulation Data
***************

The E3SMv2.1 simulation data is available on **ESGF** and **NERSC HPSS**.

The preferred retrieval method is **ESGF**. Native output is available at `ESGF <https://esgf-node.cels.anl.gov/search/e3sm/?model_version=2_1>`_, and a subset of the data is reformatted to conform to CMIP conventions and submmited to the CMIP6 ESGF archive (ESGF links are provided in the table below for published data).
Copy link
Collaborator Author

@forsyth2 forsyth2 Nov 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checked link works ✅

Copy link
Collaborator Author

@forsyth2 forsyth2 Nov 19, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually it's https://esgf-node.cels.anl.gov/search?project=CMIP6&activeFacets=%7B%22source_id%22%3A%22E3SM-2-1%22%7D that works; that edit didn't seem to make it into the last commit.


Additionally, all native model output data has also been archived on **NERSC HPSS** using `zstash <https://e3sm-project.github.io/zstash>`_.

**If you have an account on NERSC**, you can retrieve the data locally or remotely using Globus.

To download simulation data locally on a NERSC machine: ::

zstash extract --hpss=<HPSS path below>

To download simulation data remotely using the zstash Globus interface: ::

zstash extract --hpss=globus://nersc/<HPSS path below>

or ::

zstash extract --hpss=globus://9cd89cfd-6d04-11e5-ba46-22000b92c6ec/<HPSS path below>
Copy link
Collaborator

@chengzhuzhang chengzhuzhang Nov 18, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as above


Note that the data management tool `zstash <https://github.com/E3SM-Project/zstash>`_ is available from the `E3SM-Unified <https://github.com/E3SM-Project/e3sm-unified>`_ conda environment. An example of retrieving all **eam.h0** (monthly atmosphere output files) between **years 0030 and 0049** for the v2_1.LR.piControl simulation at NERSC locally is demonstrated as below in two steps:

1. To activate E3SM-Unified environment by:
::

source /global/common/software/e3sm/anaconda_envs/load_latest_e3sm_unified_pm-cpu.sh

2. To retrieve files with zstash command:
::

zstash extract --hpss=/home/projects/e3sm/www/WaterCycle/E3SMv2_1/LR/v2.LR.piControl "*.elm.h0.00[3-4]?-??.nc"
Copy link
Collaborator

@chengzhuzhang chengzhuzhang Nov 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The path is not correct. In addition, there is a type elm (_should be eam) inheriting from v2.



For more information, refer to `zstash usage <https://e3sm-project.github.io/zstash/_build/html/master/usage.html#extract>`_.


**If you do not have access to NERSC**, you can download simulation data directly through the NERSC HPSS
`web interface <https://portal.nersc.gov/archive/home/projects/e3sm/www/WaterCycle/E3SMv2_1>`_.
Note that this will be slow and inefficient since you'll have to download the tar files.

**v2_1.LR** simulations data has been archived on NERSC HPSS under: ::

/home/projects/e3sm/www/WaterCycle/E3SMv2_1
Copy link
Collaborator Author

@forsyth2 forsyth2 Nov 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checked HPSS path ✅



Original run scripts can be found `here <https://github.com/E3SM-Project/e3sm_data_docs/tree/main/run_scripts/v2.1/original>`_.
Copy link
Collaborator Author

@forsyth2 forsyth2 Nov 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This link will exist after this PR is merged.


.. toctree::
:maxdepth: 2
:caption: Contents:

simulation_table
29 changes: 29 additions & 0 deletions docs/source/v2.1/WaterCycle/simulation_data/simulation_table.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
**********************************
v2.1 WaterCycle simulation table
**********************************

+-------------------------------------------------------------------+-----------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------+
| Simulation | Data Size (TB) | ESGF Links | HPSS Path |
+===================================================================+=================+==================================================================================================================================================================================================================================================================================================================================================================================================================+==================================================================================+
| **LR > DECK** | | | |
+-------------------------------------------------------------------+-----------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------+
| v2_1.LR.piControl | 45 | `CMIP <https://esgf-node.cels.anl.gov/search/?project=CMIP6&activeFacets=%7B%22source_id%22%3A%22E3SM-2-1%22%2C%22experiment_id%22%3A%22piControl%22%2C%22variant_label%22%3A%22r1i1p1f1%22%7D>`_ | /home/projects/e3sm/www/WaterCycle/E3SMv2_1/v2_1.LR.piControl |
+-------------------------------------------------------------------+-----------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------+
| v2_1.LR.1pctCO2_0101 | 14 | `CMIP <https://esgf-node.cels.anl.gov/search/?project=CMIP6&activeFacets=%7B%22source_id%22%3A%22E3SM-2-1%22%2C%22experiment_id%22%3A%221pctCO2%22%2C%22variant_label%22%3A%22r1i1p1f1%22%7D>`_ | /home/projects/e3sm/www/WaterCycle/E3SMv2_1/v2_1.LR.1pctCO2_0101 |
+-------------------------------------------------------------------+-----------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------+
| v2_1.LR.abrupt-4xCO2_0101 | 14 | `CMIP <https://esgf-node.cels.anl.gov/search/?project=CMIP6&activeFacets=%7B%22source_id%22%3A%22E3SM-2-1%22%2C%22experiment_id%22%3A%22abrupt-4xCO2%22%2C%22variant_label%22%3A%22r1i1p1f1%22%7D>`_ | /home/projects/e3sm/www/WaterCycle/E3SMv2_1/v2_1.LR.abrupt-4xCO2_0101 |
+-------------------------------------------------------------------+-----------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------+
| v2_1.LR.amip_0101 | 1 | `CMIP <https://esgf-node.cels.anl.gov/search/?project=CMIP6&activeFacets=%7B%22source_id%22%3A%22E3SM-2-1%22%2C%22experiment_id%22%3A%22amip%22%2C%22variant_label%22%3A%22r1i1p1f1%22%7D>`_ | /home/projects/e3sm/www/WaterCycle/E3SMv2_1/v2_1.LR.amip_0101 |
+-------------------------------------------------------------------+-----------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------+
| **LR > Historical** | | | |
+-------------------------------------------------------------------+-----------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------+
| v2_1.LR.historical_0101 | 14 | `CMIP <https://esgf-node.cels.anl.gov/search/?project=CMIP6&activeFacets=%7B%22source_id%22%3A%22E3SM-2-1%22%2C%22experiment_id%22%3A%22historical%22%2C%22variant_label%22%3A%22r1i1p1f1%22%7D>`_ | /home/projects/e3sm/www/WaterCycle/E3SMv2_1/v2_1.LR.historical_0101 |
+-------------------------------------------------------------------+-----------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------+
| v2_1.LR.historical_0151 | 15 | `CMIP <https://esgf-node.cels.anl.gov/search/?project=CMIP6&activeFacets=%7B%22source_id%22%3A%22E3SM-2-1%22%2C%22experiment_id%22%3A%22historical%22%2C%22variant_label%22%3A%22r2i1p1f1%22%7D>`_ | /home/projects/e3sm/www/WaterCycle/E3SMv2_1/v2_1.LR.historical_0151 |
+-------------------------------------------------------------------+-----------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------+
| v2_1.LR.historical_0201 | 15 | `CMIP <https://esgf-node.cels.anl.gov/search/?project=CMIP6&activeFacets=%7B%22source_id%22%3A%22E3SM-2-1%22%2C%22experiment_id%22%3A%22historical%22%2C%22variant_label%22%3A%22r3i1p1f1%22%7D>`_ | /home/projects/e3sm/www/WaterCycle/E3SMv2_1/v2_1.LR.historical_0201 |
+-------------------------------------------------------------------+-----------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------+
| v2_1.LR.historical_0251 | 15 | `CMIP <https://esgf-node.cels.anl.gov/search/?project=CMIP6&activeFacets=%7B%22source_id%22%3A%22E3SM-2-1%22%2C%22experiment_id%22%3A%22historical%22%2C%22variant_label%22%3A%22r4i1p1f1%22%7D>`_ | /home/projects/e3sm/www/WaterCycle/E3SMv2_1/v2_1.LR.historical_0251 |
+-------------------------------------------------------------------+-----------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------+
| v2_1.LR.historical_0301 | 15 | `CMIP <https://esgf-node.cels.anl.gov/search/?project=CMIP6&activeFacets=%7B%22source_id%22%3A%22E3SM-2-1%22%2C%22experiment_id%22%3A%22historical%22%2C%22variant_label%22%3A%22r5i1p1f1%22%7D>`_ | /home/projects/e3sm/www/WaterCycle/E3SMv2_1/v2_1.LR.historical_0301 |
+-------------------------------------------------------------------+-----------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------+
8 changes: 8 additions & 0 deletions docs/source/v2.1/index.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
v2.1
==========================================

.. toctree::
:maxdepth: 2
:caption: Contents:

WaterCycle/index
Loading