Skip to content

Commit b90af59

Browse files
forsyth2forsyth2
committed
Generated table
1 parent 94d30e2 commit b90af59

File tree

10 files changed

+96
-71
lines changed

10 files changed

+96
-71
lines changed

‎docs/source/v2.1/BGC/simulation_data/index.rst‎

Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,56 @@
1+
***************
2+
Simulation Data
3+
***************
4+
5+
The E3SMv2.1 simulation data is available on **ESGF** and **NERSC HPSS**.
6+
7+
The data is reformatted to conform to CMIP conventions and was submmited to the CMIP6 ESGF archive. (ESGF links are provided in the table below for published data).
8+
9+
Additionally, all native model output data has also been archived on **NERSC HPSS** using `zstash <https://e3sm-project.github.io/zstash>`_.
10+
11+
**If you have an account on NERSC**, you can retrieve the data locally or remotely using Globus.
12+
13+
To download simulation data locally on a NERSC machine: ::
14+
15+
zstash extract --hpss=<HPSS path below>
16+
17+
To download simulation data remotely using the zstash Globus interface: ::
18+
19+
zstash extract --hpss=globus://nersc/<HPSS path below>
20+
21+
or ::
22+
23+
zstash extract --hpss=globus://9cd89cfd-6d04-11e5-ba46-22000b92c6ec/<HPSS path below>
24+
25+
Note that the data management tool `zstash <https://github.com/E3SM-Project/zstash>`_ is available from the `E3SM-Unified <https://github.com/E3SM-Project/e3sm-unified>`_ conda environment. An example of retrieving all **eam.h0** (monthly atmosphere output files) between **years 0030 and 0049** for the v2_1.LR.piControl simulation at NERSC locally is demonstrated as below in two steps:
26+
27+
1. To activate E3SM-Unified environment by:
28+
::
29+
30+
source /global/common/software/e3sm/anaconda_envs/load_latest_e3sm_unified_pm-cpu.sh
31+
32+
2. To retrieve files with zstash command:
33+
::
34+
35+
zstash extract --hpss=/home/projects/e3sm/www/WaterCycle/E3SMv2_1/v2_1.LR.piControl "*.eam.h0.00[3-4]?-??.nc"
36+
37+
38+
For more information, refer to `zstash usage <https://e3sm-project.github.io/zstash/_build/html/master/usage.html#extract>`_.
39+
40+
41+
**If you do not have access to NERSC**, you can download simulation data directly through the NERSC HPSS
42+
`web interface <https://portal.nersc.gov/archive/home/projects/e3sm/www/WaterCycle/E3SMv2_1>`_.
43+
Note that this will be slow and inefficient since you'll have to download the tar files.
44+
45+
**v2_1.LR** simulations data has been archived on NERSC HPSS under: ::
46+
47+
/home/projects/e3sm/www/BGC/E3SMv2_1
48+
49+
50+
Original run scripts can be found `here <https://github.com/E3SM-Project/e3sm_data_docs/tree/main/run_scripts/v2.1/original>`_.
51+
52+
.. toctree::
53+
:maxdepth: 2
54+
:caption: Contents:
55+
56+
simulation_table

‎docs/source/v2.1/BGC/simulation_data/simulation_data.rst‎

Lines changed: 0 additions & 56 deletions
This file was deleted.

‎docs/source/v2.1/BGC/simulation_data/simulation_table.rst‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
**********************************
2+
v2.1 BGC simulation table
3+
**********************************
4+
5+
+-------------------------------------------------------------------+-----------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------+
6+
| Simulation | Data Size (TB) | ESGF Links | HPSS Path |
7+
+===================================================================+=================+==================================================================================================================================================================================================================================================================================================================================================================================================================+==================================================================================+
8+
| **LR > SPINUP** | | | |
9+
+-------------------------------------------------------------------+-----------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------+
10+
| CBGCv2.r05.chrysalis.I20TRGSWCNP | 3 | | /home/projects/e3sm/www/BGC/E3SMv2_1/CBGCv2.r05.chrysalis.I20TRGSWCNP |
11+
+-------------------------------------------------------------------+-----------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------+
12+
| **LR > HISTORICAL** | | | |
13+
+-------------------------------------------------------------------+-----------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------+
14+
| v2.LR.BGC-LNDATM.CONTRL | 11 | | /home/projects/e3sm/www/BGC/E3SMv2_1/v2.LR.BGC-LNDATM.CONTRL |
15+
+-------------------------------------------------------------------+-----------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------+
16+
| **LR > SSP** | | | |
17+
+-------------------------------------------------------------------+-----------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------+
18+
| v2.LR.BGC-LNDATM.SSP370 | 6 | | /home/projects/e3sm/www/BGC/E3SMv2_1/v2.LR.BGC-LNDATM.SSP370 |
19+
+-------------------------------------------------------------------+-----------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------+

‎docs/source/v2.1/index.rst‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,4 +5,5 @@ v2.1
55
:maxdepth: 2
66
:caption: Contents:
77

8+
BGC/index
89
WaterCycle/index

‎run_scripts/v2.1/original/run.CBGCv2.r05.chrysalis.I20TRGSWCNP.sh‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
# Placeholder for actual run script

‎run_scripts/v2.1/original/run.v2.LR.BGC-LNDATM.CONTRL.sh‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
# Placeholder for actual run script

‎run_scripts/v2.1/original/run.v2.LR.BGC-LNDATM.SSP370.sh‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
# Placeholder for actual run script

‎utils/generate_html.bash‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
pr_num=52
1+
pr_num=56
22

33
# Chrysalis
44
#destination_dir=/lcrc/group/e3sm/public_html/diagnostic_output/$USER/data_docs_${pr_num}
@@ -7,7 +7,7 @@ pr_num=52
77
destination_dir=/global/cfs/cdirs/e3sm/www/$USER/data_docs_${pr_num}
88
web_page="https://portal.nersc.gov/cfs/e3sm/$USER/data_docs_${pr_num}/html/"
99

10-
#python generate_tables.py
10+
python generate_tables.py
1111
cd ../docs/ && make html
1212
rm -rf ls ${destination_dir}
1313
mv _build ${destination_dir}

‎utils/generate_tables.py‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -178,6 +178,8 @@ def read_simulations(csv_file):
178178
simulation_dict = {}
179179
for i in range(len(header)):
180180
label = header[i]
181+
if len(row) != len(header):
182+
raise RuntimeError(f"header has {len(header)} labels, but row has {len(row)} entries")
181183
simulation_dict[label] = row[i].strip()
182184
model_version_name = simulation_dict["model_version"]
183185
group_name = simulation_dict["group"]

‎utils/simulations_v2_1.csv‎

Lines changed: 13 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,13 @@
1-
model_version, group, resolution, category, simulation_name, machine, checksum, experiment, ensemble_num, cmip_only, node,
2-
v2.1, BGC, LR, DECK v2_1.LR.land-historical-transient,
3-
v2.1, BGC, LR, DECK v2.LR.BGC-LNDATM.SPINUPp,
4-
v2.1, BGC, LR, DECK v2.1.LR.land-atm-historical-trainsient,
5-
v2.1, WaterCycle, LR, DECK, v2_1.LR.piControl, chrysalis, ebc5565e182e8b1cc887c185ca3eb144, piControl, 1, cmip_only, cels.anl,
6-
v2.1, WaterCycle, LR, DECK, v2_1.LR.1pctCO2_0101, chrysalis, b7676ccc791941d1bdadde05c59d70b0, 1pctCO2, 1, cmip_only, cels.anl,
7-
v2.1, WaterCycle, LR, DECK, v2_1.LR.abrupt-4xCO2_0101, chrysalis, fe50274534774678d55cb320412d1667, abrupt-4xCO2, 1, cmip_only, cels.anl,
8-
v2.1, WaterCycle, LR, DECK, v2_1.LR.amip_0101, chrysalis, 832ca9c9db8b4988a109f8c5872b0fe7, amip, 1, cmip_only, cels.anl,
9-
v2.1, WaterCycle, LR, Historical, v2_1.LR.historical_0101, chrysalis, 43efe5ba9216b535be21bd8474b15c22, historical, 1, cmip_only, cels.anl,
10-
v2.1, WaterCycle, LR, Historical, v2_1.LR.historical_0151, chrysalis, , historical, 2, cmip_only, cels.anl,
11-
v2.1, WaterCycle, LR, Historical, v2_1.LR.historical_0201, chrysalis, , historical, 3, cmip_only, cels.anl,
12-
v2.1, WaterCycle, LR, Historical, v2_1.LR.historical_0251, chrysalis, , historical, 4, cmip_only, cels.anl,
13-
v2.1, WaterCycle, LR, Historical, v2_1.LR.historical_0301, chrysalis, , historical, 5, cmip_only, cels.anl,
1+
model_version, group, resolution, category, simulation_name, machine, checksum, experiment, ensemble_num, cmip_only, node,
2+
v2.1, BGC, LR, SPINUP, CBGCv2.r05.chrysalis.I20TRGSWCNP, chrysalis, , , , , ,
3+
v2.1, BGC, LR, HISTORICAL, v2.LR.BGC-LNDATM.CONTRL, chrysalis, , , , , ,
4+
v2.1, BGC, LR, SSP, v2.LR.BGC-LNDATM.SSP370, chrysalis, , , , , ,
5+
v2.1, WaterCycle, LR, DECK, v2_1.LR.piControl, chrysalis, ebc5565e182e8b1cc887c185ca3eb144, piControl, 1, cmip_only, cels.anl,
6+
v2.1, WaterCycle, LR, DECK, v2_1.LR.1pctCO2_0101, chrysalis, b7676ccc791941d1bdadde05c59d70b0, 1pctCO2, 1, cmip_only, cels.anl,
7+
v2.1, WaterCycle, LR, DECK, v2_1.LR.abrupt-4xCO2_0101, chrysalis, fe50274534774678d55cb320412d1667, abrupt-4xCO2, 1, cmip_only, cels.anl,
8+
v2.1, WaterCycle, LR, DECK, v2_1.LR.amip_0101, chrysalis, 832ca9c9db8b4988a109f8c5872b0fe7, amip, 1, cmip_only, cels.anl,
9+
v2.1, WaterCycle, LR, Historical, v2_1.LR.historical_0101, chrysalis, 43efe5ba9216b535be21bd8474b15c22, historical, 1, cmip_only, cels.anl,
10+
v2.1, WaterCycle, LR, Historical, v2_1.LR.historical_0151, chrysalis, , historical, 2, cmip_only, cels.anl,
11+
v2.1, WaterCycle, LR, Historical, v2_1.LR.historical_0201, chrysalis, , historical, 3, cmip_only, cels.anl,
12+
v2.1, WaterCycle, LR, Historical, v2_1.LR.historical_0251, chrysalis, , historical, 4, cmip_only, cels.anl,
13+
v2.1, WaterCycle, LR, Historical, v2_1.LR.historical_0301, chrysalis, , historical, 5, cmip_only, cels.anl,

0 commit comments

Comments
 (0)