Skip to content

Add v1 cryosphere and BGC data #64

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Oct 14, 2025
Merged

Add v1 cryosphere and BGC data #64

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
9fc035e
Add v1 cryo data
forsyth2 Aug 25, 2025
9654b8b
Address cryo comments
forsyth2 Sep 25, 2025
c46b6d6
Initial symlink work for BGC
forsyth2 Sep 25, 2025
2004d3d
Add v1 BGC data
forsyth2 Sep 26, 2025
079fd0c
Address comments
forsyth2 Oct 14, 2025
e2bc84a
Refresh generated docs
forsyth2 Oct 14, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 2 additions & 4 deletions docs/source/v1/BGC/index.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,13 +23,11 @@ refer to `Coupled E3SM v1 Model Overview <https://e3sm.org/overview-paper-on-cou
* *Description of historical and future projection simulations by the global coupled E3SMv1.0 model as used in CMIP6* `doi:10.5194/gmd-15-3941-2022 <https://doi.org/10.5194/gmd-15-3941-2022>`_
* *Ensemble Spread Behavior in Coupled Climate Models: Insights From the Energy Exascale Earth System Model Version 1 Large Ensemble* `doi:10.1029/2023MS003653 <https://doi.org/10.1029/2023MS003653>`_
* *The DOE E3SM Coupled Model Version 1: Description and Results at High Resolution* `doi:10.1029/2019MS001870 <https://doi.org/10.1029/2019MS001870>`_
* (TODO: add BGC-specific papers)
* *The DOE E3SM v1.1 Biogeochemistry Configuration: Description and Simulated Ecosystem-Climate Responses to Historical Changes in Forcing* `doi:10.1029/2019MS00176 <https://doi.org/10.1029/2019MS001766>`_

Experiments:

The datasets include the following experiments. Each case has a historical run and a ssp85 run.

(TODO: add description to the top-level bullet points below)
The datasets include the following experiments. Each case has a historical run and a ssp85 run. CTC and ETA are two approaches to modeling soil biogeochemistry. B and R refer to biogeochemical CO\ :sub:`2`\ and radiative CO\ :sub:`2`\ respectively. C and D refer to constant and diagnostic respectively. Please see section 4.1 "Experimental Design and Overview of the Simulations" and Table 3 "E3SMv1.1-BGC Simulations Overview" of the `relevant paper <https://agupubs.onlinelibrary.wiley.com/doi/full/10.1029/2019ms001766>`_ for more information.

* CTC Control

Expand Down
3 changes: 1 addition & 2 deletions docs/source/v1/BGC/simulation_data/index.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,8 +30,7 @@ Note that the data management tool `zstash <https://github.com/E3SM-Project/zsta
2. To retrieve files with zstash command:
::

zstash extract --hpss=/home/projects/e3sm/www/BGC/E3SMv1/E3SM_1_1_piControl "*.cam.h0.00[3-4]?-??.nc"
# (TODO: run an actual zstash command to make sure it works as an example)
zstash ls --hpss=/home/projects/e3sm/www/BGC/E3SMv1/E3SM_1_1_piControl "*.cam.h3.18[5-6]?-??-??-?????.nc"


For more information, refer to `zstash usage <https://e3sm-project.github.io/zstash/_build/html/master/usage.html#extract>`_.
Expand Down
126 changes: 63 additions & 63 deletions docs/source/v1/BGC/simulation_data/simulation_table.rst
View file
Open in desktop

Large diffs are not rendered by default.

42 changes: 21 additions & 21 deletions docs/source/v1/Cryosphere/simulation_data/simulation_table.rst
View file
Open in desktop

Large diffs are not rendered by default.

42 changes: 21 additions & 21 deletions machine_readable_data/v1_BGC_simulations.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,21 +1,21 @@
Simulation,Data Size (TB),ESGF Links,HPSS Path
E3SM_1_1_piControl,23,,/home/projects/e3sm/www/BGC/E3SMv1/E3SM_1_1_piControl
E3SM_1_1_piControl-ext85yr,8,,/home/projects/e3sm/www/BGC/E3SMv1/E3SM_1_1_piControl-ext85yr
E3SM_1_1_hist-BCRC,16,,/home/projects/e3sm/www/BGC/E3SMv1/E3SM_1_1_hist-BCRC
E3SM_1_1_ssp585-BCRC,8,,/home/projects/e3sm/www/BGC/E3SMv1/E3SM_1_1_ssp585-BCRC
E3SM_1_1_hist-BCRD,16,,/home/projects/e3sm/www/BGC/E3SMv1/E3SM_1_1_hist-BCRD
E3SM_1_1_ssp585-BCRD,8,,/home/projects/e3sm/www/BGC/E3SMv1/E3SM_1_1_ssp585-BCRD
E3SM_1_1_hist-BDRC,16,,/home/projects/e3sm/www/BGC/E3SMv1/E3SM_1_1_hist-BDRC
E3SM_1_1_ssp585-BDRC,8,,/home/projects/e3sm/www/BGC/E3SMv1/E3SM_1_1_ssp585-BDRC
E3SM_1_1_hist-BDRD,16,,/home/projects/e3sm/www/BGC/E3SMv1/E3SM_1_1_hist-BDRD
E3SM_1_1_ssp585-BDRD,8,,/home/projects/e3sm/www/BGC/E3SMv1/E3SM_1_1_ssp585-BDRD
E3SM_1_1_ECA_piControl,22,,/home/projects/e3sm/www/BGC/E3SMv1/E3SM_1_1_ECA_piControl
E3SM_1_1_ECA_piControl-ext85yr,8,,/home/projects/e3sm/www/BGC/E3SMv1/E3SM_1_1_ECA_piControl-ext85yr
E3SM_1_1_ECA_hist-BCRC,15,,/home/projects/e3sm/www/BGC/E3SMv1/E3SM_1_1_ECA_hist-BCRC
E3SM_1_1_ECA_ssp585-BCRC,8,,/home/projects/e3sm/www/BGC/E3SMv1/E3SM_1_1_ECA_ssp585-BCRC
E3SM_1_1_ECA_hist-BCRD,15,,/home/projects/e3sm/www/BGC/E3SMv1/E3SM_1_1_ECA_hist-BCRD
E3SM_1_1_ECA_ssp585-BCRD,8,,/home/projects/e3sm/www/BGC/E3SMv1/E3SM_1_1_ECA_ssp585-BCRD
E3SM_1_1_ECA_hist-BDRC,15,,/home/projects/e3sm/www/BGC/E3SMv1/E3SM_1_1_ECA_hist-BDRC
E3SM_1_1_ECA_ssp585-BDRC,8,,/home/projects/e3sm/www/BGC/E3SMv1/E3SM_1_1_ECA_ssp585-BDRC
E3SM_1_1_ECA_hist-BDRD,15,,/home/projects/e3sm/www/BGC/E3SMv1/E3SM_1_1_ECA_hist-BDRD
E3SM_1_1_ECA_ssp585-BDRD,8,,/home/projects/e3sm/www/BGC/E3SMv1/E3SM_1_1_ECA_ssp585-BDRD
Simulation,Data Size (TB),ESGF Links,HPSS Path,HPSS URL
E3SM_1_1_piControl,23,,/home/projects/e3sm/www/BGC/E3SMv1/E3SM_1_1_piControl,https://portal.nersc.gov/archive/home/projects/e3sm/www/BGC/E3SMv1/E3SM_1_1_piControl
E3SM_1_1_piControl-ext85yr,8,,/home/projects/e3sm/www/BGC/E3SMv1/E3SM_1_1_piControl-ext85yr,https://portal.nersc.gov/archive/home/projects/e3sm/www/BGC/E3SMv1/E3SM_1_1_piControl-ext85yr
E3SM_1_1_hist-BCRC,16,,/home/projects/e3sm/www/BGC/E3SMv1/E3SM_1_1_hist-BCRC,https://portal.nersc.gov/archive/home/projects/e3sm/www/BGC/E3SMv1/E3SM_1_1_hist-BCRC
E3SM_1_1_ssp585-BCRC,8,,/home/projects/e3sm/www/BGC/E3SMv1/E3SM_1_1_ssp585-BCRC,https://portal.nersc.gov/archive/home/projects/e3sm/www/BGC/E3SMv1/E3SM_1_1_ssp585-BCRC
E3SM_1_1_hist-BCRD,16,,/home/projects/e3sm/www/BGC/E3SMv1/E3SM_1_1_hist-BCRD,https://portal.nersc.gov/archive/home/projects/e3sm/www/BGC/E3SMv1/E3SM_1_1_hist-BCRD
E3SM_1_1_ssp585-BCRD,8,,/home/projects/e3sm/www/BGC/E3SMv1/E3SM_1_1_ssp585-BCRD,https://portal.nersc.gov/archive/home/projects/e3sm/www/BGC/E3SMv1/E3SM_1_1_ssp585-BCRD
E3SM_1_1_hist-BDRC,16,,/home/projects/e3sm/www/BGC/E3SMv1/E3SM_1_1_hist-BDRC,https://portal.nersc.gov/archive/home/projects/e3sm/www/BGC/E3SMv1/E3SM_1_1_hist-BDRC
E3SM_1_1_ssp585-BDRC,8,,/home/projects/e3sm/www/BGC/E3SMv1/E3SM_1_1_ssp585-BDRC,https://portal.nersc.gov/archive/home/projects/e3sm/www/BGC/E3SMv1/E3SM_1_1_ssp585-BDRC
E3SM_1_1_hist-BDRD,16,,/home/projects/e3sm/www/BGC/E3SMv1/E3SM_1_1_hist-BDRD,https://portal.nersc.gov/archive/home/projects/e3sm/www/BGC/E3SMv1/E3SM_1_1_hist-BDRD
E3SM_1_1_ssp585-BDRD,8,,/home/projects/e3sm/www/BGC/E3SMv1/E3SM_1_1_ssp585-BDRD,https://portal.nersc.gov/archive/home/projects/e3sm/www/BGC/E3SMv1/E3SM_1_1_ssp585-BDRD
E3SM_1_1_ECA_piControl,22,,/home/projects/e3sm/www/BGC/E3SMv1/E3SM_1_1_ECA_piControl,https://portal.nersc.gov/archive/home/projects/e3sm/www/BGC/E3SMv1/E3SM_1_1_ECA_piControl
E3SM_1_1_ECA_piControl-ext85yr,8,,/home/projects/e3sm/www/BGC/E3SMv1/E3SM_1_1_ECA_piControl-ext85yr,https://portal.nersc.gov/archive/home/projects/e3sm/www/BGC/E3SMv1/E3SM_1_1_ECA_piControl-ext85yr
E3SM_1_1_ECA_hist-BCRC,15,,/home/projects/e3sm/www/BGC/E3SMv1/E3SM_1_1_ECA_hist-BCRC,https://portal.nersc.gov/archive/home/projects/e3sm/www/BGC/E3SMv1/E3SM_1_1_ECA_hist-BCRC
E3SM_1_1_ECA_ssp585-BCRC,8,,/home/projects/e3sm/www/BGC/E3SMv1/E3SM_1_1_ECA_ssp585-BCRC,https://portal.nersc.gov/archive/home/projects/e3sm/www/BGC/E3SMv1/E3SM_1_1_ECA_ssp585-BCRC
E3SM_1_1_ECA_hist-BCRD,15,,/home/projects/e3sm/www/BGC/E3SMv1/E3SM_1_1_ECA_hist-BCRD,https://portal.nersc.gov/archive/home/projects/e3sm/www/BGC/E3SMv1/E3SM_1_1_ECA_hist-BCRD
E3SM_1_1_ECA_ssp585-BCRD,8,,/home/projects/e3sm/www/BGC/E3SMv1/E3SM_1_1_ECA_ssp585-BCRD,https://portal.nersc.gov/archive/home/projects/e3sm/www/BGC/E3SMv1/E3SM_1_1_ECA_ssp585-BCRD
E3SM_1_1_ECA_hist-BDRC,15,,/home/projects/e3sm/www/BGC/E3SMv1/E3SM_1_1_ECA_hist-BDRC,https://portal.nersc.gov/archive/home/projects/e3sm/www/BGC/E3SMv1/E3SM_1_1_ECA_hist-BDRC
E3SM_1_1_ECA_ssp585-BDRC,8,,/home/projects/e3sm/www/BGC/E3SMv1/E3SM_1_1_ECA_ssp585-BDRC,https://portal.nersc.gov/archive/home/projects/e3sm/www/BGC/E3SMv1/E3SM_1_1_ECA_ssp585-BDRC
E3SM_1_1_ECA_hist-BDRD,15,,/home/projects/e3sm/www/BGC/E3SMv1/E3SM_1_1_ECA_hist-BDRD,https://portal.nersc.gov/archive/home/projects/e3sm/www/BGC/E3SMv1/E3SM_1_1_ECA_hist-BDRD
E3SM_1_1_ECA_ssp585-BDRD,8,,/home/projects/e3sm/www/BGC/E3SMv1/E3SM_1_1_ECA_ssp585-BDRD,https://portal.nersc.gov/archive/home/projects/e3sm/www/BGC/E3SMv1/E3SM_1_1_ECA_ssp585-BDRD
18 changes: 9 additions & 9 deletions machine_readable_data/v1_Cryosphere_simulations.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
Simulation,Data Size (TB),ESGF Links,HPSS Path
20190430.A_WCYCL1850-DIB-ISMF_CMIP6.ne30_oECv3wLI.control.cori-knl,9,,/home/projects/e3sm/www/Cryosphere/E3SMv1/20190430.A_WCYCL1850-DIB-ISMF_CMIP6.ne30_oECv3wLI.control.cori-knl
20190306.A_WCYCL1850-DIB-ISMF_CMIP6.ne30_oECv3wLI.edison,14,,/home/projects/e3sm/www/Cryosphere/E3SMv1/20190306.A_WCYCL1850-DIB-ISMF_CMIP6.ne30_oECv3wLI.edison
20210614.A_WCYCL1850-DIB-ISMF_CMIP6.ne30_oECv3wLI.DIBbugfix.anvil,12,,/home/projects/e3sm/www/Cryosphere/E3SMv1/20210614.A_WCYCL1850-DIB-ISMF_CMIP6.ne30_oECv3wLI.DIBbugfix.anvil
20200610.A_WCYCL1850-DIB-ISMF_CMIP6.ne30_ECwISC30to60E1r2.cori-knl.maint1p2-3DGM,6,,/home/projects/e3sm/www/Cryosphere/E3SMv1/20200610.A_WCYCL1850-DIB-ISMF_CMIP6.ne30_ECwISC30to60E1r2.cori-knl.maint1p2-3DGM
20210614.A_WCYCL1850-DIB-ISMF_CMIP6.ne30_ECwISC30to60E1r2.anvil.DIBbugFixMGM,13,,/home/projects/e3sm/www/Cryosphere/E3SMv1/20210614.A_WCYCL1850-DIB-ISMF_CMIP6.ne30_ECwISC30to60E1r2.anvil.DIBbugFixMGM
20190923.GMPAS-IAF.T62_oEC60to30v3wLI.cori-knl,7,,/home/projects/e3sm/www/Cryosphere/E3SMv1/20190923.GMPAS-IAF.T62_oEC60to30v3wLI.cori-knl
20190225.GMPAS-DIB-IAF-ISMF.T62_oEC60to30v3wLI.cori-knl,12,,/home/projects/e3sm/www/Cryosphere/E3SMv1/20190225.GMPAS-DIB-IAF-ISMF.T62_oEC60to30v3wLI.cori-knl
20190819.GMPAS-DIB-IAF-ISMF.T62_oEC60to30v3wLI.cori-knl.testNewGM,15,,/home/projects/e3sm/www/Cryosphere/E3SMv1/20190819.GMPAS-DIB-IAF-ISMF.T62_oEC60to30v3wLI.cori-knl.testNewGM
Simulation,Data Size (TB),ESGF Links,HPSS Path,HPSS URL
20190430.A_WCYCL1850-DIB-ISMF_CMIP6.ne30_oECv3wLI.control.cori-knl,9,,/home/projects/e3sm/www/Cryosphere/E3SMv1/20190430.A_WCYCL1850-DIB-ISMF_CMIP6.ne30_oECv3wLI.control.cori-knl,https://portal.nersc.gov/archive/home/projects/e3sm/www/Cryosphere/E3SMv1/20190430.A_WCYCL1850-DIB-ISMF_CMIP6.ne30_oECv3wLI.control.cori-knl
20190306.A_WCYCL1850-DIB-ISMF_CMIP6.ne30_oECv3wLI.edison,14,,/home/projects/e3sm/www/Cryosphere/E3SMv1/20190306.A_WCYCL1850-DIB-ISMF_CMIP6.ne30_oECv3wLI.edison,https://portal.nersc.gov/archive/home/projects/e3sm/www/Cryosphere/E3SMv1/20190306.A_WCYCL1850-DIB-ISMF_CMIP6.ne30_oECv3wLI.edison
20210614.A_WCYCL1850-DIB-ISMF_CMIP6.ne30_oECv3wLI.DIBbugfix.anvil,12,,/home/projects/e3sm/www/Cryosphere/E3SMv1/20210614.A_WCYCL1850-DIB-ISMF_CMIP6.ne30_oECv3wLI.DIBbugfix.anvil,https://portal.nersc.gov/archive/home/projects/e3sm/www/Cryosphere/E3SMv1/20210614.A_WCYCL1850-DIB-ISMF_CMIP6.ne30_oECv3wLI.DIBbugfix.anvil
20200610.A_WCYCL1850-DIB-ISMF_CMIP6.ne30_ECwISC30to60E1r2.cori-knl.maint1p2-3DGM,6,,/home/projects/e3sm/www/Cryosphere/E3SMv1/20200610.A_WCYCL1850-DIB-ISMF_CMIP6.ne30_ECwISC30to60E1r2.cori-knl.maint1p2-3DGM,https://portal.nersc.gov/archive/home/projects/e3sm/www/Cryosphere/E3SMv1/20200610.A_WCYCL1850-DIB-ISMF_CMIP6.ne30_ECwISC30to60E1r2.cori-knl.maint1p2-3DGM
20210614.A_WCYCL1850-DIB-ISMF_CMIP6.ne30_ECwISC30to60E1r2.anvil.DIBbugFixMGM,13,,/home/projects/e3sm/www/Cryosphere/E3SMv1/20210614.A_WCYCL1850-DIB-ISMF_CMIP6.ne30_ECwISC30to60E1r2.anvil.DIBbugFixMGM,https://portal.nersc.gov/archive/home/projects/e3sm/www/Cryosphere/E3SMv1/20210614.A_WCYCL1850-DIB-ISMF_CMIP6.ne30_ECwISC30to60E1r2.anvil.DIBbugFixMGM
20190923.GMPAS-IAF.T62_oEC60to30v3wLI.cori-knl,7,,/home/projects/e3sm/www/Cryosphere/E3SMv1/20190923.GMPAS-IAF.T62_oEC60to30v3wLI.cori-knl,https://portal.nersc.gov/archive/home/projects/e3sm/www/Cryosphere/E3SMv1/20190923.GMPAS-IAF.T62_oEC60to30v3wLI.cori-knl
20190225.GMPAS-DIB-IAF-ISMF.T62_oEC60to30v3wLI.cori-knl,12,,/home/projects/e3sm/www/Cryosphere/E3SMv1/20190225.GMPAS-DIB-IAF-ISMF.T62_oEC60to30v3wLI.cori-knl,https://portal.nersc.gov/archive/home/projects/e3sm/www/Cryosphere/E3SMv1/20190225.GMPAS-DIB-IAF-ISMF.T62_oEC60to30v3wLI.cori-knl
20190819.GMPAS-DIB-IAF-ISMF.T62_oEC60to30v3wLI.cori-knl.testNewGM,15,,/home/projects/e3sm/www/Cryosphere/E3SMv1/20190819.GMPAS-DIB-IAF-ISMF.T62_oEC60to30v3wLI.cori-knl.testNewGM,https://portal.nersc.gov/archive/home/projects/e3sm/www/Cryosphere/E3SMv1/20190819.GMPAS-DIB-IAF-ISMF.T62_oEC60to30v3wLI.cori-knl.testNewGM
2 changes: 1 addition & 1 deletion utils/generate_html.bash
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
pr_num=64
try_num=5
try_num=11

# Chrysalis
#destination_dir=/lcrc/group/e3sm/public_html/diagnostic_output/$USER/data_docs_${pr_num}
Expand Down
21 changes: 13 additions & 8 deletions utils/generate_tables.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -390,7 +390,7 @@ def construct_pages(csv_file: str, model_version: str, group_name: str, include_
resolutions,
header_cells,
f"../docs/source/{model_version}/{group_name}/simulation_data/simulation_table.rst",
[65, 15, 220, 120, 140]
[85, 15, 400, 140, 170]
Copy link
Collaborator Author

@forsyth2 forsyth2 Oct 14, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor note: I noticed these numbers were reduced in #67:
[85, 15, 400, 140] => [65, 15, 220, 120, 140]. Typically, I only increase these numbers to fit longer simulation names, but it's not a huge deal since the script is run more-or-less independently per new data addition. In the future though, it would have been good for me to review PRs like #67. It appears my notification level for e3sm_data_docs was limited to only mentions. I've changed it to all activity now.

Copy link
Collaborator

@chengzhuzhang chengzhuzhang Oct 14, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have to increase these numbers to fit the long urls...Sounds good, I will also tag you for future PRs.

)
if include_reproduction_scripts:
header_cells_reproduction: List[str] = ["Simulation", "Machine", "10 day checksum", "Reproduction Script", "Original Script (requires significant changes to run!!)",]
Expand All @@ -408,14 +408,19 @@ def construct_pages(csv_file: str, model_version: str, group_name: str, include_
)

if __name__ == "__main__":
#construct_pages("simulations_v2.csv", "v2", "WaterCycle")
#construct_pages("simulations_v2_1.csv", "v2.1", "WaterCycle")
#construct_pages("simulations_v2_1.csv", "v2.1", "BGC")
#
# Sources for v1 data
# v1 data
# https://acme-climate.atlassian.net/wiki/spaces/ED/pages/4495441922/V1+Simulation+backfill+WIP
# https://acme-climate.atlassian.net/wiki/spaces/DOC/pages/1271169273/v1+High+Res+Coupled+Run+Output+HPSS+Archive
construct_pages("input/simulations_v1_water_cycle.csv", "v1", "WaterCycle")
#construct_pages("input/simulations_v1_water_cycle.csv", "v1", "WaterCycle")
construct_pages("input/simulations_v1_cryosphere.csv", "v1", "Cryosphere")
construct_pages("input/simulations_v1_bgc.csv", "v1", "BGC")
# construct_pages("input/simulations_v3_LR_coupled.csv", "v3", "CoupledSystem")

# v2 data
#construct_pages("simulations_v2.csv", "v2", "WaterCycle")

# v2.1 data
#construct_pages("simulations_v2_1.csv", "v2.1", "WaterCycle")
#construct_pages("simulations_v2_1.csv", "v2.1", "BGC")

# v3 data
#construct_pages("input/simulations_v3_LR_coupled.csv", "v3", "CoupledSystem")