Skip to content

Commit b712bc6

Browse files
AaronDonahuecrteraiforsyth2
authored
Asd crt/add screamv0 v1 sim data (#49)
* These commits will add simulation and reproducibility data for the SCREAM campaigns. This first commit adds a directory to document the SCREAMv0 DYAMOND2 campaign. Subsequent commits will add directories and data for the SCREAMv1 campaigns. * Add directories for SCREAMv0 simulations, start to populate tables * Fix reprocution_table formatting for FourSeasons * more fixes to repro table * Update description of directories for SCREAMv1 4-seasons * Update to SCREAMv0 data and run script description * Fix hyperlink to reference for SCREAMv0 paper * Fix title in SCREAMv0 index * Improve doc organization * Add run scripts for SCREAMv1 Four Seasons simulations. * Final doc cleanup --------- Co-authored-by: Chris Terai <[email protected]> Co-authored-by: Ryan Forsyth <[email protected]>
1 parent d038fb8 commit b712bc6

File tree

15 files changed

+760
-4
lines changed

15 files changed

+760
-4
lines changed
Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
SCREAMv0 DYAMOND2
2+
====================
3+
4+
The DYAMOND2 simulation campaign using SCREAMv0 that was submitted to the `DYAMOND2 intercomparison <https://www.esiwace.eu/the-project/past-phases/dyamond-initiative/services-dyamond-winter>`_.
5+
6+
If you use data from this simulation campaign, please cite the following overview
7+
manuscript:
8+
9+
* `Caldwell et al. 2021 (JAMES) <https://doi.org/10.1029/2021MS002544>`_
10+
11+
.. toctree::
12+
:maxdepth: 2
13+
:caption: Contents:
14+
15+
simulation_data/index
Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
***************
2+
Simulation Data
3+
***************
4+
5+
The SCREAMv0 simulation data is available on **NERSC HPSS**.
6+
7+
**If you have an account on NERSC**, you can retrieve the data locally or remotely using Globus.
8+
9+
To download simulation data locally on a NERSC machine, refer to NERSC's documentation `here <https://docs.nersc.gov/filesystems/archive/#hsi-commands>`_.
10+
11+
**If you do not have access to NERSC**, you can download simulation data directly through the NERSC HPSS
12+
`web interface <https://portal.nersc.gov/archive/home/t/terai/www/SCREAM/SCREAMv0.SCREAM-DY2.ne1024pg2.20201127>`_.
13+
Note that this will be slow given the large file sizes.
14+
15+
Scripts originally used to run SCREAMv0 DYAMOND2 simulations are available `here <https://github.com/E3SM-Project/e3sm_data_docs/tree/main/run_scripts/SCREAMv0/original/>`_.
16+
17+
.. toctree::
18+
:maxdepth: 2
19+
:caption: Contents:
20+
21+
simulation_table
Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
**********************************
2+
SCREAMv0 DYAMOND2 (2021) simulation table
3+
**********************************
4+
5+
+-------------------------------------------------------------------+-----------------+---------------------------------------------------------------------------+
6+
| Simulation | Data Size (TB) | NERSC HPSS Path |
7+
+===================================================================+=================+===========================================================================+
8+
| **Main Simulation (40 days)** |
9+
+-------------------------------------------------------------------+-----------------+---------------------------------------------------------------------------+
10+
| January 2020 (DYAMOND2 Configuration) | 46.6 | /home/t/terai/www/SCREAM/SCREAMv0.SCREAM-DY2.ne1024pg2.20201127 |
11+
+-------------------------------------------------------------------+-----------------+---------------------------------------------------------------------------+

docs/source/SCREAMv0/index.rst

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
SCREAMv0
2+
==========================================
3+
4+
.. toctree::
5+
:maxdepth: 2
6+
:caption: Contents:
7+
8+
DYAMOND2/index
Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
SCREAMv1 Four Seasons
2+
====================
3+
4+
The simulation compaign for the SCREAMv1 Four-Seasons runs. For more information
5+
about this simulation campaign see the citation below.
6+
7+
If you use data from this simulation campaign, please cite the relevant overview
8+
manuscript.
9+
10+
* `Donahue et al. 2024 (JAMES) <https://doi.org/10.1029/2024MS004314>`_
11+
12+
.. toctree::
13+
:maxdepth: 2
14+
:caption: Contents:
15+
16+
simulation_data/index
17+
reproducing_simulations/index
Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
***************
2+
Simulation Data
3+
***************
4+
5+
The SCREAMv1 FourSeasons simulation data is available on **NERSC HPSS**.
6+
7+
**If you have an account on NERSC**, you can retrieve the data locally or remotely using Globus.
8+
9+
To download simulation data locally on a NERSC machine: ::
10+
11+
zstash extract --hpss=<HPSS path below>
12+
13+
To download simulation data remotely using the zstash Globus interface: ::
14+
15+
zstash extract --hpss=globus://nersc/<HPSS path below>
16+
17+
or ::
18+
19+
zstash extract --hpss=globus://9cd89cfd-6d04-11e5-ba46-22000b92c6ec/<HPSS path below>
20+
21+
Note that the data management tool `zstash <https://github.com/E3SM-Project/zstash>`_ is available from the `E3SM-Unified <https://github.com/E3SM-Project/e3sm-unified>`_ conda environment.
22+
23+
For more information, refer to `zstash usage <https://e3sm-project.github.io/zstash/_build/html/master/usage.html#extract>`_.
24+
25+
**If you do not have access to NERSC**, you can download simulation data directly through the NERSC HPSS
26+
`web interface <https://portal.nersc.gov/archive/home/a/adonahue/www/>`_.
27+
Note that this will be slow and inefficient since you'll have to download the individual files.
28+
29+
Scripts originally used to run SCREAMv1 FourSeasons simulations are available `here <https://github.com/E3SM-Project/e3sm_data_docs/tree/main/run_scripts/SCREAMv1/original/>`_.
30+
31+
.. toctree::
32+
:maxdepth: 2
33+
:caption: Contents:
34+
35+
simulation_table
Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,47 @@
1+
**********************************
2+
SCREAMv1 Four-Seasons (2024) simulation table
3+
**********************************
4+
5+
+-------------------------------------------------------------------+-----------------+---------------------------------------------------------------------------+
6+
| Simulation | Data Size (GB) | NERSC HPSS Path |
7+
+===================================================================+=================+===========================================================================+
8+
| **Main Simulation (40 days)** |
9+
+-------------------------------------------------------------------+-----------------+---------------------------------------------------------------------------+
10+
| January 2020 (DYAMOND2 Configuration) | 129.3 | /home/a/adonahue/www/DYAMOND2_SCREAMv1 |
11+
+-------------------------------------------------------------------+-----------------+---------------------------------------------------------------------------+
12+
| April 2013 | 130.5 | /home/a/adonahue/www/Apr1_2013_SCREAMv1 |
13+
+-------------------------------------------------------------------+-----------------+---------------------------------------------------------------------------+
14+
| August 2016 (DYAMOND1 Configuration) | 130.4 | /home/a/adonahue/www/DYAMOND1_SCREAMv1 |
15+
+-------------------------------------------------------------------+-----------------+---------------------------------------------------------------------------+
16+
| October 2013 | 119.0 | /home/a/adonahue/www/Oct1_2013_SCREAMv1 |
17+
+-------------------------------------------------------------------+-----------------+---------------------------------------------------------------------------+
18+
| **Process Impact - Turbulent Mountain Stress (6 days)** |
19+
+-------------------------------------------------------------------+-----------------+---------------------------------------------------------------------------+
20+
| January 2020 (DYAMOND2 Configuration) | 38.3 | /home/a/adonahue/www/Process_Impacts_tests/Wind_Speed/January_DYAMOND2 |
21+
+-------------------------------------------------------------------+-----------------+---------------------------------------------------------------------------+
22+
| April 2013 | 19.1 | /home/a/adonahue/www/Process_Impacts_tests/Wind_Speed/April |
23+
+-------------------------------------------------------------------+-----------------+---------------------------------------------------------------------------+
24+
| August 2016 (DYAMOND1 Configuration) | 19.1 | /home/a/adonahue/www/Process_Impacts_tests/Wind_Speed/August_DYAMOND1 |
25+
+-------------------------------------------------------------------+-----------------+---------------------------------------------------------------------------+
26+
| October 2013 | 19.1 | /home/a/adonahue/www/Process_Impacts_tests/Wind_Speed/October |
27+
+-------------------------------------------------------------------+-----------------+---------------------------------------------------------------------------+
28+
| **Process Impact - *Improved Cloud Tuning (6 days)** |
29+
+-------------------------------------------------------------------+-----------------+---------------------------------------------------------------------------+
30+
| January 2020 (DYAMOND2 Configuration) | 19.1 | /home/a/adonahue/www/Process_Impacts_tests/Cloud_Tuning/January_DYAMOND2 |
31+
+-------------------------------------------------------------------+-----------------+---------------------------------------------------------------------------+
32+
| April 2013 | 19.1 | /home/a/adonahue/www/Process_Impacts_tests/Cloud_Tuning/April |
33+
+-------------------------------------------------------------------+-----------------+---------------------------------------------------------------------------+
34+
| August 2016 (DYAMOND1 Configuration) | 19.1 | /home/a/adonahue/www/Process_Impacts_tests/Cloud_Tuning/August_DYAMOND1 |
35+
+-------------------------------------------------------------------+-----------------+---------------------------------------------------------------------------+
36+
| October 2013 | 19.1 | /home/a/adonahue/www/Process_Impacts_tests/Cloud_Tuning/October |
37+
+-------------------------------------------------------------------+-----------------+---------------------------------------------------------------------------+
38+
| **Process Impact - Improved SHOC Tuning (6 days)** |
39+
+-------------------------------------------------------------------+-----------------+---------------------------------------------------------------------------+
40+
| January 2020 (DYAMOND2 Configuration) | 15.9 | /home/a/adonahue/www/Process_Impacts_tests/Warm_T/January_DYAMOND2 |
41+
+-------------------------------------------------------------------+-----------------+---------------------------------------------------------------------------+
42+
| April 2013 | 15.9 | /home/a/adonahue/www/Process_Impacts_tests/Warm_T/April |
43+
+-------------------------------------------------------------------+-----------------+---------------------------------------------------------------------------+
44+
| August 2016 (DYAMOND1 Configuration) | 15.9 | /home/a/adonahue/www/Process_Impacts_tests/Warm_T/August_DYAMOND1 |
45+
+-------------------------------------------------------------------+-----------------+---------------------------------------------------------------------------+
46+
| October 2013 | 15.9 | /home/a/adonahue/www/Process_Impacts_tests/Warm_T/October |
47+
+-------------------------------------------------------------------+-----------------+---------------------------------------------------------------------------+

docs/source/SCREAMv1/index.rst

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
SCREAMv1
2+
==========================================
3+
4+
.. toctree::
5+
:maxdepth: 2
6+
:caption: Contents:
7+
8+
FourSeasons/index

docs/source/index.rst

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,8 @@ simulations.
1616
:caption: Contents:
1717

1818
v2/index
19+
SCREAMv0/index
20+
SCREAMv1/index
1921

2022

2123
.. Indices and tables
Lines changed: 199 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,199 @@
1+
#!/bin/bash
2+
3+
# Set run options
4+
#=============================================
5+
resolution=ne1024pg2_r0125_oRRS18to6v3
6+
compset=F2010-SCREAM-HR-DYAMOND2
7+
checkout_date=20201127 #the date you *checked out* the code
8+
branch=ad0e203 #actual git hash of branch to check out
9+
run_descriptor=SCREAMv0 #will be SCREAMv0 for production run
10+
repo=scream
11+
machine=cori-knl
12+
compiler=intel
13+
stop_option="ndays"
14+
stop_n="4"
15+
rest_n="1"
16+
walltime="29:00:00"
17+
queue="regular"
18+
debug_compile='FALSE'
19+
date_string=`date`
20+
21+
# Setup processor layout
22+
nnodes_atm=1536
23+
nnodes_ocn=1536
24+
nthreads=16
25+
mpi_tasks_per_node=8
26+
ntasks_atm=$(expr ${nnodes_atm} \* ${mpi_tasks_per_node})
27+
ntasks_ocn=$(expr ${nnodes_ocn} \* ${mpi_tasks_per_node})
28+
total_tasks_per_node=$(expr ${mpi_tasks_per_node} \* ${nthreads})
29+
if [ ${ntasks_ocn} -ne ${ntasks_atm} ]; then
30+
nnodes=$(expr ${nnodes_atm} + ${nnodes_ocn})
31+
else
32+
nnodes=${nnodes_atm}
33+
fi
34+
pelayout=${nnodes}x${mpi_tasks_per_node}x${nthreads}
35+
36+
# Who to send email updates on run status to
37+
email_address="[email protected]"
38+
39+
# Set flags specific to running this script
40+
do_download=true
41+
do_newcase=true
42+
do_setup=true
43+
do_build=true
44+
do_submit=true
45+
46+
case_name=SCREAMv0.SCREAM-DY2.ne1024pg2.${date_string}
47+
48+
# Set paths
49+
#code_root=${HOME}/gitwork/scream/
50+
code_root=${CSCRATCH}/E3SM_code/screamv0_20201127/
51+
case_root=${CSCRATCH}/e3sm_scratch/cori-knl/${case_name}
52+
53+
######################################################################################
54+
### USERS PROBABLY DON'T NEED TO CHANGE ANYTHING BELOW HERE EXCEPT user_nl_* FILES ###
55+
######################################################################################
56+
57+
# Make directories created by this script world-readable:
58+
#=============================================
59+
umask 022
60+
61+
# Download code
62+
#=============================================
63+
if [ "${do_download}" == "true" ]; then
64+
65+
echo "Cloning repository repo = $repo into branch = $branch under code_root = $code_root"
66+
cdir=`pwd`
67+
mkdir -p $code_root/
68+
69+
# This will put repository, with all code, in directory $tag_name
70+
git clone [email protected]:E3SM-Project/${repo}.git $code_root
71+
72+
# Setup git hooks
73+
rm -rf $code_root/.git/hooks
74+
git clone [email protected]:E3SM-Project/E3SM-Hooks.git $code_root/.git/hooks
75+
cd $code_root
76+
git config commit.template $code_root/.git/hooks/commit.template
77+
78+
# Bring in all submodule components
79+
git submodule update --init --recursive
80+
81+
# Check out desired branch
82+
git checkout ${branch}
83+
cd ${cdir}
84+
85+
fi
86+
87+
# Create case
88+
#=============================================
89+
if [ "${do_newcase}" == "true" ]; then
90+
${code_root}/cime/scripts/create_newcase \
91+
--case ${case_root} \
92+
--compset ${compset} --res ${resolution} \
93+
--machine ${machine} --compiler ${compiler} \
94+
--queue ${queue} \
95+
--walltime ${walltime}
96+
fi
97+
98+
# Copy this script to case directory
99+
#=============================================
100+
cp -v `basename $0` ${case_root}/
101+
102+
# Setup
103+
#=============================================
104+
if [ "${do_setup}" == "true" ]; then
105+
cd ${case_root}
106+
107+
# Set run length
108+
./xmlchange STOP_OPTION=${stop_option},STOP_N=${stop_n}
109+
./xmlchange REST_N=${rest_n}
110+
111+
# Set processor layout
112+
if [ ${ntasks_ocn} -ne ${ntasks_atm} ]; then
113+
./xmlchange NTASKS=${ntasks_ocn}
114+
./xmlchange NTASKS_ATM=${ntasks_atm}
115+
./xmlchange ROOTPE_ATM=${ntasks_ocn}
116+
else
117+
./xmlchange NTASKS=${ntasks_atm}
118+
fi
119+
./xmlchange NTHRDS_ATM=${nthreads}
120+
./xmlchange MAX_MPITASKS_PER_NODE=${mpi_tasks_per_node}
121+
./xmlchange MAX_TASKS_PER_NODE=${total_tasks_per_node}
122+
123+
# Flag for debug compile
124+
./xmlchange --id DEBUG --val ${debug_compile}
125+
126+
# Set PIO format, use PIO version 2, and increase PIO buffer size
127+
./xmlchange PIO_NETCDF_FORMAT="64bit_data"
128+
./xmlchange PIO_VERSION="2"
129+
./xmlchange PIO_BUFFER_SIZE_LIMIT=134217728
130+
131+
# Edit CAM namelist to set dycore options for new grid
132+
cat <<EOF >> user_nl_eam
133+
134+
!*** By default the model dumps hundreds of vars in h0. Don't do that. ***
135+
empty_htapes=.true.
136+
!*** Outputs for DYAMOND (note fincl can only go to 10) ***
137+
nhtfrq = 12,12,12,12,-3,-3,-3,-3,-3,-3 !output freq: 12 steps=15 mi, -3=3hrs
138+
mfilt = 96,96,96,96,8,8,8,8,8,8 !new file freq: daily in all cases
139+
fincl1 = 'CLDLOW:I', 'CLDMED:I', 'CLDHGH:I', 'CLDTOT:I', 'TMCLDLIQ:I',
140+
'TMCLDICE:I', 'TMRAINQM:I', 'TMCLDRIM:I', 'TMQ:I', 'CAPE:I', 'CIN:I'
141+
fincl2 = 'PS:I', 'TS:I', 'TREFHT:I', 'QREFHT:I', 'PRECT:I','PRECSL:I',
142+
'WINDSPD_10M:I', 'TAUX:I', 'TAUY:I', 'SHFLX:I', 'LHFLX:I'
143+
fincl3 = 'FSNTOA:I', 'FLNT:I','FLNTC:I','FSNTOAC:I', 'FSNS:I', 'FSDS:I',
144+
'FLNS:I', 'FLDS:I'
145+
fincl4 = 'RH200:I', 'RH500:I', 'RH700:I', 'RH850:I',
146+
'OMEGA200:I', 'OMEGA500:I', 'OMEGA700:I', 'OMEGA850:I',
147+
'Z200:I', 'Z500:I', 'Z700:I', 'Z850:I'
148+
!*** 3 hrly (mostly 3d) variables below here ***
149+
fincl5 = 'PS:I', 'PSL:I', 'TMNUMLIQ:I', 'TMNUMICE:I', 'TMNUMRAI:I'
150+
fincl6 = 'U:I', 'V:I'
151+
fincl7 = 'T:I', 'Q:I',
152+
fincl8 = 'CLDLIQ:I', 'CLDICE:I'
153+
fincl9 = 'CLOUD:I','OMEGA:I'
154+
fincl10= 'EMIS:I', 'TOT_ICLD_VISTAU:I'
155+
!*** Rad Freq: 4x75sec=5 min which divides 15 min output freq ***
156+
iradsw = 4
157+
iradlw = 4
158+
159+
EOF
160+
161+
cat <<EOF >> user_nl_elm
162+
163+
164+
EOF
165+
166+
# Finally, run setup
167+
./case.setup
168+
169+
# The run location is determined in the bowels of CIME
170+
# Symlink to that location from user-chosen $case_root (=current dir)
171+
ln -s `./xmlquery -value RUNDIR` run
172+
173+
174+
# This disables the logic that sets tprof_n and tprof_options internally.
175+
#./xmlchange --file env_run.xml TPROF_TOTAL=-1
176+
#echo "tprof_n = 1" >> user_nl_cpl
177+
#echo "tprof_option = 'nsteps'" >> user_nl_cpl
178+
fi
179+
180+
# Build
181+
#=============================================
182+
if [ "${do_build}" == "true" ]; then
183+
cd ${case_root}
184+
./case.build
185+
186+
# Set file striping on run dir for writing large files
187+
ls -l run > /dev/null #just using this command as a check that run dir exists
188+
lfs setstripe -S 1m -c 64 run
189+
lfs getstripe run >& lfs_run.txt
190+
fi
191+
192+
# Run
193+
#=============================================
194+
if [ "${do_submit}" == "true" ]; then
195+
cd ${case_root}
196+
./case.submit --batch-args="--mail-type=ALL --mail-user=${email_address}"
197+
fi
198+
199+
echo "Done working in ${case_root}"

0 commit comments

Comments
 (0)