Skip to content

Commit 437cf97

Browse files
BrianCurtis-NOAABrianCurtis-NOAA
authored
ZStd with Netcdf; Modified tests to compare netcdf parallel output of WW3. Bring in several bug fixes affecting netcdf field output. (#2826) (#2444)
* UFSWM - Add ZStd library and enable netcdf support for it. * * WW3 - This branch of WW3 fixes several bugs in parallel netcdf output from WW3. The tests in ufs-weather-model are modified to test the WW3 netcdf output fields rather than WW3 binary output.
1 parent 6ae4a01 commit 437cf97

File tree

66 files changed

+2633
-2592
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

66 files changed

+2633
-2592
lines changed

‎WW3‎

‎tests/bl_date.conf‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
export BL_DATE=20250926
1+
export BL_DATE=20250930

‎tests/default_vars.sh‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2098,6 +2098,7 @@ export_hafs_regional ()
20982098
export FILENAME_BASE="'atm' 'sfc'"
20992099
export OUTPUT_GRID="'regional_latlon'"
21002100
export OUTPUT_FILE="'netcdf'"
2101+
export ZSTANDARD_LEVEL=0
21012102
export IDEFLATE=0
21022103
export QUANTIZE_NSD=0
21032104
export CEN_LON=-62.0

‎tests/logs/OpnReqTests_control_p8_hera.log‎

Lines changed: 24 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,9 @@
1-
Fri Sep 26 17:33:20 UTC 2025
1+
Wed Oct 1 05:46:02 UTC 2025
22
Start Operation Requirement Test
33

44

55
baseline dir = /scratch3/NCEPDEV/stmp/role.epic/ORT-stmp/role.epic/FV3_OPNREQ_TEST/OPNREQ_TEST/control_p8_bit_base_gnu
6-
working dir = /scratch3/NCEPDEV/stmp/role.epic/ORT-ptmp/role.epic/FV3_OPNREQ_TEST/opnReqTest_3105650/bit_base_bit_base
6+
working dir = /scratch3/NCEPDEV/stmp/role.epic/ORT-ptmp/role.epic/FV3_OPNREQ_TEST/opnReqTest_869041/bit_base_bit_base
77
Checking test bit_base results ....
88
Moving baseline bit_base files ....
99
Moving sfcf000.nc .........OK
@@ -51,14 +51,14 @@ Moving baseline bit_base files ....
5151
Moving RESTART/20210323.060000.sfc_data.tile5.nc .........OK
5252
Moving RESTART/20210323.060000.sfc_data.tile6.nc .........OK
5353

54-
0: The total amount of wall time = 252.238268
55-
0: The maximum resident set size (KB) = 1567616
54+
0: The total amount of wall time = 252.032484
55+
0: The maximum resident set size (KB) = 1557812
5656

5757
Test bit_base PASS
5858

5959

6060
baseline dir = /scratch3/NCEPDEV/stmp/role.epic/ORT-stmp/role.epic/FV3_OPNREQ_TEST/OPNREQ_TEST/control_p8_dbg_base_gnu
61-
working dir = /scratch3/NCEPDEV/stmp/role.epic/ORT-ptmp/role.epic/FV3_OPNREQ_TEST/opnReqTest_3105650/dbg_base_dbg_base
61+
working dir = /scratch3/NCEPDEV/stmp/role.epic/ORT-ptmp/role.epic/FV3_OPNREQ_TEST/opnReqTest_869041/dbg_base_dbg_base
6262
Checking test dbg_base results ....
6363
Moving baseline dbg_base files ....
6464
Moving sfcf000.nc .........OK
@@ -106,14 +106,14 @@ Moving baseline dbg_base files ....
106106
Moving RESTART/20210323.060000.sfc_data.tile5.nc .........OK
107107
Moving RESTART/20210323.060000.sfc_data.tile6.nc .........OK
108108

109-
0: The total amount of wall time = 883.979371
110-
0: The maximum resident set size (KB) = 1539928
109+
0: The total amount of wall time = 894.009716
110+
0: The maximum resident set size (KB) = 1533112
111111

112112
Test dbg_base PASS
113113

114114

115115
baseline dir = /scratch3/NCEPDEV/stmp/role.epic/ORT-stmp/role.epic/FV3_OPNREQ_TEST/OPNREQ_TEST/control_p8_std_base_gnu
116-
working dir = /scratch3/NCEPDEV/stmp/role.epic/ORT-ptmp/role.epic/FV3_OPNREQ_TEST/opnReqTest_3105650/dcp_dcp
116+
working dir = /scratch3/NCEPDEV/stmp/role.epic/ORT-ptmp/role.epic/FV3_OPNREQ_TEST/opnReqTest_869041/dcp_dcp
117117
Checking test dcp results ....
118118
Comparing sfcf000.nc .....USING NCCMP......OK
119119
Comparing sfcf021.nc .....USING NCCMP......OK
@@ -160,14 +160,14 @@ Checking test dcp results ....
160160
Comparing RESTART/20210323.060000.sfc_data.tile5.nc .....USING NCCMP......OK
161161
Comparing RESTART/20210323.060000.sfc_data.tile6.nc .....USING NCCMP......OK
162162

163-
0: The total amount of wall time = 229.449231
164-
0: The maximum resident set size (KB) = 1537776
163+
0: The total amount of wall time = 228.419745
164+
0: The maximum resident set size (KB) = 1552196
165165

166166
Test dcp PASS
167167

168168

169169
baseline dir = /scratch3/NCEPDEV/stmp/role.epic/ORT-stmp/role.epic/FV3_OPNREQ_TEST/OPNREQ_TEST/control_p8_std_base_gnu
170-
working dir = /scratch3/NCEPDEV/stmp/role.epic/ORT-ptmp/role.epic/FV3_OPNREQ_TEST/opnReqTest_3105650/mpi_mpi
170+
working dir = /scratch3/NCEPDEV/stmp/role.epic/ORT-ptmp/role.epic/FV3_OPNREQ_TEST/opnReqTest_869041/mpi_mpi
171171
Checking test mpi results ....
172172
Comparing sfcf000.nc .....USING NCCMP......OK
173173
Comparing sfcf021.nc .....USING NCCMP......OK
@@ -214,14 +214,14 @@ Checking test mpi results ....
214214
Comparing RESTART/20210323.060000.sfc_data.tile5.nc .....USING NCCMP......OK
215215
Comparing RESTART/20210323.060000.sfc_data.tile6.nc .....USING NCCMP......OK
216216

217-
0: The total amount of wall time = 225.564772
218-
0: The maximum resident set size (KB) = 1545868
217+
0: The total amount of wall time = 228.129983
218+
0: The maximum resident set size (KB) = 1530176
219219

220220
Test mpi PASS
221221

222222

223223
baseline dir = /scratch3/NCEPDEV/stmp/role.epic/ORT-stmp/role.epic/FV3_OPNREQ_TEST/OPNREQ_TEST/control_p8_std_base_gnu
224-
working dir = /scratch3/NCEPDEV/stmp/role.epic/ORT-ptmp/role.epic/FV3_OPNREQ_TEST/opnReqTest_3105650/rst_rst
224+
working dir = /scratch3/NCEPDEV/stmp/role.epic/ORT-ptmp/role.epic/FV3_OPNREQ_TEST/opnReqTest_869041/rst_rst
225225
Checking test rst results ....
226226
Comparing sfcf000.nc .....USING NCCMP......OK
227227
Comparing sfcf021.nc .....USING NCCMP......OK
@@ -268,14 +268,14 @@ Checking test rst results ....
268268
Comparing RESTART/20210323.060000.sfc_data.tile5.nc .....USING NCCMP......OK
269269
Comparing RESTART/20210323.060000.sfc_data.tile6.nc .....USING NCCMP......OK
270270

271-
0: The total amount of wall time = 226.686440
272-
0: The maximum resident set size (KB) = 1545180
271+
0: The total amount of wall time = 228.328580
272+
0: The maximum resident set size (KB) = 1527608
273273

274274
Test rst PASS
275275

276276

277277
baseline dir = /scratch3/NCEPDEV/stmp/role.epic/ORT-stmp/role.epic/FV3_OPNREQ_TEST/OPNREQ_TEST/control_p8_std_base_gnu
278-
working dir = /scratch3/NCEPDEV/stmp/role.epic/ORT-ptmp/role.epic/FV3_OPNREQ_TEST/opnReqTest_3105650/std_base_std_base
278+
working dir = /scratch3/NCEPDEV/stmp/role.epic/ORT-ptmp/role.epic/FV3_OPNREQ_TEST/opnReqTest_869041/std_base_std_base
279279
Checking test std_base results ....
280280
Moving baseline std_base files ....
281281
Moving sfcf000.nc .........OK
@@ -323,14 +323,14 @@ Moving baseline std_base files ....
323323
Moving RESTART/20210323.060000.sfc_data.tile5.nc .........OK
324324
Moving RESTART/20210323.060000.sfc_data.tile6.nc .........OK
325325

326-
0: The total amount of wall time = 227.696146
327-
0: The maximum resident set size (KB) = 1520100
326+
0: The total amount of wall time = 227.326396
327+
0: The maximum resident set size (KB) = 1528520
328328

329329
Test std_base PASS
330330

331331

332332
baseline dir = /scratch3/NCEPDEV/stmp/role.epic/ORT-stmp/role.epic/FV3_OPNREQ_TEST/OPNREQ_TEST/control_p8_std_base_gnu
333-
working dir = /scratch3/NCEPDEV/stmp/role.epic/ORT-ptmp/role.epic/FV3_OPNREQ_TEST/opnReqTest_3105650/thr_thr
333+
working dir = /scratch3/NCEPDEV/stmp/role.epic/ORT-ptmp/role.epic/FV3_OPNREQ_TEST/opnReqTest_869041/thr_thr
334334
Checking test thr results ....
335335
Comparing sfcf000.nc .....USING NCCMP......OK
336336
Comparing sfcf021.nc .....USING NCCMP......OK
@@ -377,11 +377,11 @@ Checking test thr results ....
377377
Comparing RESTART/20210323.060000.sfc_data.tile5.nc .....USING NCCMP......OK
378378
Comparing RESTART/20210323.060000.sfc_data.tile6.nc .....USING NCCMP......OK
379379

380-
0: The total amount of wall time = 225.980725
381-
0: The maximum resident set size (KB) = 1525128
380+
0: The total amount of wall time = 227.850498
381+
0: The maximum resident set size (KB) = 1530792
382382

383383
Test thr PASS
384384

385385
OPERATION REQUIREMENT TEST WAS SUCCESSFUL
386-
Fri Sep 26 18:38:46 UTC 2025
387-
Elapsed time: 01h:05m:26s. Have a nice day!
386+
Wed Oct 1 06:50:29 UTC 2025
387+
Elapsed time: 01h:04m:35s. Have a nice day!

‎tests/logs/OpnReqTests_cpld_control_nowave_noaero_p8_hera.log‎

Lines changed: 12 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,9 @@
1-
Fri Sep 26 16:08:02 UTC 2025
1+
Wed Oct 1 04:21:25 UTC 2025
22
Start Operation Requirement Test
33

44

55
baseline dir = /scratch3/NCEPDEV/stmp/role.epic/ORT-stmp/role.epic/FV3_OPNREQ_TEST/OPNREQ_TEST/cpld_control_c96_noaero_p8_dbg_base_gnu
6-
working dir = /scratch3/NCEPDEV/stmp/role.epic/ORT-ptmp/role.epic/FV3_OPNREQ_TEST/opnReqTest_1650304/dbg_base_dbg_base
6+
working dir = /scratch3/NCEPDEV/stmp/role.epic/ORT-ptmp/role.epic/FV3_OPNREQ_TEST/opnReqTest_3687544/dbg_base_dbg_base
77
Checking test dbg_base results ....
88
Moving baseline dbg_base files ....
99
Moving sfcf021.tile1.nc .........OK
@@ -66,14 +66,14 @@ Moving baseline dbg_base files ....
6666
Moving RESTART/iced.2021-03-23-21600.nc .........OK
6767
Moving RESTART/ufs.cpld.cpl.r.2021-03-23-21600.nc .........OK
6868

69-
0: The total amount of wall time = 2303.412957
70-
0: The maximum resident set size (KB) = 1638636
69+
0: The total amount of wall time = 2315.506476
70+
0: The maximum resident set size (KB) = 1647808
7171

7272
Test dbg_base PASS
7373

7474

7575
baseline dir = /scratch3/NCEPDEV/stmp/role.epic/ORT-stmp/role.epic/FV3_OPNREQ_TEST/OPNREQ_TEST/cpld_control_c96_noaero_p8_std_base_gnu
76-
working dir = /scratch3/NCEPDEV/stmp/role.epic/ORT-ptmp/role.epic/FV3_OPNREQ_TEST/opnReqTest_1650304/rst_rst
76+
working dir = /scratch3/NCEPDEV/stmp/role.epic/ORT-ptmp/role.epic/FV3_OPNREQ_TEST/opnReqTest_3687544/rst_rst
7777
Checking test rst results ....
7878
Comparing sfcf021.tile1.nc .....USING NCCMP......OK
7979
Comparing sfcf021.tile2.nc .....USING NCCMP......OK
@@ -135,14 +135,14 @@ Checking test rst results ....
135135
Comparing RESTART/iced.2021-03-23-21600.nc .....USING NCCMP......OK
136136
Comparing RESTART/ufs.cpld.cpl.r.2021-03-23-21600.nc .....USING NCCMP......OK
137137

138-
0: The total amount of wall time = 591.103054
139-
0: The maximum resident set size (KB) = 1651416
138+
0: The total amount of wall time = 585.879110
139+
0: The maximum resident set size (KB) = 1642544
140140

141141
Test rst PASS
142142

143143

144144
baseline dir = /scratch3/NCEPDEV/stmp/role.epic/ORT-stmp/role.epic/FV3_OPNREQ_TEST/OPNREQ_TEST/cpld_control_c96_noaero_p8_std_base_gnu
145-
working dir = /scratch3/NCEPDEV/stmp/role.epic/ORT-ptmp/role.epic/FV3_OPNREQ_TEST/opnReqTest_1650304/std_base_std_base
145+
working dir = /scratch3/NCEPDEV/stmp/role.epic/ORT-ptmp/role.epic/FV3_OPNREQ_TEST/opnReqTest_3687544/std_base_std_base
146146
Checking test std_base results ....
147147
Moving baseline std_base files ....
148148
Moving sfcf021.tile1.nc .........OK
@@ -205,11 +205,11 @@ Moving baseline std_base files ....
205205
Moving RESTART/iced.2021-03-23-21600.nc .........OK
206206
Moving RESTART/ufs.cpld.cpl.r.2021-03-23-21600.nc .........OK
207207

208-
0: The total amount of wall time = 577.610941
209-
0: The maximum resident set size (KB) = 1651444
208+
0: The total amount of wall time = 581.720080
209+
0: The maximum resident set size (KB) = 1643460
210210

211211
Test std_base PASS
212212

213213
OPERATION REQUIREMENT TEST WAS SUCCESSFUL
214-
Fri Sep 26 17:33:20 UTC 2025
215-
Elapsed time: 01h:25m:18s. Have a nice day!
214+
Wed Oct 1 05:45:46 UTC 2025
215+
Elapsed time: 01h:24m:22s. Have a nice day!

‎tests/logs/OpnReqTests_regional_control_hera.log‎

Lines changed: 15 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,9 @@
1-
Fri Sep 26 15:14:59 UTC 2025
1+
Wed Oct 1 03:27:31 UTC 2025
22
Start Operation Requirement Test
33

44

55
baseline dir = /scratch3/NCEPDEV/stmp/role.epic/ORT-stmp/role.epic/FV3_OPNREQ_TEST/OPNREQ_TEST/regional_control_bit_base_gnu
6-
working dir = /scratch3/NCEPDEV/stmp/role.epic/ORT-ptmp/role.epic/FV3_OPNREQ_TEST/opnReqTest_976059/bit_base_bit_base
6+
working dir = /scratch3/NCEPDEV/stmp/role.epic/ORT-ptmp/role.epic/FV3_OPNREQ_TEST/opnReqTest_2966052/bit_base_bit_base
77
Checking test bit_base results ....
88
Moving baseline bit_base files ....
99
Moving dynf000.nc .........OK
@@ -15,14 +15,14 @@ Moving baseline bit_base files ....
1515
Moving NATLEV.GrbF00 .........OK
1616
Moving NATLEV.GrbF06 .........OK
1717

18-
0: The total amount of wall time = 619.576254
19-
0: The maximum resident set size (KB) = 885380
18+
0: The total amount of wall time = 607.253608
19+
0: The maximum resident set size (KB) = 902116
2020

2121
Test bit_base PASS
2222

2323

2424
baseline dir = /scratch3/NCEPDEV/stmp/role.epic/ORT-stmp/role.epic/FV3_OPNREQ_TEST/OPNREQ_TEST/regional_control_std_base_gnu
25-
working dir = /scratch3/NCEPDEV/stmp/role.epic/ORT-ptmp/role.epic/FV3_OPNREQ_TEST/opnReqTest_976059/dcp_dcp
25+
working dir = /scratch3/NCEPDEV/stmp/role.epic/ORT-ptmp/role.epic/FV3_OPNREQ_TEST/opnReqTest_2966052/dcp_dcp
2626
Checking test dcp results ....
2727
Comparing dynf000.nc .....USING NCCMP......OK
2828
Comparing dynf006.nc .....USING NCCMP......OK
@@ -33,14 +33,14 @@ Checking test dcp results ....
3333
Comparing NATLEV.GrbF00 .....USING CMP......OK
3434
Comparing NATLEV.GrbF06 .....USING CMP......OK
3535

36-
0: The total amount of wall time = 495.569593
37-
0: The maximum resident set size (KB) = 854412
36+
0: The total amount of wall time = 492.265676
37+
0: The maximum resident set size (KB) = 865680
3838

3939
Test dcp PASS
4040

4141

4242
baseline dir = /scratch3/NCEPDEV/stmp/role.epic/ORT-stmp/role.epic/FV3_OPNREQ_TEST/OPNREQ_TEST/regional_control_std_base_gnu
43-
working dir = /scratch3/NCEPDEV/stmp/role.epic/ORT-ptmp/role.epic/FV3_OPNREQ_TEST/opnReqTest_976059/std_base_std_base
43+
working dir = /scratch3/NCEPDEV/stmp/role.epic/ORT-ptmp/role.epic/FV3_OPNREQ_TEST/opnReqTest_2966052/std_base_std_base
4444
Checking test std_base results ....
4545
Moving baseline std_base files ....
4646
Moving dynf000.nc .........OK
@@ -52,14 +52,14 @@ Moving baseline std_base files ....
5252
Moving NATLEV.GrbF00 .........OK
5353
Moving NATLEV.GrbF06 .........OK
5454

55-
0: The total amount of wall time = 490.978149
56-
0: The maximum resident set size (KB) = 841936
55+
0: The total amount of wall time = 490.937094
56+
0: The maximum resident set size (KB) = 863660
5757

5858
Test std_base PASS
5959

6060

6161
baseline dir = /scratch3/NCEPDEV/stmp/role.epic/ORT-stmp/role.epic/FV3_OPNREQ_TEST/OPNREQ_TEST/regional_control_std_base_gnu
62-
working dir = /scratch3/NCEPDEV/stmp/role.epic/ORT-ptmp/role.epic/FV3_OPNREQ_TEST/opnReqTest_976059/thr_thr
62+
working dir = /scratch3/NCEPDEV/stmp/role.epic/ORT-ptmp/role.epic/FV3_OPNREQ_TEST/opnReqTest_2966052/thr_thr
6363
Checking test thr results ....
6464
Comparing dynf000.nc .....USING NCCMP......OK
6565
Comparing dynf006.nc .....USING NCCMP......OK
@@ -70,11 +70,11 @@ Checking test thr results ....
7070
Comparing NATLEV.GrbF00 .....USING CMP......OK
7171
Comparing NATLEV.GrbF06 .....USING CMP......OK
7272

73-
0: The total amount of wall time = 493.580164
74-
0: The maximum resident set size (KB) = 876664
73+
0: The total amount of wall time = 488.797432
74+
0: The maximum resident set size (KB) = 852532
7575

7676
Test thr PASS
7777

7878
OPERATION REQUIREMENT TEST WAS SUCCESSFUL
79-
Fri Sep 26 16:08:02 UTC 2025
80-
Elapsed time: 00h:53m:04s. Have a nice day!
79+
Wed Oct 1 04:21:24 UTC 2025
80+
Elapsed time: 00h:53m:54s. Have a nice day!

0 commit comments

Comments
 (0)