Skip to content
Merged
Show file tree
Hide file tree
Changes from 24 commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
7bc760c
add notebook to visualize native grid data
chengzhuzhang Apr 16, 2025
cc21bd2
add skeleton for a the new lat_lon_native set
chengzhuzhang Apr 18, 2025
e902660
add dependencies; make simple plotting/viewer work
chengzhuzhang Apr 18, 2025
5e2741b
fix variable description in viewer
chengzhuzhang Apr 18, 2025
c33df95
make variable derivation work
chengzhuzhang Apr 21, 2025
cb49bf5
make 2d model only run plotting work
chengzhuzhang Apr 22, 2025
4740a8b
model vs model works
chengzhuzhang Apr 23, 2025
108a660
fix for region e.g. 60S60N work
chengzhuzhang Apr 23, 2025
05cb53a
add uxarray's nearest neighbor regridding and make diff plot work
chengzhuzhang Apr 23, 2025
1b2c5ff
clean up debug lines
chengzhuzhang Apr 24, 2025
f3a9a89
reorg plot funcs
chengzhuzhang Apr 24, 2025
5291d78
create subset region metrics
chengzhuzhang Apr 28, 2025
99fdf26
update standard variable list for native grid plotting
chengzhuzhang Apr 28, 2025
fefb25f
fix sets in cfg
chengzhuzhang Apr 28, 2025
bab6b04
more code clean up
chengzhuzhang Apr 28, 2025
2898e8d
temporal fix for input data with n time snapshots
chengzhuzhang Apr 29, 2025
bb7dd22
add TGCLDLWP variable
chengzhuzhang May 1, 2025
4605fac
update default var in cfg
chengzhuzhang May 2, 2025
da3b9a3
fix pre-commit errors
chengzhuzhang Jul 28, 2025
4340524
pin uxarray to avoid latest version(2025.6.0)
chengzhuzhang Jul 29, 2025
399a95e
simplify derived var logic;code cleanup
chengzhuzhang Jul 31, 2025
8f9521e
create new get_native_dataset function
chengzhuzhang Aug 1, 2025
8fa1800
add time snapshot selection function for visualize native history file
chengzhuzhang Aug 1, 2025
60f1084
update granulation list to avoid looping over seasons; fix viewer
chengzhuzhang Aug 2, 2025
0233432
Refactor Jill's initial PR for maintainability
chengzhuzhang Aug 29, 2025
54ab380
fix importing ClimoFreq and using it
chengzhuzhang Sep 17, 2025
4f70de2
relax uxarray version by remove dependency on periodic_elements
chengzhuzhang Sep 17, 2025
8e5b847
Merge branch 'main' into native_grids_viz
chengzhuzhang Sep 17, 2025
d887088
refactor to use utils in plotting
chengzhuzhang Sep 18, 2025
8944b82
Merge branch 'native_grids_viz' of github.com:E3SM-Project/e3sm_diags…
chengzhuzhang Sep 18, 2025
8b72480
refactor metrics generation and using weighted mean,rsme,corr
chengzhuzhang Sep 18, 2025
5a71782
reformat testing run script
chengzhuzhang Sep 18, 2025
3af5471
make metrics work on regional subset
chengzhuzhang Sep 19, 2025
bf31962
Merge branch 'main' into native_grids_viz
chengzhuzhang Sep 30, 2025
ed2b5f0
clean up
chengzhuzhang Oct 3, 2025
33ba31e
update example script
chengzhuzhang Oct 3, 2025
11f8cd9
Merge branch 'native_grids_viz' of github.com:E3SM-Project/e3sm_diags…
chengzhuzhang Oct 3, 2025
2cca3ec
fix provenance command-line; fix metrics for model_only case
chengzhuzhang Oct 3, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion auxiliary_tools/LCF_mixed-phase_partition_diagram.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -220,7 +220,7 @@
],
"metadata": {
"kernelspec": {
"display_name": "Python [conda env:e3sm_unified_1.6.0_nompi] *",
"display_name": "Python [conda env:e3sm_unified_1.6.0_nompi]",
"language": "python",
"name": "conda-env-e3sm_unified_1.6.0_nompi-py"
},
Expand Down
1,454 changes: 1,454 additions & 0 deletions auxiliary_tools/plot_native_grid_data.ipynb

Large diffs are not rendered by default.

35 changes: 35 additions & 0 deletions auxiliary_tools/plot_native_grid_data.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
import uxarray as ux
import matplotlib.pyplot as pl
import cartopy.crs as ccrs
import cartopy.feature as cfeature

base_path = "/Users/zhang40/Documents/ACME_simulations/E3SM_v2/native_grid_data/"
grid_info = "ne30pg2"
grid_path = base_path + f"{grid_info}.nc"
data_path = base_path + f"PRECC.{grid_info}.nc"

uxds = ux.open_dataset(grid_path, data_path)

pc = uxds["PRECT"].to_polycollection()
#pc = uxds["PRECT"].to_polycollection(periodic_elements="split")

# disables grid lines
pc.set_antialiased(False)

pc.set_cmap("plasma")

fig, ax = plt.subplots(
1,
1,
figsize=(10, 5),
facecolor="w",
constrained_layout=True,
subplot_kw=dict(projection=ccrs.PlateCarree()),
)

ax.add_feature(cfeature.COASTLINE)
ax.add_feature(cfeature.BORDERS)

ax.add_collection(pc)
ax.set_global()
plt.title("PolyCollection Plot with Projection & Features")
1 change: 1 addition & 0 deletions conda-env/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ dependencies:
- numpy >=2.0.0,<3.0.0
- pywavelets
- scipy
- uxarray >=2023.3.0
- xarray >=2024.3.0
- xcdat >=0.9.1,<1.0.0
- xesmf >=0.8.7
Expand Down
1 change: 1 addition & 0 deletions conda-env/dev.yml
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ dependencies:
- numpy >=2.0.0,<3.0.0
- pywavelets
- scipy
- uxarray >=2023.3.0,<2025.6.0
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any reason why we're constraining <2025.6.0 here? Also ci.yml doesn't have this constraint.

- xcdat >=0.9.1,<1.0.0
- xesmf >=0.8.7
- xskillscore >=0.0.20
Expand Down
2 changes: 2 additions & 0 deletions e3sm_diags/derivations/default_regions_xr.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,8 @@
"20N50N": {"lat": (20.0, 50)},
"50N90N": {"lat": (50.0, 90)},
"60S90N": {"lat": (-60.0, 90)},
"45S45N-120E60W": {"lat": (-45.0, 45), "lon": (120, 300)},
"30S30N-150E90W": {"lat": (-30.0, 30), "lon": (150, 270)},
"60S60N": {"lat": (-60.0, 60)},
"75S75N": {"lat": (-75.0, 75)},
"ocean": {"value": 0.65},
Expand Down
12 changes: 12 additions & 0 deletions e3sm_diags/derivations/derivations.py
Original file line number Diff line number Diff line change
Expand Up @@ -624,6 +624,18 @@
("hfss",): rename,
("surf_sens_flux",): rename, # EAMxx
},
"TGCLDLWP": OrderedDict(
[
(
("TGCLDLWP",),
lambda x: convert_units(x, target_units="g/m^2"),
),
(
("LiqWaterPath",),
lambda x: convert_units(x, target_units="g/m^2"),
), # EAMxx
]
),
Comment on lines +627 to +638
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Replace with regular dictionary

"TGCLDLWP_OCN": OrderedDict(
[
(
Expand Down
119 changes: 119 additions & 0 deletions e3sm_diags/driver/default_diags/lat_lon_native_model_vs_model.cfg
Original file line number Diff line number Diff line change
@@ -0,0 +1,119 @@
[#]
sets = ["lat_lon_native"]
case_id = "model_vs_model"
variables = ["PRECT"]
seasons = ["ANN", "01", "02", "03", "04", "05", "06", "07", "08", "09", "10", "11", "12", "DJF", "MAM", "JJA", "SON"]
regions = ["global", "60S60N", "30S30N-150E90W"]
test_colormap = "WhiteBlueGreenYellowRed.rgb"
reference_colormap = "WhiteBlueGreenYellowRed.rgb"
diff_colormap = "BrBG"
contour_levels = [0.5, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 12, 13, 14, 15, 16]
diff_levels = [-2.5, -2, -1.5, -1, -0.5, -0.25, 0.25, 0.5, 1, 1.5, 2, 2.5]

[#]
sets = ["lat_lon_native"]
case_id = "model_vs_model"
variables = ["PRECC"]
seasons = ["ANN", "01", "02", "03", "04", "05", "06", "07", "08", "09", "10", "11", "12", "DJF", "MAM", "JJA", "SON"]
regions = ["global", "60S60N", "30S30N-150E90W"]
test_colormap = "WhiteBlueGreenYellowRed.rgb"
reference_colormap = "WhiteBlueGreenYellowRed.rgb"
diff_colormap = "BrBG"
contour_levels = [0.5, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 12, 13, 14, 15, 16]
diff_levels = [-2.5, -2, -1.5, -1, -0.5, -0.25, 0.25, 0.5, 1, 1.5, 2, 2.5]

[#]
sets = ["lat_lon_native"]
case_id = "model_vs_model"
variables = ["TGCLDLWP"]
seasons = ["ANN", "01", "02", "03", "04", "05", "06", "07", "08", "09", "10", "11", "12", "DJF", "MAM", "JJA", "SON"]
regions = ["global", "60S60N", "30S30N-150E90W"]
#test_colormap = "Blues"
#reference_colormap = "Blues"
diff_colormap = "RdBu"
#contour_levels = [10, 25, 50, 75, 100, 125, 150, 175, 200,225, 250]
diff_levels = [-35, -30, -25, -20, -15, -10, -5, 5, 10, 15, 20, 25, 30, 35]



Comment on lines +37 to +38
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change

[#]
sets = ["lat_lon_native"]
case_id = "model_vs_model"
variables = ["TREFHT"]
regions = ["global"]
seasons = ["ANN", "01", "02", "03", "04", "05", "06", "07", "08", "09", "10", "11", "12", "DJF", "MAM", "JJA", "SON"]
contour_levels = [-35, -30, -25, -20, -15, -10, -5, 0, 5, 10, 15, 20, 25, 30, 35, 40]
diff_levels = [-5, -4, -3, -2, -1, -0.5, -0.2, 0.2, 0.5, 1, 2, 3, 4, 5]


[#]
sets = ["lat_lon_native"]
case_id = "model_vs_model"
variables = ["SWCFSRF"]
seasons = ["ANN", "01", "02", "03", "04", "05", "06", "07", "08", "09", "10", "11", "12", "DJF", "MAM", "JJA", "SON"]
contour_levels = [-170, -150, -135, -120, -105, -90, -75, -60, -45, -30, -15, 0, 15, 30, 45]
diff_levels = [-30, -25, -20, -15, -10, -5, -2, 2, 5, 10, 15, 20, 25, 30]


Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change

[#]
sets = ["lat_lon_native"]
case_id = "model_vs_model"
variables = ["LWCFSRF"]
seasons = ["ANN", "01", "02", "03", "04", "05", "06", "07", "08", "09", "10", "11", "12", "DJF", "MAM", "JJA", "SON"]
contour_levels = [0, 10, 20, 30, 40, 50, 60, 70, 80]
diff_levels = [-30, -25, -20, -15, -10, -5, -2, 2, 5, 10, 15, 20, 25, 30]


[#]
sets = ["lat_lon_native"]
case_id = "model_vs_model"
variables = ["LHFLX"]
seasons = ["ANN", "01", "02", "03", "04", "05", "06", "07", "08", "09", "10", "11", "12", "DJF", "MAM", "JJA", "SON"]
contour_levels = [0,5, 15, 30, 60, 90, 120, 150, 180, 210, 240, 270, 300]
diff_levels = [-75, -50, -25, -10, -5, -2, 2, 5, 10, 25, 50, 75]


Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change

[#]
sets = ["lat_lon_native"]
case_id = "model_vs_model"
variables = ["SHFLX"]
seasons = ["ANN", "01", "02", "03", "04", "05", "06", "07", "08", "09", "10", "11", "12", "DJF", "MAM", "JJA", "SON"]
contour_levels = [-100, -75, -50, -25, -10, 0, 10, 25, 50, 75, 100, 125, 150]
diff_levels = [-75, -50, -25, -10, -5, -2, 2, 5, 10, 25, 50, 75]


Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change

[#]
sets = ["lat_lon_native"]
case_id = "model_vs_model"
variables = ["NET_FLUX_SRF"]
seasons = ["ANN", "01", "02", "03", "04", "05", "06", "07", "08", "09", "10", "11", "12", "DJF", "MAM", "JJA", "SON"]
contour_levels = [-200, -160, -120, -80, -40, 0, 40, 80, 120, 160, 200]
diff_levels = [-75, -50, -25, -10, -5, -2, 2, 5, 10, 25, 50, 75]

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change


[#]
sets = ["lat_lon_native"]
case_id = "model_vs_model"
variables = ["TMQ"]
seasons = ["ANN", "01", "02", "03", "04", "05", "06", "07", "08", "09", "10", "11", "12", "DJF", "MAM", "JJA", "SON"]
contour_levels = [5, 10, 15, 20, 25, 30, 35, 40, 45, 50, 55, 60]
diff_levels = [-12, -9, -6, -4, -3, -2, -1, 1, 2, 3, 4, 6, 9, 12]


Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change

[#]
sets = ["lat_lon_native"]
case_id = "model_vs_model"
variables = ["QREFHT"]
seasons = ["ANN", "01", "02", "03", "04", "05", "06", "07", "08", "09", "10", "11", "12", "DJF", "MAM", "JJA", "SON"]
contour_levels = [0.2, 0.5, 1, 2.5, 5, 7.5, 10, 12.5, 15, 17.5]
diff_levels = [-5, -4, -3, -2, -1, -0.25, 0.25, 1, 2, 3, 4, 5]

[#]
sets = ["lat_lon_native"]
case_id = "model_vs_model"
variables = ["U10"]
seasons = ["ANN", "01", "02", "03", "04", "05", "06", "07", "08", "09", "10", "11", "12", "DJF", "MAM", "JJA", "SON"]
test_colormap = "PiYG_r"
reference_colormap = "PiYG_r"
contour_levels = [2, 4, 6, 8, 10, 12, 14, 16, 18, 20]
diff_levels = [-8, -6, -5, -4, -3, -2, -1, 1, 2, 3, 4, 5, 6, 8]
16 changes: 16 additions & 0 deletions e3sm_diags/driver/default_diags/lat_lon_native_model_vs_obs.cfg
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
[#]
sets = ["lat_lon_native"]
case_id = "model_vs_obs"
variables = ["PRECT"]
seasons = ["ANN", "DJF", "MAM", "JJA", "SON"]
regions = ["global"]
test_colormap = "WhiteBlueGreenYellowRed.rgb"
reference_colormap = "WhiteBlueGreenYellowRed.rgb"
diff_colormap = "BrBG"
contour_levels = [0.5, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 12, 13, 14, 15, 16]
diff_levels = [-2.5, -2, -1.5, -1, -0.5, -0.25, 0.25, 0.5, 1, 1.5, 2, 2.5]

# Native grid settings
grid_file = ""
split_periodic_elements = True
antialiased = False
Loading
Loading