v3.1.0rc2

Extend snapshot support for other core sets and doc updates (#1013)

* Extend snapshot support for other core sets

* add new util module

* fix tests

* fix mean_2d_viewer

* add testing scripts

* clean up; clarify on input format for time_slices

* update testing scripts

* update README.md for 3.1.0 new feature

* Update documentation for v3.1.0 features and parameters

This commit updates the documentation to reflect new features and
parameters added in v3.1.0:

1. Updated available-parameters.rst:
   - Added time_slices parameter for snapshot analysis
   - Added lat_lon_native set-specific parameters (test_grid_file,
     ref_grid_file, antialiased)
   - Added test_file and ref_file parameters
   - Updated regrid_tool default from 'esmf' to 'xesmf'
   - Updated granulate default to include 'time_slices'
   - Updated sets default list to include all current diagnostic sets
   - Added notes about mutual exclusivity of time_slices and seasons

2. Updated examples.rst:
   - Added Example 8: Native Grid Visualization
   - Added Example 9: Snapshot Analysis for Core Sets
   - Updated running instructions to include new examples
   - Updated batch script example with ex8 and ex9

3. Created example files:
   - examples/ex8-native-grid-visualization/ (ex8.py, diags.cfg, README.md)
   - examples/ex9-snapshot-analysis/ (ex9.py, diags.cfg, README.md)

These changes document the two major features introduced in v3.1.0:
- Native grid visualization using UXarray
- Snapshot analysis for core diagnostic sets using time_slices

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>

* fix examples; update default set for lat_lon_native

* Refactor implementation of I/O for retrieving time-sliced datasets
- Refactor drivers for polar, meriodional_mean_2d, zonal_mean_2d, zonal_mean_xy to use `driver.utils.io._get_xarray_datasets()`
- Add `driver.io._get_xarray_datasets()` utility to simplify fetching of xarray datasets based on time selection
- Update references to `season` to `time_selection` with `TimeSelection` annotation
- Move `_set_time_slice_name_yrs_attrs()` back to `LatLonNativeParameter` because it is only used there -- consider refactoring later
- Remove `time_slice.py` as this functions were converted to `CoreParameter` methods

* Add unit tests
- Test coverage for get_time_sliced_dataset(), _add_time_series_filepath_attr(), check_values(), and get_xarray_datasets()`

* Fix method reference in lat_lon_native_driver.py

* Update e3sm_diags/driver/lat_lon_driver.py

Co-authored-by: Copilot <[email protected]>

* Apply Copilot review suggestions

* minor doc change

---------

Co-authored-by: Claude <[email protected]>
Co-authored-by: tomvothecoder <[email protected]>
Co-authored-by: Copilot <[email protected]>

v3.1.0rc1

Bump to 3.1.0rc1 (#1010)

v3.0.1

This release includes enhancements and bug fixes that primarily impact users working with high-frequency EAMxx output, polar maps, and high-resolution streamflow diagnostics, as well as those affected by bugs introduced in the major v3 release. Notable updates include tropical_subseasonal set support for eamxx, improved regridding accuracy through explicit 2D masking, dynamic-resolution support in streamflow diagnostics, and the addition of new variables. Visualization layout fixes are also included for improved PDF output handling.

User-facing changes

• Update tropical_subseasonal set for eamxx output by @chengzhuzhang @jjbenedict in #982
• Creates a mask for 2d datasets before regridding by @chengzhuzhang @tomvothecoder in #974
• Add pad_year function for xarray time slicing with years <1000 by @tomvothecoder in #972
• Add U10 and CERES EBAF TOA v4.1 to polar maps by @chengzhuzhang in #967
• Fix streamflow diagnostics to support dynamic resolution by @chengzhuzhang @hydrotian in #969
• EAMxx: add AODvis to derivations by @mahf708 in #983
• Fix layout for sub-plot saved in pdf format by @chengzhuzhang in #984
• Add support for saving regridded datasets to netCDF files by @chengzhuzhang in #963

Other Changes

• Documentation: Add jupyter notebooks documenting metrics diffs by @chengzhuzhang in #964
• Update setup-miniconda from v2 to v3 by @tomvothecoder in #980
• Investigate performance degradation with latest dependencies by @tomvothecoder in #986
• Fix premature plt.close() resulting in no subplots saved by @tomvothecoder in #989
• Drop Python 3.9 support and add support for Python 3.13 by @tomvothecoder in #921
• Fix xcdat bounds dim issue with v0.9.1 by @tomvothecoder in #992
• Update pre-commit dependencies and ruff hook ID by @altheaden in #991
• Bump to 3.0.1 by @chengzhuzhang in #993

Full Changelog: v3.0.0...v3.0.1

v3.0.0 Modernization with Xarray and xCDAT

What's Changed

v3.0.0 marks a major milestone after nearly two years of work by the core development team. This release introduces a completely new back-end, replacing CDAT with Xarray and xCDAT. Due to the significant scale of code changes, this has been incremented as a major release. The user-facing API for running E3SM Diagnostics remains backward-compatible between v2 and v3.

The modernization improves performance, usability, and maintainability, paving the way for future enhancements to E3SM development. The refactored codebase is now more robust and extensively covered by unit tests, setting a solid foundation for ongoing testing and development.

This release also includes several new enhancements including support for more EAMxx variables, standardization of regridding with xESMF's conservative_normed method, and improving diagnostic run performance by skipping unnecessary regridding of datasets with the same lat-lon grid.

We are excited to share this release with the E3SM community and remain committed to continuously improving the software!

Enhancements

• Add support for more EAMxx variables by @chengzhuzhang, @tomvothecoder, and @czender in #880
• Standardize regridding to conservative_normed by @tomvothecoder and @chengzhuzhang in #940
• Improve diagnostic run performance by skipping regridding for two datasets on same lat-lon grids by @chengzhuzhang in #959
• Revamp logging to improve readability and output E3SM diagnostic configuration, prevent duplicate log messages, and save complete console output to log file in prov directory by @tomvothecoder in #924
• Make viewer accessible at the top-level of results directory by @tomvothecoder in #927

Bug Fixes

• Fix prov files not printing with links in prov index.html by @tomvothecoder in #937

Refactor and testing

• Merge v3.0.0 development branch (cdat-migration-fy24) into main with the new Xarray/xCDAT backend by @tomvothecoder, @chengzhuzhang, @forsyth2, and @xylar in #902

DevOps

• Replace deprecated SciPy cwt with PyWavelets cwt by @tomvothecoder, @chengzhuzhang, and @whannah1 in #913
• Make scipy a base dependency in ci.yml and dev.yml by @tomvothecoder in #905
• Constrain dask !=2024.12.0,!=2024.12.1 for performance by @tomvothecoder in #926
• Replace data downloads from LCRC webserver to avoid throttling by using data containers by @mahf708 in #901
• Adopt ruff as the central tool for linting and formatting by @tomvothecoder in #920

Full Changelog: v2.12.1...v3.0.0

v3.0.0rc4

Refer to the v3.0.0 changelog for more information.

v3.0.0rc3

Refer to the v3.0.0 changelog for more information.

v3.0.0rc2

Refer to the v3.0.0 changelog for more information.

v3.0.0rc1

Refer to the v3.0.0 changelog for more information.

v2.12.1 A Fond Farewell to CDAT

As we release version v2.12.1 of E3SM Diags, we would like to take a moment to acknowledge and express our gratitude to the developers and contributors from the CDAT (Climate Data Analysis Tools) community. Over the years, CDAT has been an invaluable tool for our development and analysis processes. This release marks a milestone, as it will be the final version of E3SM Diags that includes a dependency on CDAT.

Looking ahead, E3SM Diags will transition to utilizing xarray/xCDAT tools, which align with our future goals and enhances the performance and usability of our software. We are committed to ensuring a smooth transition for our users and will provide backward compatibility in upcoming releases.

User-facing changes

• add Qbo wavelet analysis by @justin-richling @whannah1 in #836
• add H2OLNZ in zonal mean 2d stratosphere by @chengzhuzhang in #868
• make --closedcontourcmd in TC detection resolution-specific by @chengzhuzhang in #824
• Fixing African easterly wave density plots in TC analysis by @chengzhuzhang in #851
• Convert H2OLNZ units to ppm by volume by @chengzhuzhang in #874
• add_TCO_60S60N to default lat-lon variables by @chengzhuzhang in #879

Other Update

• Updates for tutorial2024 by @chengzhuzhang in #811
• Update tutorial by @forsyth2 in #812
• Fix a variable name displayed in viewer by @chengzhuzhang in #820
• fix save_ncfiles for annual_cycle_zonal_mean by @chengzhuzhang in #822
• Update cfg files by @chengzhuzhang in #830
• Fix integration test download script by @tomvothecoder in #835
• Drop support for python 3.8 by @altheaden in #844
• Tweak behaviors of build and release CI workflows by @altheaden in #847
• Release 2.12.1 by @chengzhuzhang in #893

New Contributors

• @altheaden made their first contribution in #844
• @justin-richling made their first contribution in #836

Full Changelog: v2.12.0...v2.12.1

v2.12.0

New Feature

• Add Wheeler and Kiladis diagram for tropical subseasonal diags - Wheeler and Kiladis by @jjbenedict @brianpm @chengzhuzhang @tomvothecoder in #732

Updates and bug fixes

• Add intermediate output in json for ARM Diags by @yunpengshan2014 in #806
• Edit in-place modified derived variables for fixing aerosol budget table by @mahf708 in #805

Full Changelog: v2.11.0...v2.12.0