v3.0.0

Summary of changes

• Non-backwards-compatible ("breaking") changes; this is a major release for zppy. Breaking changes are noted here:
  • The e3sm_to_cmip functionality is now accessed via its own task rather than via the ts task with ts_fmt = cmip. Users wanting to use the most up-to-date code for e3sm_to_cmip can do so more easily now. It also creates more independence between functionalities -- for example, if the ts job succeeds but the e3sm_to_cmip job fails, only the latter would have to be rerun.
    • Any users who have been using ts_fmt="cmip" will have to update their zppy cfg files accordingly. That primarily affects current ILAMB users. Parameter changes are noted below. More details can be found in the parameter documentation on default.ini.
    • Parameters no longer used anywhere and thus removed: e3sm_to_cmip_environment_commands, ts_fmt
    • Use this parameter in the e3sm_to_cmip task rather than the ts task: cmip_metadata
    • New parameters for the new e3sm_to_cmip task: cmip_vars, ts_grid, ts_subsection (it is also possible to set ts_atm_grid, ts_land_grid, ts_atm_subsection, and ts_land_subsection)
    • New parameters for the ilamb task: e3sm_to_cmip_atm_subsection, e3sm_to_cmip_land_subsection
  • As part of the update to v3.0.0, we also removed several deprecated parameters unrelated to e3sm_to_cmip and ilamb:
    • The scratch parameter, which had previously been used in the tc_analysis task but is no longer used in any form.
    • The atmosphere_only parameter in the global_time_series task. Now, simply remove the 3 ocean plots (change_ohc,max_moc,change_sea_level) from plots_original.
    • The plot_names parameter in the global_time_series task. Now, use plots_original.
• The global_time_series task is now implemented with a new package, zppy-interfaces. This should have little impact to most users, but it does open up the opportunity for users to create custom development environments of zppy-interfaces to use the most up-to-date plotting code. The zppy-interfaces package is meant for Python scripts that extend external packages for easy use by E3SM users, whereas zppy itself is meant for workflow automation (i.e., calling code written outside zppy).
  • As part of this effort, global_time_series has been extended to work for Land variables and to generate viewers similar to those in e3sm_diags.
• Parameter inference and validation have been improved and formalized. Users now have the ability to specify if they want zppy to infer parameters (for a file path or for the section name of a task dependency). The parameter space is checked thoroughly for possible error-producing configurations. This can prevent debugging hassles during runtime.
• CDAT is no longer a dependency for zppy. xCDAT is now user for handling global_time_series and e3sm_diags.
• The tc_analysis task underwent a number of improvements to be more robust.

Full list of changes

Incompatible API changes (user-facing)

• Refactoring: e3sm_to_cmip now has its own task and can no longer be called via the ts task.
  • #650: Add e3sm_to_cmip task
• Removing deprecated parameters for tc_analysis task (scratch), the global_time_series task ( atmosphere_only, plot_names) and the ts task (e3sm_to_cmip_environment_commands, ts_fmt).
  • #654: Remove deprecated parameters

Features (user-facing)

• Refactoring: global_time_series code moved to the new zppy-interfaces repo
  • #642: zppy-interfaces refactor
• CDAT Migration
  • #519: Replace CDAT
  • #611: Add center_times to xcdat
  • #651: Enable CDAT-migrated E3SM Diags
• Improved parameter inference
  • #628: Improve input validation and testing
  • (bug fix) #668: Use ref_final_yr for model vs model
  • #675: Explain parameter errors
  • #672: Allow e3sm_to_cmip to use ilamb parameter names
  • #682: Improve parameter inference
• Added global_time_series support for Land variables & viewers similar to those in e3sm_diags
  • #648: Add Viewer support
  • #697: Improve Land variable handling
• Updates to TC Analysis
  • #631: Make TC Analysis parallel
  • #632: Update TC Analysis template
  • #689: Improve res handling in tc_analysis
• #603: Handle dependencies with smaller time increments

Small Improvements (user-facing)

• #659: Skip oceanConservation task in MPAS-Analysis
• #661: Update conda recipe to be more like conda-forge feedstock
• #663: Use updated NCO
  • (bug fix) #686: Remove npo flag
• #696: Update data config for ilamb

Bug Fixes (user-facing)

• #602: Fix clm2 support
• #623: Improve carryover dependency handling
  • (follow-up bug fix) #679: Remove climo_land_subsection
• #633: Update to diags handling
• #652: Fix tmp directories
• #678: Remove MPAS-Analysis ocean conservation from default config
• #702: Fix ilamb template for piControl year range (with leading0)
  • (follow-up bug fix) #705: Fix year in ilamb template

Documentation/examples only (user-facing)

• #694: add example cfg for v3.LR.historical run on LCRC
• #706: Update parameters reference

Tests only

• #608: Add minimal case cfgs
• #614: Fix output path in min cases
• #615: Add TC Analysis tests
• #604: Testing update
• #617: Use latest diags in test cfgs
• #638: Fix dir issues in comparing images
• #646: Test fixes
• #664: Update testing paths and commands
• #676: Use Unified unless specified otherwise
• #677: Add min-case cfg for deprecated parameters
• #684: Factor integration tests
• #699: Improve image checker
• #700: Update test names to reflect new image checker
• #701: Further image checker updates

DevOps

• #620: Update supported python versions
• #621: Tweak behaviors of build and release CI workflows
• #630: Update python version in setup.py
• #627: Update pre-commit dependencies

Version Updating

• #660: Bump to 3.0.0rc1
• #665: Bump to 3.0.0rc2
• #683: Bump to 3.0.0rc3
• #687: Bump to 3.0.0rc4
• #691: Bump to 3.0.0rc5
• #707: Bump to 3.0.0

Other code changes

• #612: Remove climo_diurnal_input_files internal parameter

Repository maintenance

• #618: Add PR template
• #658: Update PR template
• #662: Further PR template updates