Releases: E3SM-Project/zppy
Releases · E3SM-Project/zppy
v3.0.0
Summary of changes
- Non-backwards-compatible ("breaking") changes; this is a major release for
zppy. Breaking changes are noted here:- The
e3sm_to_cmipfunctionality is now accessed via its own task rather than via thetstask withts_fmt = cmip. Users wanting to use the most up-to-date code fore3sm_to_cmipcan do so more easily now. It also creates more independence between functionalities -- for example, if thetsjob succeeds but thee3sm_to_cmipjob fails, only the latter would have to be rerun.- Any users who have been using
ts_fmt="cmip"will have to update theirzppycfgfiles accordingly. That primarily affects current ILAMB users. Parameter changes are noted below. More details can be found in the parameter documentation ondefault.ini. - Parameters no longer used anywhere and thus removed:
e3sm_to_cmip_environment_commands,ts_fmt - Use this parameter in the
e3sm_to_cmiptask rather than thetstask:cmip_metadata - New parameters for the new
e3sm_to_cmiptask:cmip_vars,ts_grid,ts_subsection(it is also possible to setts_atm_grid,ts_land_grid,ts_atm_subsection, andts_land_subsection) - New parameters for the
ilambtask:e3sm_to_cmip_atm_subsection,e3sm_to_cmip_land_subsection
- Any users who have been using
- As part of the update to v3.0.0, we also removed several deprecated parameters unrelated to
e3sm_to_cmipandilamb:- The
scratchparameter, which had previously been used in thetc_analysistask but is no longer used in any form. - The
atmosphere_onlyparameter in theglobal_time_seriestask. Now, simply remove the 3 ocean plots (change_ohc,max_moc,change_sea_level) fromplots_original. - The
plot_namesparameter in theglobal_time_seriestask. Now, useplots_original.
- The
- The
- The
global_time_seriestask is now implemented with a new package,zppy-interfaces. This should have little impact to most users, but it does open up the opportunity for users to create custom development environments ofzppy-interfacesto use the most up-to-date plotting code. Thezppy-interfacespackage is meant for Python scripts that extend external packages for easy use by E3SM users, whereaszppyitself is meant for workflow automation (i.e., calling code written outsidezppy).- As part of this effort,
global_time_serieshas been extended to work for Land variables and to generate viewers similar to those ine3sm_diags.
- As part of this effort,
- Parameter inference and validation have been improved and formalized. Users now have the ability to specify if they want
zppyto infer parameters (for a file path or for the section name of a task dependency). The parameter space is checked thoroughly for possible error-producing configurations. This can prevent debugging hassles during runtime. - CDAT is no longer a dependency for
zppy.xCDATis now user for handlingglobal_time_seriesande3sm_diags. - The
tc_analysistask underwent a number of improvements to be more robust.
Full list of changes
Incompatible API changes (user-facing)
- Refactoring:
e3sm_to_cmipnow has its own task and can no longer be called via thetstask.- #650: Add e3sm_to_cmip task
- Removing deprecated parameters for
tc_analysistask (scratch), theglobal_time_seriestask (atmosphere_only,plot_names) and thetstask (e3sm_to_cmip_environment_commands,ts_fmt).- #654: Remove deprecated parameters
Features (user-facing)
- Refactoring:
global_time_seriescode moved to the newzppy-interfacesrepo- #642: zppy-interfaces refactor
- CDAT Migration
- Improved parameter inference
- Added
global_time_seriessupport for Land variables & viewers similar to those ine3sm_diags - Updates to TC Analysis
- #603: Handle dependencies with smaller time increments
Small Improvements (user-facing)
- #659: Skip oceanConservation task in MPAS-Analysis
- #661: Update conda recipe to be more like conda-forge feedstock
- #663: Use updated NCO
- (bug fix) #686: Remove npo flag
- #696: Update data config for ilamb
Bug Fixes (user-facing)
- #602: Fix clm2 support
- #623: Improve carryover dependency handling
- (follow-up bug fix) #679: Remove climo_land_subsection
- #633: Update to diags handling
- #652: Fix tmp directories
- #678: Remove MPAS-Analysis ocean conservation from default config
- #702: Fix ilamb template for piControl year range (with leading0)
- (follow-up bug fix) #705: Fix year in ilamb template
Documentation/examples only (user-facing)
Tests only
- #608: Add minimal case cfgs
- #614: Fix output path in min cases
- #615: Add TC Analysis tests
- #604: Testing update
- #617: Use latest diags in test cfgs
- #638: Fix dir issues in comparing images
- #646: Test fixes
- #664: Update testing paths and commands
- #676: Use Unified unless specified otherwise
- #677: Add min-case cfg for deprecated parameters
- #684: Factor integration tests
- #699: Improve image checker
- #700: Update test names to reflect new image checker
- #701: Further image checker updates
DevOps
- #620: Update supported python versions
- #621: Tweak behaviors of build and release CI workflows
- #630: Update python version in setup.py
- #627: Update pre-commit dependencies
Version Updating
- #660: Bump to 3.0.0rc1
- #665: Bump to 3.0.0rc2
- #683: Bump to 3.0.0rc3
- #687: Bump to 3.0.0rc4
- #691: Bump to 3.0.0rc5
- #707: Bump to 3.0.0
Other code changes
- #612: Remove climo_diurnal_input_files internal parameter
Repository maintenance
v2.5.0
Incompatible API changes (user-facing)
- N/A
Features (user-facing)
- #400: Generic plotting for global time series
- #560: Add wavenumber frequency set from e3sm_diags
- #582: Add reservation parameter
Small Improvements (user-facing)
Bug Fixes (user-facing)
- N/A
Documentation (user-facing)
Tests
- #576: Fixes and test/doc improvements
- #580: Small testing fixes
- #584: Updates for 2.5.0rc2 on Chrysalis
- #585: Run test utils on Perlmutter for v2.5.0rc2
- #586: Run test utils on Compy for v2.5.0rc2
- #589: Fix failures dir removal in tests
- #590: Fix failures dir removal in tests on Perlmutter
- #591: Fix failures dir removal in tests on Compy
DevOps
- N/A
Version Updating
Other code changes
- N/A
v2.4.0
Incompatible API changes (user-facing)
- N/A
Features (user-facing)
- #548: Add lat_lon_land from e3sm_diags for land diagnostics
Small Improvements (user-facing)
- #558: Specify year range for e3sm_to_cmip in ts
- #547: Remove waves and eke from MPAS-Analysis generate option
Bug Fixes (user-facing)
- N/A
Documentation (user-facing)
- #529: Dev guide updates
Tests
- #527: Add unique id to avoid overwrites
- #424: Native EAMxx support
- #554: Testing updates
- #556: Add e3sm_to_cmip_environment_commands parameter
- #564: Chrysalis changes to tests
- #565: Compy changes to tests
- #566: Perlmutter changes to tests
DevOps
- N/A
Version Updating
Other code changes
- N/A
v2.4.0rc1
New Features and User Facing Changes
- Native EAMxx support by @golaz in #424
- Add lat_lon_land from e3sm_diags for land diagnostics by @chengzhuzhang in #548
- Remove waves and eke from MPAS-Analysis generate option by @xylar in #547
- Add
e3sm_to_cmip_environment_commandsparameter by @forsyth2 in #556 - Make ref_name optional in e3sm diags output figure names by @forsyth2 in #524
Bug fixes
- Specify year range for e3sm_to_cmip in
tsby @chengzhuzhang in #558
Others
- Add unique id to avoid overwrites by @forsyth2 in #527
- Updates to GitHub templates by @forsyth2 in #525
- Update discussion link in template by @forsyth2 in #526
- Dev guide updates by @forsyth2 in #529
- Testing updates by @forsyth2 in #554
- Bump to 2.4.0rc1 by @chengzhuzhang in #559
Full Changelog: v2.3.0...v2.4.0rc1
Contributors
xylar, chengzhuzhang, and 2 other contributors
Assets 2
v2.3.1
Incompatible API changes (user-facing)
- N/A
Features (user-facing)
- N/A
Small Improvements (user-facing)
- #512: Fix url printed when submitting zppy jobs
- #511: Fix machine discovery on compute nodes
- #524: Make ref_name optional in e3sm diags output figure names
Bug Fixes (user-facing)
- #521: Reorder diags sets
Documentation (user-facing)
- #440: Dev Guide updates
Tests
- #506: Test Unified 1.9.0 on Chrysalis
- #508: Test Unified 1.9.0 on Compy
- #509: Test Unified 1.9.0 on Perlmutter
- #515: Automated doc updates
DevOps
- N/A
Version Updating
- #528: Bump to 2.3.1
Other code changes
v2.3.0
Incompatible API changes (user-facing)
- N/A
Features (user-facing)
- #402: Plugins for zppy
- #453: Allow generation of all variables
- #452: parallel option
- #459: Add hemispheric time series plots
Small Improvements (user-facing)
- #374: Fix input file error message
- #380: Print job ids to file
- #382: Refactor slurm header
- #389: Refactor coupled_global.py
- #381: Determine mapping file
- #417: Add support for clm2 cam in e3sm_to_cmip call
- #421: Support clm2 files
- #426: Allow more concise cfg
- #451: Help text
- #458: Update diags defaults
- #461: Make parallel optional for climo
Bug Fixes (user-facing)
- #384: Fix Mache invocations
- #457: Automatically determine climo_diurnal input files
- #466: Fixes for rc4
- #472: Fix mvm paths
- #497: add srun for running e3sm_diags
Documentation (user-facing)
- #375: Include Chrysalis cfgs
- #376: Include Perlmutter cfgs
- #377: Include Compy cfgs
- #378: Documentation on debugging
- #379: Improve git checkout in docs
- #433: Update release docs
- #436: Update defaults in docs
- #462: ncclimo vars documentation
- #460: Skip glb-only plots for hemispheric ts
- #504: Update defaults in docs
Tests
- #447: Fixes for Perlmutter
- #448: Fixes for Compy
- #449: Testing fixes
- #464: Perlmutter fixes for rc3
- #463: Compy fixes for rc3
- #487: Testing fixes for Chrysalis
- #486: Testing fixes for Compy
- #488: Testing fixes for Perlmutter
- #492: Testing fixes for Chrysalis Unified rc14
- #493: Testing fixes for Compy Unified rc14
- #494: Testing fixes for Perlmutter Unified rc14
- #501: Testing fixes for Compy Unified rc16
- #502: Testing fixes for Chrysalis Unified rc16
- #503: Testing fixes for Perlmutter Unified rc16
DevOps
- #395: Make setup.py consistent with conda recipe
- #410: Update release process
- #428: Switch docs to mamba and Mambaforge
- #429: Change workflow to mamba
- #434: Fix GitHub Actions
- #435: Revert mamba
- #438: Change workflows to mamba, take 2
Version Updating
- #437: Bump to v2.3.0rc1 without tbump
- #443: Bump to v2.3.0rc2 without tbump
- #465: Bump to 2.3.0rc3
- #469: Bump to 2.3.0rc4
- #473: Bump to 2.3.0rc5
- #499: Bump to 2.3.0rc6
- #505: Bump to 2.3.0
Other code changes
- #432: Remove AMWG task
v2.2.0
Incompatible API changes (user-facing)
- #333: Perlmutter support (NOTE: This removes Cori support and is thus a breaking change. However, after Cori is retired, it won't be possible to notice this change. Therefore, we will not be incrementing the major version.)
Features (user-facing)
- #306: Hemispheric Averaging
Small Improvements (user-facing)
- #294: Refactor coupled_global plots (new
plot_namesparameter available) - #301: Update global time series plotting (multiple pages of plots will be made if there are more than 8 plots)
- #289: add aerosol aeronet as a default e3sm_diags set
- #335: Update
prc_typ - #338: Years fix (workaround for missing comma in
years) - #352: add aerosol budget tables as a default set for e3sm_diags (new in rc3)
Bug Fixes (user-facing)
- #219: ocean_month updates (allow for
global_time_seriesplots withyearsnot in increments of 10) - #337: Fix cdscan call
Documentation (user-facing)
- #317: Version defaults docs
- #308: Update global time series plotting docs
- #332: Update obs paths
- #344: Update E3SM Unified paths (new in rc2)
Tests
- #283: Integration test templates
- #284: Compy integration tests
- #282: Cori integration tests
- #322: Update NERSC paths to use CFS
DevOps
Version Updating
- #339: Update defaults in docs
- #341: Bump to 2.2.0rc1
- #345: Bump to 2.2.0rc2 (new in rc2)
- #353: Update defaults in docs (new in rc3)
- #354: Bump to 2.2.0rc3 (new in rc3)
- #363: Update defaults in docs (new in rc4)
- #364: Bump to 2.2.0rc4 (new in rc4)
- #372: Bump to 2.2.0 (new in final release)
Other code changes
- #330: Update mache references
- #349: Chrysalis fixes for rc3 (new in rc3)
- #350: Compy fixes for rc3 (new in rc3)
- #351: Perlmutter fixes for rc3 (new in rc3)
- #360: Chrysalis fixes for rc4 (new in rc4)
- #361: Compy fixes for rc4 (new in rc4)
- #362: Perlmutter fixes for rc4 (new in rc4)
- #371: Fixes for v2.2.0 (new in final release)
v2.2.0rc4
Incompatible API changes (user-facing)
- #333: Perlmutter support (NOTE: This removes Cori support and is thus a breaking change. However, after Cori is retired, it won't be possible to notice this change. Therefore, we will not be incrementing the major version.)
Features (user-facing)
- #306: Hemispheric Averaging
Small Improvements (user-facing)
- #294: Refactor coupled_global plots (new
plot_namesparameter available) - #301: Update global time series plotting (multiple pages of plots will be made if there are more than 8 plots)
- #289: add aerosol aeronet as a default e3sm_diags set
- #335: Update
prc_typ - #338: Years fix (workaround for missing comma in
years) - #352: add aerosol budget tables as a default set for e3sm_diags (new in rc3)
Bug Fixes (user-facing)
- #219: ocean_month updates (allow for
global_time_seriesplots withyearsnot in increments of 10) - #337: Fix cdscan call
Documentation (user-facing)
- #317: Version defaults docs
- #308: Update global time series plotting docs
- #332: Update obs paths
- #344: Update E3SM Unified paths (new in rc2)
Tests
- #283: Integration test templates
- #284: Compy integration tests
- #282: Cori integration tests
- #322: Update NERSC paths to use CFS
DevOps
Version Updating
- #339: Update defaults in docs
- #341: Bump to 2.2.0rc1
- #345: Bump to 2.2.0rc2 (new in rc2)
- #353: Update defaults in docs (new in rc3)
- #354: Bump to 2.2.0rc3 (new in rc3)
- #363: Update defaults in docs (new in rc4)
- #364: Bump to 2.2.0rc4 (new in rc4)
Other code changes
v2.2.0rc3
Incompatible API changes (user-facing)
- #333: Perlmutter support (NOTE: This removes Cori support and is thus a breaking change. However, after Cori is retired, it won't be possible to notice this change. Therefore, we will not be incrementing the major version.)
Features (user-facing)
- #306: Hemispheric Averaging
Small Improvements (user-facing)
- #294: Refactor coupled_global plots (new
plot_namesparameter available) - #301: Update global time series plotting (multiple pages of plots will be made if there are more than 8 plots)
- #289: add aerosol aeronet as a default e3sm_diags set
- #335: Update
prc_typ - #338: Years fix (workaround for missing comma in
years) - #352: add aerosol budget tables as a default set for e3sm_diags (new in rc3)
Bug Fixes (user-facing)
- #219: ocean_month updates (allow for
global_time_seriesplots withyearsnot in increments of 10) - #337: Fix cdscan call
Documentation (user-facing)
- #317: Version defaults docs
- #308: Update global time series plotting docs
- #332: Update obs paths
- #344: Update E3SM Unified paths (new in rc2)
Tests
- #283: Integration test templates
- #284: Compy integration tests
- #282: Cori integration tests
- #322: Update NERSC paths to use CFS
DevOps
Version Updating
- #339: Update defaults in docs
- #341: Bump to 2.2.0rc1
- #345: Bump to 2.2.0rc2 (new in rc2)
- #353: Update defaults in docs (new in rc3)
- #354: Bump to 2.2.0rc3 (new in rc3)
Other code changes
v2.2.0rc2
Incompatible API changes (user-facing)
- #333: Perlmutter support (NOTE: This removes Cori support and is thus a breaking change. However, after Cori is retired, it won't be possible to notice this change. Therefore, we will not be incrementing the major version.)
Features (user-facing)
- #306: Hemispheric Averaging
Small Improvements (user-facing)
- #294: Refactor coupled_global plots (new
plot_namesparameter available) - #301: Update global time series plotting (multiple pages of plots will be made if there are more than 8 plots)
- #289: add aerosol aeronet as a default e3sm_diags set
- #335: Update
prc_typ - #338: Years fix (workaround for missing comma in
years)
Bug Fixes (user-facing)
- #219: ocean_month updates (allow for
global_time_seriesplots withyearsnot in increments of 10) - #337: Fix cdscan call
Documentation (user-facing)
- #317: Version defaults docs
- #308: Update global time series plotting docs
- #332: Update obs paths
- #344: Update E3SM Unified paths (new in rc2)
Tests
- #283: Integration test templates
- #284: Compy integration tests
- #282: Cori integration tests
- #322: Update NERSC paths to use CFS
DevOps
Version Updating
Other code changes
- #330: Update mache references