v0.9.0

Earth2Studio general release 0.9.0

🚨 Breaking Changes 🚨

• This is the last release with Python 3.10 support! Please upgrade Python version to at least 3.11 for main branch and future releases.

📕 Changelog

🌎 Added

• Async Zarr IO backend with non-blocking write calls
• Different compression codec support in the ZarrBackend with zarr_codecs parameter
• IO performance example
• Unified CorrDiff Wrapper
• Added UV script dependencies to all examples
• New metrics: Brier score, fractions skill score, log spectral distance, mean absolute error.
• Option to compute error of ensemble mean in rmse and mae.
• Added FourCastNet 3 model

⚙️ Changed

• Zarr IO Backend now uncompressed by default
• Allow HCBV perturbation to handle constant outputs (like land sea mask, or
  geopotential at surface)
• test/models/dx/test_corrdiff.py is now test/models/dx/test_corrdiff_taiwan.py
• Updated APIs for optional dependency managment utils with improved error messages
• Allow Zarr backends to user datetime and timedelta arrays for Zarr 3.0

🔨 Fixed

• Incorrect datetime utc timezone calculation in SFNO wrapper was fixed.
• DLWP output coords lead_time array to have proper shape
• Fixed data sources using GCFS throwing error at end of script from aiohttp session
  clean up
• Fixed HRRR_FX valid lead time check for date times not on 6 hour interval
• Removed time limits for WB2 climatology data source

📦 Dependencies

• Adding rich to core dependencies
• Updated torch-harmonics to 0.8.0
• Updated Makani to 0.2.1

v0.8.1

Earth2Studio patch release 0.8.1

🚨 Breaking Changes 🚨

• No notable breaks this release 🎉

📕 Changelog

⚙️ Changed

• Updated default StormCast package version to 1.0.2

🔨 Fixed

• NGC filesystem from API change in version >=3.158.1 of ngcsdk

📦 Dependencies

• Removed ngcsdk dependency requirement for public NGC packages

v0.8.0

Earth2Studio general release 0.8.0

🚨 Breaking Changes 🚨

• No notable breaks this release 🎉

📕 Changelog

🌎 Added

• Added GraphCast operational model (0.25 degree resolution)
• Added Graphcast 1 degree model
• Added SolarRadiationAFNO diagnostic model for predicting surface solar radiation
• Added DataArrayPathList for reading local data using glob patterns or explicit file lists
• Added Climate in a Bottle (cBottle) data source
• Added Climate in a Bottle (cBottle) Infilling diagnostic model
• Added Climate in a Bottle (cBottle) Super Resolution diagnostic model
• Added S2S recipe

⚙️ Changed

• In recipes, renamed requirements.txt -> recipe-requirements.txt

🔨 Fixed

• Fixed NCAR data source lat / lon labels and cache reads
• Fixed FuXi tp06 field input to be mm
• Fixed fsspec async filesystem initialization in data sources
• Fixed bug in GFS_FX forecast source which had lead time fixed at 0

📦 Dependencies

• Moved NGC SDK to optional dependencies due to it causing slow version resolutions
• Removing upper Python restriction on Rapids install for TC trackers

v0.7.0

Earth2Studio general release 0.7.0

🚨 Breaking Changes 🚨

• Many data sources have been completely refactored with new Async functionality, main call APIs remain the same but could impact more advanced use cases
• WeatherBench2 updated to Zarr 3.0 async only, this data source does not support Zarr 2.0
• Stormcast coordinates have been updated to HRRR index based coordinates versus lat/lon
• Curvi-linear grid functionality removed from the Random data source

📕 Changelog

🌎 Added

• Added AIFS model wrapper with state caching functionality for improved performance
• Added two cyclone trackers and related utilities
• Added HENS checkpoint example
• Added Earth2Studio recipes folder, documentation and template
• Added DLESyM and DLESyMLatLon atmosphere and ocean prognostic models
• Added HENS recipe with TC tracking and storm ensembling examples

⚙️ Changed

• Hemispheric centred bred vector perturbation now supports single/odd batch sizes
• Refactored NCAR ERA5 source to have async structure
• Refactored GFS and GFS_FX to have async structure
• Refactored GEFS and GEFS_FX to have async structure
• Refactored HRRR and HRRR_FX to have async structure
• Refactored WB2ERA5 and WB2Climatology for async Zarr 3.0
• Expanded the data source protocol to also include async fetch functions for async data sources
• Updated StormCast coords to be HRRR index, output coords still provide lat lon
• Interpolation AFNO model load_model now accepts prognostic model

❌ Removed

• Removed curvilinear from Random data source

🔨 Fixed

• Fixed the asyncio zarr access in the ARCO data source
• Partially fixed multiple tqdm print outs when using the built in workflows
• Generalized CorrelatedSphericalGaussian to support input tensors of higher dims

🔒 Security

• Remove pickle load from Aurora model with direct numpy array loads

📦 Dependencies

• Set default torch version to cuda 12.8

v0.6.0

Earth2Studio general release 0.6.0

🚨 Breaking Changes 🚨

• PhysicsNeMo and other packages removed from core dependencies. Optional dependencies needed for all models now. See install guide for details
• UV package manager is used now for development and is the recommended way to interact with Earth2studio. This allows us to give users a better package install experience for the growing number of models Earth2Studio supports

📕 Changelog

🌎 Added

• Hemispheric centered bred vector perturbation from HENS
• Add Aurora model to prognostic models
• Added check_extra_imports util for informative errors when optional dependencies are not installed
• Added wind gust AFNO diagnostic model
• Added diagnostic for relative humidity from temperature and specific humidity
• Added diagnostic for relative humidity from temperature and dew point
• Added diagnostic for wind speed magnitude
• Added diagnostic for vapor-pressure deficit
• Added PrecipitationAFNOv2 model for predicting tp06
• Added InterpModAFNO model for temporal interpolation of forecasts

🔨 Fixed

• WeatherBench2 Climatology data source with Zarr 3.0

📦 Dependencies

• Migrated repo / package to uv package manager
• Removed physics-nemo, torch harmonics from base packages to enable CPU install
• Added optional dependency groups for all models
• Added optional dependency groups for other submodules
• Added docuementation for build, install and package management for developers
• Migrated build system to hatch
• Moved dev and doc optional dependencies to uv dependency groups

v0.5.0

Earth2Studio general release 0.5.0

🚨 Breaking Changes 🚨

• For Zarr users, newer Zarr 3.0 support removes datetime / timedelta support in the coordinates. Will be fixed in later versions.
• Modulus package renamed to PhysicsNeMo, dependency updated accordingly

📕 Changelog

🌎 Added

• Add StormCast model to prognostic models
• Interpolation between arbitrary lat-lon grids
• Added hybrid level support to HRRR data source
• Added NCAR ERA5 data source
• Added multidim IO support
• Added forecast data source support to fetch_data
• Added stormcast deterministic and ensemble examples
• Added Random_FX as a random forecast data source
• Added interpolation support to run functions
• Added fair CRPS metric
• Added basic coordinate roll support in map_coords

⚙️ Changed

• Switched HRRR data source back to AWS grib
• Make source an argument for IFS, default of aws
• Changed CorrDiff output coordinates to actual lat/lon instead of ilat/ilon
• Changed the NetCDF4Backend to use proleptic gregorian calendar for time
• Changed the units assigned from the NetCDF4Backend to hours instead of h

🔨 Fixed

• Fixed bug in prep_data_array that implicitly assumed order of coordinates
• Fixed bug in rank_histogram that assumed broadcastable ensemble dimension
• Fixed spread/skill ratio to actually return spread/skill instead of skill/spread
• Fixed NGC download APIs and public API fetching for model files
• Fixed bug when using HRRR datasource in Jupyter notebooks
• Fixed ARCO for Zarr 3.0 and made proper async running with notebook support
• Fixed WB2 data source for Zarr 3.0 support
• Fixed Zarr IO for Zarr 3.0, for Zarr 3.0 datetime and timedeltas stored as int64
• Fixed CorrDiff and Stormcast for Zarr 3.0 support
• Fixed examples for Zarr 3.0 support updates

📦 Dependencies

• Updates to multiple dependencies for Python 3.12 support
• Added StormCast to optional dependencies
• Update to physicsnemo version 1.0.0
• Added nest asyncio to data dependencies for async data sources

v0.4.0

Earth2Studio general release 0.4.0

🚨 Breaking Changes 🚨

• Removed available function from CDS datasource
• Moved HRRR data source to Zarr datastore on S3, resulting in reduced historical range

📕 Changelog

🌎 Added

• Added NCEP data store to GFS data source for real-time forecast apps

⚙️ Changed

• Set zarr chunks for lead time to size 1 in examples.
• Updated HRRR tp to be hourly accumulated (Grib index 090)
• Added tp to GFS_FX datasource (not supported by GFS)
• Moved HRRR data source to Zarr datastore on S3

❌ Removed

• Removed available function from CDS datasource

🔨 Fixed

• Fixed caching of data sources to be controlled with EARTH2STUDIO_CACHE env var

📦 Dependencies

• Moving several ECMWF dependencies to optional
• Adding minimum version for numpy
• Bump minimum CDS API version for new API
• Moving unique data packages to optional deps
• Removed Herbie as dependency

v0.3.0

Earth2Studio general release 0.3.0

🚨 Breaking Changes 🚨

• Removed tp06 from ARCO, use WB2 instead
• Updated NetCDF4 constructor API for better control

📕 Changelog

🌎 Added

• Forecast datasource API
• GFS forecast datasource
• GEFS (0.5deg and 0.25deg) forecast datasource
• HRRR forecast datasource
• Support for private NGC model packages

⚙️ Changed

• Refactored ARCO datasource to use asyncio
• Updated NetCDF4 and Zarr IO to take kwargs for root storage objects allowing better control over storage behavior. Breaking changes to NetCDF4 init API
• Changed the da property to DataSetFile and DataArrayFile to no longer be a property and moved xr_args to object instantiation.
• Improved map_coords to handle slices and identical coords more efficiently; removed unused ignore_batch argument.

❌ Removed

• Removed tp06 from ARCO, use WB2 instead

🔨 Fixed

• Fixed caching of data sources to be controlled with EARTH2STUDIO_CACHE env var

📦 Dependencies

• Restrict torch_harmonics version to >=0.5.0, <0.7.1
• Removed specific ONNX version requirement, newer ORT-gpu versions appear to operate fine with CUDA 12

v0.2.0

Earth2Studio general release 0.2.0

🚨 Breaking Changes 🚨

• Changed utility function extract_coords to split_coords.
• Updated perturbation methods API PerturbationMethod -> Perturbation.
• Change 'input_coords' and 'output_coords' for models from a property to methods 'output_coords' accepts an input coordinates to determine the corresponding outputs.

📕 Changelog

🌎 Added

• Built in diagnostic workflow
• Basic diagnostic example
• Batch dimension userguide
• Parallel inference example
• Perturbation method section in userguide
• WeatherBench Climatology and ERA5 data source
• Added datasource_to_file utility function
• Add lagged ensemble perturbation method
• Add ACC and CRPS metrics
• Added the ability to reload zarr and netcdf backends
• Added the ability to read from an IOBackend
• Add spread/skill ratio
• Added FuXi weather model
• Added rank histogram
• Added reduction_dimensions as required property in statistics and metrics API
• Added Lexicon and Automodel userguide
• Added an 'output_coords' method to Statistics and Metrics.
• Added IMERG data source

⚙️ Changed

• Changed utility function extract_coords to split_coords
• Batched coordinate arrays now use np.empty(0) instead of np.empty(1)
• Improving user guide layout and developer documentation
• Updated perturbation methods API PerturbationMethod -> Perturbation
  These now generate noise and apply it to the input tensor
• Removed original Perturbation class
• Updated SFNO coordinates to include optional batch dimension
• NetCDF reads are now mode='r+' instead of 'w'
• Change 'input_coords' and 'output_coords' for models from a property to methods 'output_coords' accepts an input coordinates to determine the corresponding outputs
• Updated Package to use WholeFileCacheFileSystem. Extend package API to open and resolve. Deprecation warning added to get()

🔨 Fixed

• Enable version switch in documentation site
• Longitude coordinates of precip and climatenet diagnostic models
• Fixed pressure levels of IFS datasource to include all available

📦 Dependencies

• Bump Modulus required version to 0.6.0
• PyUpgrade pre-commit hook for Python 3.10
• Removed boto3
• Added ruamel.yaml, torch-harmonics, tensorly and tensorly-torch as optional deps for SFNO

v0.1.0

Earth2Studio general release 0.1.0

(Initial OSS release)