CHANGELOG.md

Changelog

All notable changes to this project will be documented in this file.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.

Please add your functional changes to the appropriate section in the PR. Keep it human-readable, your future self will thank you!

0.5.42 (2026-08-13)

Features

• Clarify-timezone-datetime (#723) (05262c2)
• Cut dependency of anemoi-registy on anemoi-datasets (#696) (be8b64b)
• cutout: User-provided pre-computed cutout masks (#627) (448a96c)
• Fix timezone corner case (#720) (87771c9)

Bug Fixes

• Fix issue 718 (#724) (cffbe79)
• Repair tendency statistics across group boundaries (#727) (6751b19)

Documentation

• Update usage doc (#712) (e127d7f)

0.5.41 (2026-07-27)

Features

• Add consistency check for tabular (#687) (2b4db63)
• Broadcast-to-trajectories-in-open-dataset (#717) (3a14296)
• Create and ship py.typed (#714) (63dba5f)
• Deprecation warning for group by instead of silent ignore (#682) (f997ea3)
• Do not mutate source_config in bufr (#683) (9815ca5)
• Improve memory for trajectories dataset building (#693) (5ecabf0)
• Keep compatibility with zarr 3 (#694) (e2f5f68)
• Remove validation of dates by default (#691) (09d1903)
• Use group_by 1 for trajectories by default (#676) (6f6e964)

Bug Fixes

• create: Height coord standard name (#707) (943672d)
• Interpolate_frequency now preserves dtype (#710) (9e5218a)
• Negative index support (#703) (c1bbaa1)
• Pass options as kwarg in usage/_open (#690) (eff502e)
• Release rolling average assumption. (#678) (a86df41)

Documentation

• Replace "Coming soon" placeholders in CF data howto with actual content (#702) (3091241)

0.5.40 (2026-06-22)

Features

• Add missing dependency, psutil (#670) (e913047)
• Add window and cycle date column option to ODB source (#662) (d388399)
• Extract constants and climatologies from a dataset for use in inference (#628) (55af3ce)
• metadata: Store GRIB paramId in variables_metadata (#664) (f283884)
• Not resolution check for tabular (#672) (8956a2a)
• Option when checking variables compatibility (#669) (1345ba9)
• Support empty data for bufr (#673) (ca78ec0)
• Synthetic gridded dataset (#637) (8cbfc01)

Bug Fixes

• Netcdf level and unit metadata handling (#663) (c636598)
• Update metadata specific (#671) (ace93c6)

0.5.39 (2026-06-11)

Features

• Analyse-recipe command for workflow automation (#659) (836b6ba)
• Trajectories missing features 3 (#655) (ecd9d85)
• Trajectory select file (#656) (5d507a1)

0.5.38 (2026-06-09)

Features

• Add create-test-recipe workflow for test dataset creation and stabilize missing-date CI tests (#639) (8b82da8)
• Missing features for trajectories (#646) (62bdbb5)
• Renaming a dataset (#640) (2ae6c89)
• Trajectories datasets (#618) (46fe4b9)
• Trajectories missing features 2 (#652) (139ef75)

Bug Fixes

• Inspect failure when dataset is not finalised (#648) (1d73c01)
• Missing-dates-efficiency (#649) (a32e7d8)
• Trajectory metadata (#654) (fa92481)

0.5.37 (2026-05-27)

Features

• Add BUFR source (#635) (07cbdd2)
• Add units to datasets (#577) (2545ad9)
• Datasets tendencies overwrite (#633) (460bee4)
• Export json schema (#630) (f40ad9f)
• Fetch multiple chunks in parallel when reading from s3 (#617) (88e8c08)
• Keep compatibility with zarr 3 (#623) (7597d85)
• Pydantic recipes (#575) (0ded992)

Bug Fixes

• Allow ODB source to work with multiple input files (#634) (fbe30e2)

0.5.36 (2026-04-22)

Features

• Add CycleIntervalProvider and set_start_step_to_zero patch (#564) (2c8824c)
• Add planetary-computer-multipart source, tests, and docs (#610) (42117db)
• create: Add workaround for missing data at step zero (#565) (9fd4733)
• Fetch files from ecfs if path starts with ec: or ectmp: (#585) (9fb443a)
• Fix issue 569 (#574) (7f4e40a)
• Fix typo with duplicates (#580) (f33333e)
• Make anemoi-datasets agnostic to Zarr version (Optional support Zarr3) (#220) (ab8cd71)
• Observations feature branch (#480) (92d5ac9)
• Open datasets analytics (#576) (561dbd2)
• Remove https test (#608) (048e419)

Bug Fixes

• create: Repeated-dates (#572) (b73d533)
• Example accumulations section to user current accumulate API (#601) (9434007)
• Fix corner cases (#594) (bdd31ff)
• Fix race condition during build (#593) (66e2070)
• Fix read ahead while building (#611) (6d18e5e)
• Fix weatherbench test (#609) (f434a15)
• grib-index: Support querying float values (#520) (b089cd2)
• Improve MARS request handling for forecast datasets (#562) (f9efe39)
• Make dataset naming function public (#579) (b089bb0)
• Netcdf date/time metadata type should be int (#555) (9937fbe)
• Propagate resolution metadata when using anemoi_dataset source (#614) (784695c)
• Remove duplicate code (#590) (8e54420)
• Remove empty accumulators from accumulation computation (#561) (3bc087d)
• Replace pydantic class Config with ConfigDict (#592) (ce6b2ff)
• Rolling average regression (#587) (04f5b0b)

Documentation

• Docs minor fixes update concat yaml (#539) (dd73fda)

0.5.35 (2026-03-09)

Features

• Accept only copy from *.zarr to *.zarr, no '/' are allowed (#548) (d584e6e)
• create: Support FDB as base source for accumulate (#550) (506e397)
• Fix corner case when using cutout (#558) (a2594ab)

Bug Fixes

• Update argument name in complement.py and bump anemoi-transform dep version (#544) (d8ffa01)

Documentation

• Explain option allow_nans (#547) (00396bb)
• Fix accumulation availability specification (#551) (c26a9d8)

0.5.34 (2026-02-17)

Bug Fixes

• Bump anemoi-transform version (#541) (09de1b2)
• Bump anemoi-transform version to 0.1.25 (#543) (f66c269)
• Utils provenance update (#533) (5a453bd)

0.5.33 (2026-02-09)

Bug Fixes

• Expose kwarg in factory (#531) (abfa553)

0.5.32 (2026-02-06)

Bug Fixes

• Filter out timespan as key for accumulator (#516) (6fb7f2b)
• max_distance_km: Add new max_distance_km argument to cutout (#530) (a94f3bc)

0.5.31 (2026-02-05)

Bug Fixes

• Use spatial.py (anemoi-transform) instead of grids.py (#526) (719669e)

0.5.30 (2026-02-03)

Features

• Add optional patch for accumulation when metadata is wrong (#525) (8016551)

Documentation

• Correct typo of part to parts in incremental build #518 (#519) (78e8d75)

0.5.29 (2026-01-28)

Features

• Abstracting accumulation (#326) (57a2895)
• Add id and location for point data (#500) (8cf8a10)
• Add xarray patches for changing time dimension and summing variables (#494) (7c44d99)
• apply mask: Apply a boolean mask when opening a dataset (#496) (820eb39)

Bug Fixes

• '--test' changes grid/number (#386) (71238c6)
• Complement with max distance (#469) (a8666fa)
• Coordinate (multiple) indexing - check indices before comparison (#506) (859ed19)
• Grids for masked datasets, field_shape thinning (#504) (e7b8d58)
• Issue #477 (#478) (42b9e27)
• Pin min version of ruamel-yaml (#514) (e6037b0)
• Update README to reflect project maturity status (#465) (ec8a15e)

Documentation

• Document the option to specify types when selecting a GRIB message (#466) (8940839)
• Fix interpolate to interpolation key in combining page (#461) (990a5c3)

0.5.28 (2025-10-30)

Features

• Added rolling average on read (#448) (5cef720)
• Fix repeat dates (#458) (16b5c83)
• Refactor dataset create (#379) (c5408da)

Bug Fixes

• Allow grib source to be used for constant data (#428) (c1d7ff1)
• Broken source imports (#450) (#454) (d6ed174)
• Ensure missing value indicator in not a valid value in accumulat… (#316) (3541a1f)
• S3 access #453 (#459) (2d810e3)
• Xarray-zarr filters (#457) (5f2973f)

Documentation

• Documentation updates (#419) (4588175)

0.5.27 (2025-10-06)

Features

• Add 'model_level_number' to coordinate guesser (ecmwf/anemoi-datasets#402) (#403) (fa0b7e8)
• Extend xarray Variable.sel to handle scalar variables (#381) (1306d62)
• Improve conftest types (#395) (bd8143d)
• Remove python 3.9 from pyproject.toml. (#392) (7f3a250)
• sources: Add FDB source (#405) (9222d59)
• Validate dataset (#330) (34d06df)

Bug Fixes

• Allow multiple options to be provided to anemoi-datasets inspect (#372) (dea680e)
• Anemoi inspect wrong message (#429) (984ccef)
• Better handling of xarray metadata (#322) (4304cb4)
• Cleaning _build when building datasets (#413) (046a7eb)
• Define the missing forwards_subclass_metadata_specific method in the ZipBase class (#377) (2816314)
• Remove config_path in Config (#431) (2bf674f)
• Remove-obsolete-code-and-rename (#423) (ab3da59)
• Save_dataset (#383) (05b8f51)
• Support for sources with already opened datasets (#417) (9bc2d17)

Documentation

• Reference filters transform (#389) (4e5d278)

0.5.26 (2025-07-11)

Features

• Add planetary planetary source (#368) (c85337f)
• Extend xarray coordinate guesser to include pressure as a long_name (#335) (bcbffaa)
• Extented complement nearest functionality to support k>1 (#371) (a2c5f92)
• Point data (#358) (78c4f85)
• Relaxed compatibility check for horizontal grid. (#339) (a7dc859)
• Sort xarray coordinate with patch (#356) (638cbdb)
• Use pytest-xdist to run tests in parallel. (#346) (1d656ee)

Bug Fixes

• Deprecated filters in anemoi datasets (#350) (d08c6f5)
• grib-index: Match against filename, not full path (#374) (e71a952)
• Missing method in xarray flavour (#369) (d0b9588)
• Remove trailing slash from zarr paths. (#362) (9a9bfe5)

Documentation

• Fix pre-commit (#367) (6e6ebe2)
• Zarr dataset creation (#365) (622c261)

0.5.25 (2025-06-11)

Features

• Integrating non-regular datasets in anemoi for observations. (#306) (95a0fe4)

Bug Fixes

• Incremental dataset build tasks called regardless of presence of debug flag in CLI code (#294) (37afc0d)
• Regression in accumulations #354 (#355) (f9769d7)
• Remove 2 layers of build function (#348) (7a904c4)

0.5.24 (2025-05-23)

Features

• verify command (#279) (aed36d2)

Bug Fixes

• adapt to earthkit 0.14 (ignore_keys issue) (#331) (fb3ab8d)

0.5.23 (2025-05-07)

Features

• support accumulations that resets regularly (e.g. every 24h) (#314) (0cd772b)

0.5.22 (2025-05-05)

Features

• add command line tool to check naming conventions (#287) (38a81e2)
• copy datasets from http(s) sources (#307) (cedd9db)
• downloading zip files if needed (#288) (3024099)
• optionally search for zarr files at various location if not found (#281) (8bd174c)
• reset accumulations (#300) (b12666a)
• work on check (#311) (c6a8db9)

Bug Fixes

• bug in complement (#296) (4e57f7c)
• bugs affecting creation of datasets from xarray-zarr (#299) (09fcb51)
• fix copy (#289) (a79759f)
• Proper indent for parallel additions (#303) (0643fb8)

Documentation

• update some docstrings (#283) (d3fe801)

0.5.21 (2025-04-08)

Features

• more tests (#277) (0ea7d46)

Bug Fixes

• grib-index command default (#275) (04a37d8)
• pin version of numcodecs (#285) (d86f317)
• update version of transform (#284) (e72cdde)

Documentation

• some cleanup (#276) (496b82a)
• time interpolation example (#274) (9d19bd6)
• update documentation (#271) (85ea386)

0.5.20 (2025-03-31)

Features

• interpolate nearest spatial (#260) (e6c9af4)
• new data sources (#258) (708c816)
• save opened anemoi_dataset (#259) (4759dd9)

Bug Fixes

• Remove hardcoded indices and get pressure levels from keys directly (#257) (777fbac)

Documentation

• Add API Docs (#255) (be13424)

0.5.19 (2025-03-27)

Features

• 24h accumulations for era5 (#266) (33f7919)
• add environment variables in recipe (#228) (6d898aa)

Bug Fixes

• broken references in recipes (#269) (bbdf339)
• fix typo (#268) (a12f58f)
• swap meshgrid dimension ordering in xarray grid creation (#249) (938f3c9)

Documentation

• add reference to anemoi contributing guidelines (#265) (7322e8a)

0.5.18 (2025-03-25)

Features

• better error message (#252) (e74cbe9)

Bug Fixes

• modify execute function signature (#253) (e4ad1a4)

0.5.17 (2025-03-24)

Features

• add coordinate standard_name for rotated pole grid (#192) (5464347)
• better logging for group of dates (#235) (d7fc616)
• fix to_index (#225) (baefd70)
• plugin support (#241) (376ef1c)
• support sub-hourly steps (#188) (7f219e9)
• update test breaking due to noaa external data change (#204) (4b99ea6)

Bug Fixes

• allow xarrays with single value dimensions (#230) (ee9fbf8)
• create forcings based on xarray-zarr template (again) (#244) (84cb312)
• fix for bug #237 (#247) (de3cab8)
• Use set in computing statistics, faster and use less memory (#209) (e93dbc1)

Documentation

• add animation (#208) (2af2fd6)
• Docathon 2025 (#234) (fb68b95)
• fix readthedocs (#223) (ce1b44e)
• update doc with eccc-fstd, cdsapi and regrid (#201) (57a53fc)
• update project name (#246) (5ddd4d1)
• use new logo (#211) (76ecf15)

0.5.16 (2025-02-04)

What's Changed

Other Changes 🔗

• chore: synced file(s) with ecmwf-actions/reusable-workflows by @DeployDuck in #186

New Contributors

• @DeployDuck made their first contribution in #186

Full Changelog: https://github.com/ecmwf/anemoi-datasets/compare/0.5.15...0.5.16

Unreleased

Changed

• Fix metadata serialization handling of numpy.integer (#140)
• Fix negative variance for constant variables (#148)
• Fix cutout slicing of grid dimension (#145)
• Use cKDTree instead of KDTree
• Implement 'complement' feature
• Add ability to patch xarrays (#160)
• Add support of ECCC file formats (fstd)
• Add use_cdsapi_dataset option to mars and accumulations

Added

• Call filters from anemoi-transform
• Make test optional when adls is not installed Pull request #110
• Add wz_to_w, orog_to_z, and sum filters (#149)

0.5.8 - 2024-10-26

Changed

• Bugfix in auto_adjust
• Fixed precommit CI errors
• Improve tests
• More verbosity

Added

• Add anemoi-transform link to documentation
• Various bug fixes
• Control compatibility check in xy/zip
• Add merge feature
• Add support for storing supporting_arrays in checkpoint files
• Allow naming of datasets components
• Contributors file (#105)

Changed

• Upload with ssh (experimental)
• Remove upstream dependencies from downstream-ci workflow (temporary) (#83)
• ci: pin python versions to 3.9 ... 3.12 for checks (#93)
• Fix __version__ import in init

0.5.7 - 2024-10-09

Changed

• Add support to fill missing dates

Allow for unknown CF coordinates - 2024-10-04

Changed

• Add variables_metadata entry in the dataset metadata
• Update documentation

Changed

• Add variables_metadata entry in the dataset metadata

Changed

• Add variables_metadata entry in the dataset metadata

0.5.5 - 2024-10-04

Changed

• Allow for unknown coordinates when parsing CF input

Add support for (semi-)constant fields - 2024-10-03

Changed

• Fix failing zarr lat/lon tests

Bug fixes - 2024-10-01

Added

• Adding the user recipe in the dataset PR #59.
• Add repeated_dates action in create.

Changed

• Bug fix in create/rename

0.5.0 - Incremental builds and Rescaling - 2024-09-25

Added

• New rescale keyword in open_dataset to change units of variables #36
• Add support for constant fields when creating datasets
• Simplify imports

Changed

• Added incremental building of datasets
• Add missing dependency for documentation building
• Fix failing test due to previous merge
• Bug fix when creating dataset from zarr
• Bug fix with area selection in cutout operation
• add paths-ignore to ci workflow
• call provenance less often

Removed

• pytest for notebooks

0.4.5

Added

• New interpolate_frequency keyword in open_dataset
• CI workflow to update the changelog on release
• adds the reusable cd pypi workflow
• merge strategy for changelog in .gitattributes #25
• adds ci hpc config (#43)

Changed

• update CI to reusable workflows for PRs and releases
• Support sub-hourly datasets.
• Change negative variance detection to make it less restrictive
• Fix cutout bug that left some global grid points in the lam part
• Fix bug in computing missing dates in cutout option

Removed

0.4.4 Bug fixes

• Bug fix in accumulations() that did not work with webmars
• Bug fix when using hindcasts input (missing dates on leap years)

0.4.0 Minor Release

Added

• earthkit-data replaces climetlab

Removed

• climetlab

0.3.0 Minor Release

Added

• hindcast source

Changed

• updated documentation

0.2.0 Minor Release

Added

• statistics tendencies

Removed

• CubesFilter

0.1.0 Initial Release

Added

• Documentation
• Initial code release for anemoi-datasets: create datasets for data-driven weather-models
• open datasets
• combine datasets

Git Diffs: