Merge pull request #179 from emileten/relocate-data-loading-clean-cmip6

relocate data loading in standardize-gcm

Author: emileten

CHANGELOG.md

@@ -5,7 +5,8 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
 ## [Unreleased]
-
+### Fixed
+- Move in-memory data loading where it is needed for 360-days calendar conversion in clean-cmip6 (PR #179, @emileten)
 
 ## [0.16.2] - 2022-02-15
 ### Fixed

dodola/core.py

@@ -381,6 +381,11 @@ def standardize_gcm(ds, leapday_removal=True):
         if ds_cleaned.time.dtype == "int64":
             ds_cleaned["time"] = xr.decode_cf(ds_cleaned).time
         if cal == "360_day":
+
+            # Cannot have chunks in time dimension for 360 day calendar conversion so loading
+            # data into memory.
+            ds_cleaned.load()
+
             if leapday_removal:  # 360 day -> noleap
                 ds_converted = xclim_convert_360day_calendar_interpolate(
                     ds=ds_cleaned,

dodola/services.py

@@ -593,10 +593,6 @@ def clean_cmip6(x, out, leapday_removal):
     """
     ds = storage.read(x)
 
-    # Cannot have chunks in time dimension for 360 day calendar conversion so loading
-    # data into memory.
-    ds.load()
-
     cleaned_ds = standardize_gcm(ds, leapday_removal)
     storage.write(out, cleaned_ds)