dask-aware rasterize, geometry_mask, and exactextract

[dcherian]

I wrote some some dask-aware wrappers for rasterio's rasterize, geometry_mask, and exactextract

They're built using the GeoBoxTiles abstraction (which I quite like, nice work!).

Would you all be interested in merging that code here? The graph isn't as smart as your reprojection graph, but it could get there with some work (if needed).

[RichardScottOZ]

Where are tjose based Deepak?

[dcherian]

Apologies, those links weren't clear: See https://github.com/dcherian/rasterix/

[Kirill888]

@dcherian these look interesting, main concerns I have:

• not super clear which methods are external API and which are implementation details
  • adding doc comments and using _ prefix for internal functions would help
• extra lib dependencies, will require some rework to make code "import without error" when extra dependencies are not installed
  • odc-geo has minimal deps with many features only being supported if extras are installed
  • osgeo in particular is problematic for non-conda users, but I guess it's only for assert so can be dropped
• unit tests might need to be expanded, they look like "smoke test only" at the moment

ping @robbibt can you chime in on the proposed functionality, how useful these might be for data science workflows, is it duplicating something we already have outside of odc.geo?

[dcherian]

You are absolutely right on all counts. I whipped these up somewhat quickly for a customer of ours, and will definitely rework it once I figure out where it's going to land.

These links --- rasterize, geometry_mask, and exactextract --- link to the intended entrypoints and have reasonable docstrings now.

AFAICT there are no real equivalents anywhere else today (that's why I built them).

how useful these might be for data science workflows, is it duplicating something we already have outside of odc.geo?

The closest in odc-geo is rasterize which confusingly is a wrapper around rasterio's geometry_mask not rasterio's rasterize; and it works eagerly in-memory.

[robbibt]

Sorry for the delay in replying here @dcherian and @Kirill888 - I haven't had a chance to have an in-depth look at the code, but Dask-aware rasterization and masking tools would definitely be very useful for downstream applications. This one here also looks very handy too:

https://github.com/dcherian/rasterix/blob/eb7119a3808903a055e55c8861df8a49b904f643/rasterize.py#L478

Below is a list of similar existing functionality in odc-geo (which should be all accessible via the .odc. extension that gets added to xarray data when you run import odc.geo.xr). It would be pretty neat to be able to re-factor the backend for some of these public facing APIs so that we still kept the overall functionality but in a more Dask-friendly way!

https://odc-geo.readthedocs.io/en/latest/_api/odc.geo.xr.ODCExtension.crop.html
https://odc-geo.readthedocs.io/en/latest/_api/odc.geo.xr.ODCExtension.mask.html
https://odc-geo.readthedocs.io/en/latest/_api/odc.geo.xr.rasterize.html

(rasterize being not the same as rasterio's rasterize is a bit tricky, as it would be difficult to rename at this point... does geometry_mask just produce a boolean version of what rasterize produces? If so, perhaps we could just have a param to switch between them)