Description
The regridding support in datashader is based on numba code which expects float arrays. Therefore when a user passes integer arrays to canvas.raster
they are first cast to floats, then regridded and finally cast back to integers. Unfortunately this introduces highly undesirable artifacts which are very evident when using 'mean' aggregation. This is because numerical precision issues will cause values to fluctuate slightly around the integer values even in regions which are uniform in value, and when casting the floats with minor numerical precision differences to integers the minor differences are potentially amplified to a whole integer difference.
The question then is how we should handle mean/median/var/std aggregation on an integer array. I see a number of options:
- 'mean'/'var'/'median'/'std' aggregation on an integer arrays returns floats
canvas.raster
always returns float arrays- After 'mean' aggregation the float array is rounded before casting back to integers
My inclination is to go with option 1) but would welcome other suggestions.
Cc: @mmccarty