Missing data in SPI results

[monocongo]

SPI processing results show missing values in locations where we expect data. See attached image result of recent CMORPH SPI processing (gamma with missing data vs. Pearson3)

[monocongo]

My initial guess is that this is related to missing values being included in the calculation someplace. I will re-ingest with all missing values converted to Numpy.NaN and also make sure that all operations on arrays where missing values may be present are the numpy.nan*() whenever possible.

[monocongo]

The original suspicion that this is only happening with the gamma fitting appears to be incorrect, as we are also seeing missing values for Pearson fittings as well (dark blue areas in the SPI Pearson3 image below are missing values, clipped to the smallest SPI value for the plot).

[mmaoyi]

Hi James

I am using climate-indices from pypi and SPI shows missing values for both Pearson3 and Gamma.

[monocongo]

Thanks for your input, @mmaoyi . If you can send me your precipitation data that causes this result then I will look into this further and hopefully determine the cause of the error once and for all. If you can isolate which lat/lon values are having the missing/NaN results seen above then that will help me narrow down the problem input data. I assume it is a missing or out of range value that is not being appropriately handled. If so then I will add some data cleaning/preprocessing code to alleviate the issue.

[mmaoyi]

Hi James
Here is the link to the rainfall data https://drive.google.com/drive/folders/1DwxdYNCRQ5-JSrjNFFwWdlsjH4G-xXj1?usp=sharing

The missing values are mainly occuring over the following area LON=10W:30E and LAT=20N:0S.
There are some areas that also have missing values if you look at the global map.

Kind regards
Linda

[AYSEGULKZC]

Dear James Adams,

I calculated spi with gamma distribution. When the precipitation (input) is zero, i got the null spi values as a result. I know that the gamma function is undefined for zero values. But I might missed something.

Could you inform me about the problem and the solution?
Thank you for your time and consideration.

Yours sincerely,

Ayşegül Kuzucu

[monocongo]

Hello @AYSEGULKZC

If you can please post a link to your dataset and I will try to find the problem. I apologize but I am not actively maintaining this project as much as before because I am no longer using this for my work, so my response may be a long time coming. Perhaps you can find someone in the PyAOS group who can help, they seem to know both Python and climate science quite well.