LoweringError: list(array64, 1d, C): unsupported nested memory-managed object

[pandasambit15]

@vlouf

Hi,

I was using the unravel.unravel_3D_pyart utility to dealias a volume scan data from C-band radar, but I got this error:

LoweringError: list(array64, 1d, C): unsupported nested memory-managed object

I tried using the simple unravel.dealiasing_process_2D function as well, by passing on data for one elevation: range, azimuth, velocity, and nyquist_velocity arguments, but still the same error popped up.

My data has 11 sweeps/elevs, 1600 bins and 360 azimuth. Its a C-band radar with nyquist velocity of 8.0 m/s.

I am using Python 3.6, on Windows 10 64 bit machine, for the record.

Can you please help me out?
Regards,
Sambit

[vlouf]

Hi,

I've never seen this error before but by googling a bit it seems to be related to the Numba library. Can you tell me which version of Numba you are using, or post your python environment so that I can try to reproduce this error.

Cheers

[pandasambit15]

cfrad.CDWRTERLS_weather-DP_2017_11_30_05_03_47.nc <https://drive.google.com/file/d/1dpDBoG54qKNAwpgGAsCFHAbuiU5flVfZ/view?usp=drive_web> Hi Valentin, Thanks for your response. You might be right about the error being related to numba installation. As a matter of fact, I tried the code again on my laptop at home, with proper installations using pip/conda over the internet; and the code ran without an error message. My earlier error was from my office PC where I had installed numba from source, since it doesn't have internet access. Coming to the unravel package results, I am not sure what I am getting after using the unravel_3D_pyart module is correct or not. The results after the correction are weird. (see the sample images attached). I am attaching my code as well for your reference, as well as a sample raw radar ncfile. Note: Since the raw file is not exactly as per PyART format, I am using a simple ncfile ingest function to convert the file into a compatible radar object before ingesting it. Thanks and regards, Sambit *Sambit Kumar Panda* *Scientist/Engineer-SC, ASD/AOSG/EPSA,* *SAC(ISRO), Ahmedabad*

…

On Tue, Jul 28, 2020 at 5:40 AM Valentin Louf ***@***.***> wrote: Hi, I've never seen this error before but by googling a bit it seems to be related to the Numba library. Can you tell me which version of Numba you are using, or post your python environment so that I can try to reproduce this error. Cheers — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#8 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/APYEBN5FXMOZ3QPTHFXU7VLR5YJQVANCNFSM4PIOICHQ> .

[pandasambit15]

Hi @vlouf Did you manage to look the file and the script that I sent you? Regards, Sambit

…

On Tue, Jul 28, 2020, 2:54 PM Sambit Panda ***@***.***> wrote: cfrad.CDWRTERLS_weather-DP_2017_11_30_05_03_47.nc <https://drive.google.com/file/d/1dpDBoG54qKNAwpgGAsCFHAbuiU5flVfZ/view?usp=drive_web> Hi Valentin, Thanks for your response. You might be right about the error being related to numba installation. As a matter of fact, I tried the code again on my laptop at home, with proper installations using pip/conda over the internet; and the code ran without an error message. My earlier error was from my office PC where I had installed numba from source, since it doesn't have internet access. Coming to the unravel package results, I am not sure what I am getting after using the unravel_3D_pyart module is correct or not. The results after the correction are weird. (see the sample images attached). I am attaching my code as well for your reference, as well as a sample raw radar ncfile. Note: Since the raw file is not exactly as per PyART format, I am using a simple ncfile ingest function to convert the file into a compatible radar object before ingesting it. Thanks and regards, Sambit *Sambit Kumar Panda* *Scientist/Engineer-SC, ASD/AOSG/EPSA,* *SAC(ISRO), Ahmedabad* On Tue, Jul 28, 2020 at 5:40 AM Valentin Louf ***@***.***> wrote: > Hi, > > I've never seen this error before but by googling a bit it seems to be > related to the Numba library. Can you tell me which version of Numba you > are using, or post your python environment so that I can try to reproduce > this error. > > Cheers > > — > You are receiving this because you authored the thread. > Reply to this email directly, view it on GitHub > <#8 (comment)>, or > unsubscribe > <https://github.com/notifications/unsubscribe-auth/APYEBN5FXMOZ3QPTHFXU7VLR5YJQVANCNFSM4PIOICHQ> > . >

[vlouf]

Hi @pandasambit15 ,

I had a look at the file you've sent. I was able to read the file (even thought start/end sweep data are incorrect) and attempted to dealiase it (well the processing worked, as in no error, but the results are wrong)

Aliased velocity, dealiasing results:

First thing that is weird is the file is that it seems that there's no fill value and the missing reflectivities are equals to 25???? rhohv is so low that we can't really use it for filtering data. I will send you my step by step jupyter notebook by mail. I tried play with threshold but it's not really working.

I don't think unravel will work on this case, too much noise and variability. A better way to dealias this case would be probably to use some kind of VAD or double VAD technique using radiosounding data (I don't have code for that sorry, but look into pyart)?

Regards,
Valentin

[pandasambit15]

Hi @vlouf I am sorry that I forgot to mention about the rho hv values and fill values. You are right. rhohv values in the file are incorrect, its a problem with the scale factor and offset applied. The data writing software is outsourced, so I don't have any control over that. Although we have asked them for this correction. I was actually under the impression that UNRAVEL only uses only radial velocity and possibly spectrum width data only. Didn't know it requires the full suit of parameters. Anyways, thanks a lot for spending so much time on this and helping me out. I will try out to correct the rhohv values and the fill value issues once, and will run the unravel code again. I had tried the Zhang et al. 2006: Multi-pass velocity de-aliasing algorithm earlier. In fact I had written the entire code myself in python following the paper, but it still gives me some artifacts. That was the reason I was hoping UNRAVEL would come to my rescue. Nevertheless, your support is much appreciated. Thanks and regards, Sambit *Sambit Kumar Panda* *Scientist/Engineer-SC, ASD/AOSG/EPSA,* *SAC(ISRO), Ahmedabad*

…

On Wed, Aug 5, 2020 at 10:37 AM Valentin Louf ***@***.***> wrote: Hi @pandasambit15 <https://github.com/pandasambit15> , I had a look at the file you've sent. I was able to read the file (even thought start/end sweep data are incorrect) and attempted to dealiase it (well the processing worked, as in no error, but the results are wrong) [image: image] <https://user-images.githubusercontent.com/19787525/89373579-6c13ef00-d72c-11ea-9d21-6d552f7fbafb.png> Aliased velocity, dealiasing results: [image: image] <https://user-images.githubusercontent.com/19787525/89373609-81891900-d72c-11ea-9cfd-45f7d869f71a.png> First thing that is weird is the file is that it seems that there's no fill value and the missing reflectivities are equals to 25???? rhohv is so low that we can't really use it for filtering data. I will send you my step by step jupyter notebook by mail. I tried play with threshold but it's not really working. I don't think unravel will work on this case, too much noise and variability. A better way to dealias this case would be probably to use some kind of VAD or double VAD technique using radiosounding data (I don't have code for that sorry, but look into pyart)? Regards, Valentin — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#8 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/APYEBN3RCQ7KPKJPM3QJAJLR7DSILANCNFSM4PIOICHQ> .