h5coro error reading ATL06 file

[andypbarrett]

When running the https://github.com/nsidc/NSIDC-Data-Tutorials/blob/cryo-184/notebooks/ICESat-2_Cloud_Access/ATL10-h5coro.ipynb notebook for the whole "Antarctic" region, h5coro gives the following error.

H5Coro encountered error reading gt1r/freeboard_segment/latitude: invalid heap signature: 0x0
H5Coro encountered error reading gt1r/freeboard_segment/longitude: invalid heap signature: 0x0
H5Coro encountered error reading gt1r/freeboard_segment/delta_time: invalid heap signature: 0x0
H5Coro encountered error reading gt1r/freeboard_segment/seg_dist_x: invalid heap signature: 0x0
H5Coro encountered error reading gt1r/freeboard_segment/heights/height_segment_length_seg: invalid heap signature: 0x0
H5Coro encountered error reading gt1r/freeboard_segment/beam_fb_height: invalid heap signature: 0x0
H5Coro encountered error reading gt1r/freeboard_segment/heights/height_segment_type: invalid heap signature: 0x0

This causes a TypeError to be returned instead of a GeoPandas.Dataframe. The concatenation step in read_atl10 then fails.

File ~/NSIDC-Data-Tutorials/notebooks/ICESat-2_Cloud_Access/h5cloud/read_atl10.py:132, in read_atl10(files, bounding_box, executors, environment, credentials)
    129     return df
    131 dfs = pqdm(files, read_h5coro, n_jobs=executors)
--> 132 combined = pd.concat(dfs)
    134 return combined

I think we need a try, except block so that a None or some other value is returned.

We also need to then filter out the Nones so that pd.concat works.

[andypbarrett]

I am working on solving this issue

[andypbarrett]

The problem files is this one

Could not read gt1r/freeboard_segment/latitude from nsidc-cumulus-prod-protected/ATLAS/ATL10/006/2019/09/27/ATL10-02_20190927144209_00110501_006_02.h5
'NoneType' object is not subscriptable

I'm implement a try except block but am running into the following error with the returned list

File ~/NSIDC-Data-Tutorials/notebooks/ICESat-2_Cloud_Access/h5cloud/read_atl10.py:144, in <listcomp>(.0)
    142 dfs = pqdm(files, read_h5coro, n_jobs=executors)
    143 print(set([type(df) for df in dfs]))
--> 144 combined = pd.concat([df for df in dfs if isinstance(df (gpd.GeoDataFrame, gpd.GeoSeries))])
    146 return combined

TypeError: 'GeoDataFrame' object is not callable

[andypbarrett]

Now that I have finally tracked down the offending granule, I can work on a tighter feedback loop in a test case.

[asteiker]

Testing Jira integration: CRYO-188