Pangeo upgrade + geoparquet support

[wildintellect]

Note: made changes in github so didn't run precommit.

[wildintellect]

Interesting

Importing pyogrio resulted in: libgdal.so.36: cannot open shared object file: No such file or directory
  +        Importing fiona resulted in: No module named 'fiona'

[wildintellect]

pyogrio is the default now in geopandas, looking into the load error. Maybe the test needs an upgrade.
Should we continue to also include fiona, are there still some common ogr formats pyogrio doesn't do well or other missing features? @jsignell @kylebarron @weiji14

[weiji14]

pyogrio is the default now in geopandas, looking into the load error. Maybe the test needs an upgrade. Should we continue to also include fiona, are there still some common ogr formats pyogrio doesn't do well or other missing features? @jsignell @kylebarron @weiji14

Full error message:

  =================================== FAILURES ===================================
  ____________________ notebook: nbregression(test-notebook) _____________________
  nbclient.exceptions.CellExecutionError: An error occurred while executing the following cell:
  ------------------
  URL = "https://geodata.ucdavis.edu/gadm/gadm4.1/json/gadm41_FRA_0.json"
  df = gpd.read_file(URL)
  ------------------
  
  ---------------------------------------------------------------------------
  ImportError                               Traceback (most recent call last)
  Cell In[4], line 2
        1 URL = "https://geodata.ucdavis.edu/gadm/gadm4.1/json/gadm41_FRA_0.json"
  ----> 2 df = gpd.read_file(URL)
  
  File /srv/conda/envs/notebook/lib/python3.12/site-packages/geopandas/io/file.py:278, in _read_file(filename, bbox, mask, columns, rows, engine, **kwargs)
      190 def _read_file(
      191     filename, bbox=None, mask=None, columns=None, rows=None, engine=None, **kwargs
      192 ):
      193     """
      194     Returns a GeoDataFrame from a file or URL.
      195 
     (...)
      276 
      277     """
  --> 278     engine = _check_engine(engine, "'read_file' function")
      280     filename = _expand_user(filename)
      282     from_bytes = False
  
  File /srv/conda/envs/notebook/lib/python3.12/site-packages/geopandas/io/file.py:139, in _check_engine(engine, func)
      137     _check_fiona(func)
      138 elif engine is None:
  --> 139     raise ImportError(
      140         f"The {func} requires the 'pyogrio' or 'fiona' package, "
      141         "but neither is installed or imports correctly."
      142         f"\nImporting pyogrio resulted in: {pyogrio_import_error}"
      143         f"\nImporting fiona resulted in: {fiona_import_error}"
      144     )
      146 return engine
  
  ImportError: The 'read_file' function requires the 'pyogrio' or 'fiona' package, but neither is installed or imports correctly.
  Importing pyogrio resulted in: libgdal.so.36: cannot open shared object file: No such file or directory
  Importing fiona resulted in: No module named 'fiona'
  ImportError: The 'read_file' function requires the 'pyogrio' or 'fiona' package, but neither is installed or imports correctly.
  Importing pyogrio resulted in: libgdal.so.36: cannot open shared object file: No such file or directory
  Importing fiona resulted in: No module named 'fiona'
  ----------------------------- Captured stderr call -----------------------------

Can you add a conda list somewhere in the build process to show what dependencies are being installed. I'm hoping it's just 1) some mixing of conda/pip packages that's causing this, or 2) (more complicated) a silly error caused by importing packages in the wrong order (mdsumner/gdal-builds#6), and I do see geopandas being imported after xarray in that notebook, but am hoping this isn't the reason.

[jsignell]

This change to include !mamba list in the notebook means that we will definitely have to regerate the output every time we update the image. We might want to consider moving that out of the notebook and into the github actions workflow steps.

[sunu]

@wildintellect Please feel free to merge. I'm happy to update the base images on the hubs once this is merged.

[jsignell]

This change to include !mamba list in the notebook means that we will definitely have to regerate the output every time we update the image. We might want to consider moving that out of the notebook and into the github actions workflow steps.

I just pushed this change. So the list of packages is visible in the GH actions logs.

[wildintellect]

Merging this. @sunu you can prepare the PR to update the hubs, however:

1. I need to check which hubs should be updated (new ticket).
2. I need to notify all of them, and confirm they are ready for it.