Skip to content

DataTree roundtrip fails on None group lookup #9960

Open
@sjperkins

Description

What happened?

I'm attempting to use zarr 3.0.0 in conjunction with xarray 2025.1.1 in this PR

A roundtirpping test case started to fail

What did you expect to happen?

I would've expected the test case to succeed, but AFAICT on the open_datatree call to the local zarr store, one of the zarr groups resolves to None, resulting in the error below.

Minimal Complete Verifiable Example

import xarray
import xarray.testing as xt
import numpy as np

if __name__ == "__main__":
  ds = xarray.Dataset({
    "A": (("x", "y"), np.ones((128, 256))),
    "B": (("y", "x"), np.ones((256, 128))*2)
  })

  dt = xarray.DataTree.from_dict({
    "/root/a": ds,
    "/root/b": ds,
  })

  import shutil
  path = "/tmp/test_dt.zarr"
  shutil.rmtree(path, ignore_errors=True)
  dt.to_zarr(path, compute=True, mode="w")
  dt2 = xarray.open_datatree(path)
  xt.assert_identical(dt, dt2)

MVCE confirmation

  • Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray.
  • Complete example — the example is self-contained, including all data and the text of any traceback.
  • Verifiable example — the example copy & pastes into an IPython prompt or Binder notebook, returning the result.
  • New issue — a search of GitHub Issues suggests this is not a duplicate.
  • Recent environment — the issue occurs with the latest version of xarray and its dependencies.

Relevant log output

$ python tests/test_zarr_roundtrip.py 
/home/simon/.cache/pypoetry/virtualenvs/xarray-ms-jDhc3Ane-py3.11/lib/python3.11/site-packages/zarr/api/asynchronous.py:197: UserWarning: Consolidated metadata is currently not part in the Zarr format 3 specification. It may not be supported by other zarr implementations and may change in the future.
  warnings.warn(
Traceback (most recent call last):
  File "/home/simon/code/xarray-ms/tests/test_zarr_roundtrip.py", line 20, in <module>
    dt2 = xarray.open_datatree(path)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/simon/.cache/pypoetry/virtualenvs/xarray-ms-jDhc3Ane-py3.11/lib/python3.11/site-packages/xarray/backends/api.py", line 1113, in open_datatree
    backend_tree = backend.open_datatree(
                   ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/simon/.cache/pypoetry/virtualenvs/xarray-ms-jDhc3Ane-py3.11/lib/python3.11/site-packages/xarray/backends/zarr.py", line 1614, in open_datatree
    groups_dict = self.open_groups_as_dict(
                  ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/simon/.cache/pypoetry/virtualenvs/xarray-ms-jDhc3Ane-py3.11/lib/python3.11/site-packages/xarray/backends/zarr.py", line 1665, in open_groups_as_dict
    stores = ZarrStore.open_store(
             ^^^^^^^^^^^^^^^^^^^^^
  File "/home/simon/.cache/pypoetry/virtualenvs/xarray-ms-jDhc3Ane-py3.11/lib/python3.11/site-packages/xarray/backends/zarr.py", line 662, in open_store
    return {
           ^
  File "/home/simon/.cache/pypoetry/virtualenvs/xarray-ms-jDhc3Ane-py3.11/lib/python3.11/site-packages/xarray/backends/zarr.py", line 663, in <dictcomp>
    group: cls(
           ^^^^
  File "/home/simon/.cache/pypoetry/virtualenvs/xarray-ms-jDhc3Ane-py3.11/lib/python3.11/site-packages/xarray/backends/zarr.py", line 744, in __init__
    self._read_only = self.zarr_group.read_only
                      ^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'read_only'

Anything else we need to know?

No response

Environment

INSTALLED VERSIONS

commit: None
python: 3.11.11 (main, Dec 4 2024, 08:55:08) [GCC 13.2.0]
python-bits: 64
OS: Linux
OS-release: 6.8.0-51-generic
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: None
LANG: en_ZA.UTF-8
LOCALE: ('en_ZA', 'UTF-8')
libhdf5: None
libnetcdf: None

xarray: 2025.1.1
pandas: 2.2.3
numpy: 2.1.3
scipy: None
netCDF4: None
pydap: None
h5netcdf: None
h5py: None
zarr: 3.0.0
cftime: None
nc_time_axis: None
iris: None
bottleneck: None
dask: 2024.10.0
distributed: 2024.10.0
matplotlib: None
cartopy: None
seaborn: None
numbagg: None
fsspec: 2024.10.0
cupy: None
pint: None
sparse: None
flox: None
numpy_groupies: None
setuptools: 74.1.2
pip: 24.2
conda: None
pytest: 8.3.3
mypy: None
IPython: 8.29.0
sphinx: 8.1.3

Metadata

Assignees

No one assigned

    Labels

    bugtopic-DataTreeRelated to the implementation of a DataTree classtopic-zarrRelated to zarr storage library

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions