-
Notifications
You must be signed in to change notification settings - Fork 175
Description
Please describe your wishes and possible alternatives to achieve the desired result.
I think that every datastructure that can be opened as dask can also be opened as not-dask like sparse has https://anndata.readthedocs.io/en/stable/generated/anndata.io.sparse_dataset.html and even categoricals within xarray have https://anndata.readthedocs.io/en/stable/generated/anndata.experimental.backed.CategoricalArray.html.
Sometimes, there is a use-case where the lazy-compute feature of dask is not necessary, and thus lazy-loading can be done without the overhead of dask.
I would propose updating read_lazy
to have use_dask
kwarg that uses dask in all instances where possible. The main change here to default behavior (i.e., True
) would be that categoricals and nullables within Dataset2D would be dask arrays by default.
Other than that I would expect no change to defaults since even things like dense arrays are opened with dask instead of the zarr.Array
or h5py.Dataset