Skip to content

read_lazy kwarg for not using dask #2147

@ilan-gold

Description

@ilan-gold

Please describe your wishes and possible alternatives to achieve the desired result.

I think that every datastructure that can be opened as dask can also be opened as not-dask like sparse has https://anndata.readthedocs.io/en/stable/generated/anndata.io.sparse_dataset.html and even categoricals within xarray have https://anndata.readthedocs.io/en/stable/generated/anndata.experimental.backed.CategoricalArray.html.

Sometimes, there is a use-case where the lazy-compute feature of dask is not necessary, and thus lazy-loading can be done without the overhead of dask.

I would propose updating read_lazy to have use_dask kwarg that uses dask in all instances where possible. The main change here to default behavior (i.e., True) would be that categoricals and nullables within Dataset2D would be dask arrays by default.

Other than that I would expect no change to defaults since even things like dense arrays are opened with dask instead of the zarr.Array or h5py.Dataset

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions