Remove zarr plugin and decouple core from zarr#333
Open
mpiannucci wants to merge 2 commits into
Open
Conversation
The zarr plugin has been ported to its own project (xpublish-community/xpublish-zarr#1). This drops the built-in ZarrPlugin, the xpublish.utils.zarr module, and the zarr/numcodecs dependencies from xpublish core. - Delete xpublish/plugins/included/zarr.py and xpublish/utils/zarr.py - Rewrite dataset_info's /info endpoint to read dims/dtype/attrs directly from the xarray Dataset (with a small helper to make numpy scalars JSON-serializable), removing the zmetadata round-trip - Drop zarr/numcodecs from module_version's reported modules - Remove zarr, numcodecs, and dask dependencies, the zarr entry point, and the zarr keyword from pyproject.toml - Update README, tutorial, and included-plugins docs to point at xpublish-zarr instead of describing built-in zarr endpoints - Remove zarr-only tests and prune zarr cases from the remaining suite - Drop the zarr-focused example notebook and zarr/numcodecs from the binder environment Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- Drop dask from the upstream dev-build install list since it's no longer a dependency; installing it with --no-deps left cloudpickle missing in dev builds, which broke xarray decoding - Reorder xpublish/plugins/hooks.py so Plugin is defined before Dependencies, eliminating the 'Plugin' forward reference in Dependencies.plugins that tripped autodoc-pydantic during the docs build - Disable autodoc_pydantic_model_show_json globally; the Dependencies model is all Callable fields and pydantic can't generate a JSON schema for it (the coerce strategy didn't fully cover this case) Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
abkfenris
approved these changes
May 22, 2026
Member
abkfenris
left a comment
abkfenris
left a comment
There was a problem hiding this comment.
I think the main thing is making it more clear that Zarr was ripped out and now needs to also be installed for previous functionality.
Comment on lines
-59
to
-70
| The Zarr plugin provides consolidated Zarr v2 access to the loaded datasets. | ||
|
|
||
| ```{eval-rst} | ||
| .. autosummary:: | ||
| :toctree: generated/ | ||
|
|
||
| plugins.included.zarr.ZarrPlugin | ||
|
|
||
| .. openapi:: ./openapi.json | ||
| :include: | ||
| /datasets/{dataset_id}/zarr/* | ||
| ``` |
Member
There was a problem hiding this comment.
Lets put a notice here that Zarr has migrated to it's own library as well.
Comment on lines
-55
to
+68
| from ...utils.zarr import attrs_key, get_zmetadata, get_zvariables # type: ignore | ||
|
|
||
| zvariables = get_zvariables(dataset, cache) | ||
| zmetadata = get_zmetadata(dataset, cache, zvariables) | ||
|
|
||
| info = {} | ||
| info: dict = {} |
Member
There was a problem hiding this comment.
Is there a slowdown from the lack of caching here? Could we cache info?
| return d | ||
|
|
||
|
|
||
| class Dependencies(BaseModel): |
Comment on lines
+74
to
+75
| Additional data access endpoints (such as Zarr-compatible access via | ||
| [xpublish-zarr]) can be added by installing or writing plugins. |
Member
There was a problem hiding this comment.
This may be worth an admonition to be more explicit that Zarr was ripped out into a standalone library for the next version and now has to be independently installed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The zarr plugin has been ported to its own project (xpublish-community/xpublish-zarr#1). This PR removes the built-in
ZarrPlugin, thexpublish.utils.zarrmodule, and the zarr / numcodecs / dask dependencies from xpublish core.What changed
Removed
xpublish/plugins/included/zarr.py(the plugin) andxpublish/utils/zarr.py(the zmetadata/chunk helpers)zarrentry point,zarr/numcodecs/daskdependencies, andzarrkeyword frompyproject.tomlzarrandnumcodecsfrom the modules reported byModuleVersionPlugintests/test_zarr_compat.py,tests/test_fsspec_compat.py, and the now-unusedtests/utils.py(TestStore/create_datasetwere only consumed by those files)tests/test_rest_api.py,tests/test_core.py, andtests/test_plugin_management.pyexamples/open_dataset.ipynbnotebook and zarr/numcodecs from.binder/environment.yml/.binder/test.pyDecoupled
/infoThe dataset_info plugin's
/infoendpoint previously ran the dataset through the zarr consolidated-metadata pipeline to read attrs/dims/dtypes. It now reads them directly from thexarray.Dataset, with a small_jsonablehelper that converts numpy scalars / arrays into JSON-serializable types. The response shape is unchanged for the test fixtures.Docs
docs/source/api/included_plugins.mdno longer describe built-in zarr endpoints; they point at xpublish-zarr instead.🤖 Generated with Claude Code