Skip to content

stable versions#20

Closed
abarciauskas-bgse wants to merge 2 commits intomainfrom
ab/stable-versions
Closed

stable versions#20
abarciauskas-bgse wants to merge 2 commits intomainfrom
ab/stable-versions

Conversation

@abarciauskas-bgse
Copy link
Contributor

To support replicability of demonstration notebooks (specifically for virtualizarr+icechunk work), I propose to create a version of this image with tagged versions of the core dependencies, currently installed from main branches. This way we can share a tagged version of the image which can be referenced in documentation to reproduce workflows.

@abarciauskas-bgse
Copy link
Contributor Author

@sharkinsspatial here I was trying create an image that could read and write virtual stores using icechunk and virtualizarr with currently released versions (VirtualiZarr 1.3.2) but ran into the following issue I'm sure you are aware of - basically that the _FillValue is not encoded in a way that xarray accepts (as a string or bytes) given the float64 dtype of the array, so xarray is throwing an error on this line assert isinstance(value, str | bytes). value is -999 in my example for a float64 dtype.

I understand that we are working towards a Virtualizarr 2.0 release but are you aware of any way around this? Or perhaps I could use an example that has a compatible FillValue?

Here is the notebook I am testing with: https://gist.github.com/abarciauskas-bgse/9dff4a2a8751e792cda95bbdd3cec489

@sharkinsspatial
Copy link
Member

@abarciauskas-bgse This _FillValue Zarr v3 encoding issue should be fixed for most dtypes (there are still several unsupported value->target dtype combinations in the current zarr-python) by zarr-developers/VirtualiZarr@d6b404d. Your notebook works for me against the current develop branch with the small change of removing indexes={} option from the open_virtual_dataset since it is no longer supported.

@abarciauskas-bgse
Copy link
Contributor Author

@sharkinsspatial ok thank you! I'm going to close this PR for now in favor of just using specific library versions for the MUR SST icechunk reading and writing, until we have a new VirtualiZarr release.

icechunk==0.2.15
virtualizarr==1.3.2
xarray==2025.4.0
zarr==3.0.8

seems to work for both reading and writing for MUR SST.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants