Implement scale/offset codec in the EOPF data model

[emmanuelmathot]

Context

The scale/offset codec extension has been merged in the zarr-extensions repository, providing a standardized Zarr V3 array-to-array codec for scale/offset transformation. This closes the specification gap documented in zarr-developers/zarr-extensions#42.

The next step is implementation in zarr-python, which would make the codec transparently available through xarray and our downstream stack (data pipeline, TiTiler).

What this enables

With a standardized and implemented scale/offset codec, our GeoZarr data arrays can:

• Store reflectance data efficiently as uint16 (0–10000) while transparently presenting float32 values (0.0–1.0) to users
• Follow CF conventions for physical quantities without requiring users to manually apply scale_factor / add_offset attributes
• Decouple storage encoding from data presentation — the Zarr V3 design intent

This directly addresses the reflectance data convention discussion with EOPF CPM, where both sides agree on integer storage with codec-based float presentation.

Tasks

• Confirm zarr-python implementation status — @d-v-b: is the scale/offset codec from zarr-extensions#43 being implemented in zarr-python? If so, this is an activity we can do in this project.
• Update the data pipeline to use the codec in the converter and the data pipeline @lhoupert
• Validate downstream compatibility and confirm that TiTiler correctly decode data through the codec without additional handling @vincentsarago
• Document the convention in the data model spec @emmanuelmathot

Related issues

• FixedScaleOffset codec not preserved during zarr v2 to v3 conversion #106
• consistently export data as float64 #108
• Request for Standardized FixedScaleOffset Codec Extension zarr-developers/zarr-extensions#42
• scale-offset codec zarr-developers/zarr-extensions#43
• https://gitlab.cpm.eopf.copernicus.eu/eopf/eopf-cpm/-/issues/985 — Data presentation convention for reflectance values
• Support for fixedscaleoffset manzt/numcodecs.js#49
• Implement fixedscaleoffset codec manzt/zarrita.js#312

[emmanuelmathot]

@d-v-b @lhoupert @vincentsarago, please create the relevant sub-tasks or related tasks in the appropriate repository.

[d-v-b]

• Confirm zarr-python implementation status — @d-v-b: is the scale/offset codec from zarr-extensions#43 being implemented in zarr-python? If so, this is an activity we can do in this project.

yes, I have 2 working implementations. One is pure python + numpy, in this PR against zarr-python: zarr-developers/zarr-python#3774. If this is hard to merge, we can factor it out into a separate package.

Because I was concerned about the performance of python + numpy and because I anticipated the need to support a javascript implementation, I commissioned a rust-based implementation: https://github.com/d-v-b/zarr-cast-value. This is ~4 - 7x faster, according to some rough benchmarks.

for context, the cast_value operation is performance-sensitive because it requires iterating over each array scalar and potentially replacing that scalar with an arbitrary element from the destination data type. This was necessary to generically express numpy casting behavior.

My recommendation: if we don't have fast reviewer activity on that PR, or people think this functionality is too complicated to exist in zarr-python, then we split this off into a separate python package. And in that package, we can depend on rust bindings if we feel like the performance is worth it.