DatasetCodec details

[bogovicj]

• generic types of encoded / decoded blocks, which of these:
  • DataBlock<T> encode(DataBlock<T> blk)
  • DataBlock<S> encode(DataBlock<T> blk)
    • John prefers this because future such codec will change types

[tpietzsch]

In #190 I went with the simpler DataBlock<T> encode(DataBlock<T> blk) version.

DatasetCodec is merged into BlockCodec on creation, so in the actual reading/writing implementation we (still) only need to care about BlockCodec.

There is nothing fundamentally preventing the <S,T> scenario. What is not clear to me is how it works out with figuring types from the json metadata:
Currently, DatasetAttributes only knows about one dataType, and that becomes the T.
Assuming we have a BlockCodec<T> and prepend a DatasetCodec<S,T> and another DatasetCodec<R,S>, the concatenation of all of these becomes a BlockCodec<R> (maybe order of type arguments is different, but this is the gist of it). What is the DatasetAttributes.dataType now? The T or the R. And where do the other datatypes come from? We presumably need three dataTypes, for R, S, and T, to construct the individual codecs. How does that play out with the metadata?

[bogovicj]

Summary of what talked about in virtual meeting:

I completely agree regarding the challenges of figuring out types from the metadata, and deciding what type DatasetAttributes should return (the stored type of the data block type?). We will definitely need to figure this out.
It is also unfortunate that we don't have an example of such a codec from Zarr, so we will just have to figure it out whatever it is.

Given that such a codec is coming, if we go with DatasetCodec<T>, we will have to break API when it comes. So my preference would be to have DatasetCodec<S,T> so that whatever changes we do have to make can be hidden in the implementation, and we will not have a breaking change.

[tpietzsch]

In #190 I switched to DatasetCodec<S,T> now.
What remains is to rename DatasetCodec to something more meaningful. We will do a "big renaming" after various open branches are finished and merged. We should make a separate issue for that, so I'm closing this...