Do not allow `Identity` transformations in multiscale transformations

[jo-mueller]

Issue originating from implementation work with @dstansby on 0.6.dev4 metadata. Essentially, Identity transforms are a bit tricky to use in the multiscales transforms (i.e., under ome > multiscales > datasets > coordinateTransformations, because they don't declare their own dimensionality.

One may assume that an Identity transform is equivalent to a scale transform with scale factors of 1 along every axis, but it's not possible to validate without parsing all transforms, coordinate systems and the axes stored therein.

An easy fix will be to simply only allow the following ome > multiscales > datasets > coordinateTransformations:

• Scale
• Sequence(Scale, Translation)

cc @bogovicj @clbarnes @thewtex @d-v-b

[d-v-b]

where is the identity transform actually useful?

[clbarnes]

Identity transforms are now useful as a component of byDimensions. byDimensions could be updated to implicitly identity any remaining axes but that's more of a wrinkle.

Why specifically is not declaring its dimensionality a problem in multiscales and not anywhere else?

[jo-mueller]

where is the identity transform actually useful?

I could also see it being useful in tiled data, where one would like to store the information about a tile's position in world coordinates with the tile itself, but still organize many tiles in an 0.6 scene layout.

In that case, every thile would have two coordinate systems:

• a "physical" coordinate system
• a "world coordinate system
• the two are connected with a translate transform

in the scene metadata one would have

• a single "world" coordinate system
• a long list of identity transforms connecting the "world" coordinate systems.

Why specifically is not declaring its dimensionality a problem in multiscales and not anywhere else?

It is a problem everywhere else, that's true. In the multiscale transforms, being restricted to scale/translation at least allows to check for the ordering of the multiscales in a validation step. A validator would still not necessarily know whether the specified scale/translate values fit the number of axes in the output coordinate system, though.

[d-v-b]

i feel like any place where identity transforms would be used is also a place where you could introduce a convention like "an empty transforms list / null value means the input coordinates are unchanged". Maybe the identity transform is useful in programs that implement coordinate transformations, but in JSON metadata I don't see the value.

[d-v-b]

that being said, I don't see why the identity transform is particularly problematic for validation? if you know the input dimensionality, then you know the output dimensionality. and if you don't know the input dimensionality, then isn't that the real problem?

[jo-mueller]

and if you don't know the input dimensionality, then isn't that the real problem?

This occurs only in one situation, i.e. in the multiscale transforms where the input is not a coordinate system that specifies its own axes, but a zarr array. I mean a validator could open/inspect this zarr array and check for the dimensionality. That would just require IO which I would like to avoid in validation, where possible.

[d-v-b]

if the spec says "<metadata> MUST match the dimensionality of the array", then any complete validator has to read the array metadata documents, which means the array metadata document is in-scope for validation at some level. IIRC ome-zarr-models-py already works this way? there are internal consistency checks (don't chain a scale[dimensionality3] with translation[dimensionality4]), and then higher up in the validation tree you get external consistency checks (ensure all arrays have the same dimensionality)

[thewtex]

Perhaps we add an integer to the identity to specify its dimension.

[jo-mueller]

I guess the "problem" is what @d-v-b points out. At the point where a coordinate transformation connects to a zarr array rather than an ome-zarr CoordinateSystem instance, any complete validator will have to look into the zarr array('s metadata) to make sure dimensions match.

For all other transforms, most of them declare their own dimensionality implicitly, but a complete validator would still need to look into the referenced coordinate systems and check whether they match the declared dimensionality.

[clbarnes]

Displacement, coordinate, and path-type affine/ rotation transforms don't declare their own dimensionality, I think?

[jo-mueller]

Displacement, coordinate, and path-type affine/ rotation transforms don't declare their own dimensionality, I think?

True.