Merge branch 'main' into translations-in-multiscales

Author: jo-mueller

index.md

@@ -280,74 +280,33 @@ Here, we refer to any method that obtains values at real-valued coordinates usin
 As such, label images may be interpolated using "nearest neighbor" to obtain labels at points along the continuum.
 ```
 
-#### Array coordinate systems
-
-The dimensions of an array do not have an interpretation
-until they are associated with a coordinate system via a coordinate transformation.
-Nevertheless, it can be useful to refer to the "raw" coordinates of the array.
-Some applications might prefer to define points or regions-of-interest in "pixel coordinates" rather than "physical coordinates," for example.
-Indicating that choice explicitly will be important for interoperability.
-This is possible by using **array coordinate systems**.
-
-Every array has a default coordinate system whose parameters need not be explicitly defined.
-The dimensionality of each array coordinate system equals the dimensionality of its corresponding Zarr array.
-Its name is the path to the array in the container,
-its axes have `"type": "array"`, are unitless, and have default names.
-The i-th axis has `"name": "dim_i"` (these are the same default names used by [xarray](https://docs.xarray.dev/en/stable/user-guide/terminology.html)).
-As with all coordinate systems, the dimension names must be unique and non-null.
-
-:::{dropdown} Example
-```json
-{
-  "arrayCoordinateSystem" : {
-    "name" : "myDataArray",
-    "axes" : [
-      {"name": "dim_0", "type": "array"},
-      {"name": "dim_1", "type": "array"},
-      {"name": "dim_2", "type": "array"}
-    ]
-  }
-}
-
-```
-
-For example, if 0/zarr.json contains:
-```json
-{
-    "zarr_format": 3,
-    "node_type": "array",
-    "shape": [4, 3, 5],
-    //...
-}
-```
-
-Then `dim_0` has length 4, `dim_1` has length 3, and `dim_2` has length 5.
-
-:::
-
-The axes and their order align with the shape of the corresponding Zarr array,
-and whose data depends on the byte order used to store chunks.
-As described in the [Zarr array metadata](https://zarr.readthedocs.io/en/stable/spec/v3.html#arrays),
-the last dimension of an array in "C" order are stored contiguously on disk or in-memory when directly loaded.
-
-The name and axes names MAY be customized by including a `arrayCoordinateSystem` field
-in the user-defined attributes of the array whose value is a coordinate system object.
-The length of `axes` MUST be equal to the dimensionality.
-The value of `type` for each object in the axes array MUST equal `"array"`.
-
 #### Coordinate convention
 
 **The pixel/voxel center is the origin of the continuous coordinate system.**
 
-It is vital to consistently define relationship
-between the discrete/array and continuous/interpolated coordinate systems.
-A pixel/voxel is the continuous region (rectangle) that corresponds to a single sample in the discrete array, i.e.,
-the area corresponding to nearest-neighbor (NN) interpolation of that sample.
-The center of a 2d pixel corresponding to the origin `(0,0)` in the discrete array
-is the origin of the continuous coordinate system `(0.0, 0.0)` (when the transformation is the identity).
-The continuous rectangle of the pixel is given
-by the half-open interval `[-0.5, 0.5) x [-0.5, 0.5)` (i.e., -0.5 is included, +0.5 is excluded).
-See chapter 4 and figure 4.1 of the ITK Software Guide.
+It is vital to consistently define relationship between the discrete/array and continuous/interpolated coordinate systems.
+The following conventions apply in this specification:
+
+- The discrete coordinate grid for a Zarr array of shape `[N₀, N₁, ..., Nₖ]`
+  is defined as zero-based, with indices ranging from 0 to Nᵢ - 1 for each dimension i.
+  For example, given an array with shape (2, 3),
+  the discrete coordinate system for that array defines the following array of points:
+  ```
+  [
+    [(0, 0), (0, 1)],
+    [(1, 0), (1, 1)],
+    [(2, 0), (3, 1)],
+  ]
+  ```
+- A "pixel"/"voxel" is the continuous region (rectangle/box) that corresponds to a single sample in the discrete array,
+  i.e., the area corresponding to nearest-neighbor (NN) interpolation of that sample.
+- The center of a 2d pixel corresponding to the origin (0,0) in the discrete array
+  is the origin of the continuous coordinate system (0.0, 0.0) (when the transformation is the identity).
+- The continuous rectangle of the pixel is given
+  by the half-open interval [-0.5, 0.5) x [-0.5, 0.5) (i.e., -0.5 is included, +0.5 is excluded).
+
+For a more formal and in-depth definition,
+see chapter 4 and figure 4.1 of the [ITK Software Guide](https://itk.org/ItkSoftwareGuide.pdf).
 
 ### bioformats2raw.layout
 
@@ -603,6 +562,113 @@ to do so by estimating the transformations' inverse if they choose to.
 ```
 :::
 
+**Transformations in array coordinate units**:
+Some applications might prefer to define points, regions-of-interest or transformation parameters
+in array coordinates (also referred to as pixel coordinates) rather than physical units.
+Because transformations are agnostic to whether they operate on array or physical coordinates,
+indicating that choice explicitly will be important for interoperability.
+This can be expressed in the metadata in multiple ways, including:
+- One can embed a transformation defined in array units into a `sequence` transformation
+  that includes the appropriate scale transformation and its inverse to convert to physical units (see example below).
+- One can define a unitless coordinate system and connect it to the "intrinsic" coordinate system
+  with a scale transformation that has the appropriate scale factors to convert to physical units.
+
+:::{dropdown} Example: Embedded expression
+
+In the context of [`scene`](#scene-md), one may want to express a transformation between two images in dimensionless units,
+even though the coordinate systems of the two images are in physical units.
+This can be achieved by embedding the transformation into a `sequence` transformation like this:
+
+```json
+{ "scene": 
+  {
+    "type": "sequence",
+    "input": {"name": "intrinsic", "path": "imageA"},
+    "output": {"name": "intrinsic", "path": "imageB"},
+    "transformations": [
+      {
+        "type": "scale",
+        "scale": [2, 2],
+      },
+      {
+        "type": "translation",
+        "translation": [10, 20],
+        "name": "translation in dimensionless units"
+      },
+      {
+        "type": "scale",
+        "scale": [0.5, 0.5],
+      }
+    ]
+  }
+}
+```
+
+This example assumes that the coordinate system named `"intrinsic"` in both referenced images is in physical units,
+and is linked to the lowest resolution level (e.g., `s0`) of the multiscale image with a `scale` transformation that has the scale factors `[0.5, 0.5]`.
+In this case, the the first `scale` transformation in this example converts the input coordinates from physical to dimensionless units.
+The `translation` transformation is applied in dimensionless units,
+and finally the second `scale` transformation converts the coordinates back to physical units.
+:::
+
+:::{dropdown} Example: Unitless coordinate system
+
+Alternatively, users may choose to define a unitless coordinate system and connect it to the "intrinsic" coordinate system
+with a scale transformation that has the appropriate scale factors to convert to physical units.
+In the context of multiscales metadata, this could look like this:
+
+```json
+{
+  "multiscales": [
+    {
+      "coordinateSystems": [
+        {
+          "name": "intrinsic",
+          "axes": [
+            {"name": "y", "type": "space", "unit": "micrometer"},
+            {"name": "x", "type": "space", "unit": "micrometer"}
+          ]
+        },
+        {
+          "name": "array",
+          "axes": [
+            {"name": "y", "type": "space"},
+            {"name": "x", "type": "space"}
+          ]
+        }
+      ],
+      "datasets": [
+        {
+          "path": "s0",
+          "coordinateTransformations": [
+            {
+              "type": "scale",
+              "scale": [0.5, 0.5],
+              "input": "s0",
+              "output": "intrinsic"
+            }
+          ]
+        }
+      ],
+      "coordinateTransformations": [ 
+        {
+          "type": "scale",
+          "scale": [2.0, 2.0],
+          "input": {"name": "intrinsic"},
+          "output": {"name": "array"}
+        }
+      ]
+    }
+  ]
+}
+```
+In this case, the `scale` transformation under `coordinateTransformations`
+defines the mapping from the "intrinsic" coordinate system to the unitless "array" coordinate system.
+Another transformation (e.g. in a `scene`) could then use the "array" coordinate system as an input or output to define transformations in array units.
+:::
+
+
+
 #### Matrix transformations
 (matrix-trafo-md)=