Skip to content

Displacements are multiscales #145

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
jo-mueller merged 14 commits into from
Jun 25, 2026
Merged

Displacements are multiscales #145

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
4a03a56
spec: displacements and coordinates are just multiscales
jo-mueller Jun 12, 2026
fe9ff56
tests: add config to displacements example
jo-mueller Jun 12, 2026
8dc51c1
chore: typo
jo-mueller Jun 12, 2026
e177f42
spec: Tie together "channel" infor for coords/displacements
jo-mueller Jun 16, 2026
f0a0527
spec: note on channel/custom in input space
jo-mueller Jun 16, 2026
12994d6
Merge branch 'main' into displacements-are-multiscales
jo-mueller Jun 22, 2026
7c5875b
Merge branch 'main' into displacements-are-multiscales
jo-mueller Jun 22, 2026
fc5d8ae
spec: minor fixes in examples
jo-mueller Jun 22, 2026
1eb1304
spec: remove duplicated text
jo-mueller Jun 22, 2026
e0ea729
spec: improve example around displacements
jo-mueller Jun 22, 2026
12497b7
spec: improve indexing example
jo-mueller Jun 22, 2026
9b81015
spec: declare version as 0.6
jo-mueller Jun 24, 2026
c61fdf4
Merge branch 'main' into displacements-are-multiscales
jo-mueller Jun 24, 2026
6878a79
Merge branch 'main' into displacements-are-multiscales
jo-mueller Jun 24, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 0 additions & 14 deletions examples/transformations/coordinates1d.json
View file
Open in desktop

This file was deleted.

14 changes: 0 additions & 14 deletions examples/transformations/displacement1d.json
View file
Open in desktop

This file was deleted.

3 changes: 3 additions & 0 deletions examples/transformations/displacements/.config.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{
"schema": "schemas/image.schema"
}
32 changes: 32 additions & 0 deletions examples/transformations/displacements/displacement_field.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
{
"ome": {
"version": "0.6",
"name": "the_dfield",
"multiscales": [
{
"coordinateSystems": [
{
"name": "physical",
"axes": [
{"name": "c", "type": "displacement", "discrete": true},
{"name":"y", "type": "space", "unit": "micrometer"},
{"name":"x", "type": "space", "unit": "micrometer"}
]
}
],
"datasets": [
{
"path": "s0",
"coordinateTransformations": [
{
"type": "scale",
"input" : {"path": "s0"},
"output" : {"name": "physical"},
}
]
}
]
}
]
}
}
45 changes: 45 additions & 0 deletions examples/transformations/displacements/multiscales.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
{
"ome": "0.6",
"name": "some_image",
"multiscales": [
{
"coordinateSystems": [
{
"name": "physical",
"axes": [
{"name": "y", "type": "space", "unit": "micrometer"},
{"name":"x", "type": "space", "unit": "micrometer"}
]
},
{
"name": "output",
"axes": [
{"name": "y", "type": "space", "unit": "micrometer"},
{"name":"x", "type": "space", "unit": "micrometer"}
]
}
],
"datasets": [
{
"path": "s0",
"coordinateTransformations": [
{
"type": "scale",
"input": {"path": "s0"},
"output": {"name": "physical"},
"scale": [2.0, 2.0]
}
]
}
],
"coordinateTransformations" : [
{
"type": "displacements",
"input" : {"name": "physical"},
"output" : {"name": "output"},
"path" : "coordinateTransformations/displacementField"
}
]
}
]
}
197 changes: 20 additions & 177 deletions index.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1037,7 +1037,7 @@ and is invertible.
##### coordinates and displacements
(coordinates-displacements-md)=

`coordinates` and `displacements` transformations store a vector field of arbitrary sampling density in an array,
`coordinates` and `displacements` transformations store a vector field of arbitrary sampling density in an ome-zarr [multiscale group](#multiscale-md),
defining a mapping from an input coordinate system to an output coordinate system.
The array contains either coordinates (absolute positions)
or displacements (relative shifts) for each point in the input space.
Expand All @@ -1060,7 +1060,7 @@ The array containing the coordinates or displacements MUST:
Metadata for these coordinate transforms have the following fields:

**path**
: The location of the coordinate array in this (or another) container.
: The location of the coordinate multiscale group in this (or another) container.

**interpolation**
: The interpolation attributes MAY be provided.
Expand Down Expand Up @@ -1088,36 +1088,17 @@ implementations of the specified interpolation methods may still differ in their
An exact reproducibility of pixel values for images transformed and resampled by this transformation is therefore out of the scope of this specification.
```

**Array metadata**

For both `coordinates` and `displacements`,
the array data referred to by `path` MUST define the following metadata fields:

* `coordinateSystems`: MUST contain a [coordinate system](#coordinatesystems-metadata) with the following properties:
- Include all axes of the input coordinate system (in the same order).
- Include one additional axis of `"type": "coordinate"` (for coordinates transformations) or `"type": "displacement"` (for displacements transformations).
- The additional axis should be the last axis (for contiguous memory layout in C-order).
- The `name` of this coordinate system SHOULD be the same as the `name` of the corresponding coordinate transformation.

* `coordinateTransformations`: Defines how to map from the coordinate system of the array into a physical coordinate system
(e.g. the resolution at which the vector field is sampled). MUST contain a single transformation with the following properties:
- `type`: The type of the transformation; MUST be one of [`identity`](#identity-md), [`scale`](#scale-md)
or a [`sequence`](#sequence-md) of a [scale](#scale-md) followed by a [translation](#translation-md).
- `output`: The name of the coordinate system defined in the `coordinateSystems` field of the array metadata.

*Note*: The `input` field is omitted, as it is implicitly the pixel coordinate system of the array
(defined by the first `N` axes of the array's `coordinateSystem`).

**Constraints**

The array at `path` MUST satisfy:

- **Dimensionality**: If the input coordinate system has `N` axes, the array at location `path` MUST have `N+1` dimensions.
The multiscale group at `path` MUST satisfy:
- **Dimensionality**: If the input coordinate system has `N` axes, the multiscale image at location `path` MUST have `N+1` dimensions.
- **Vector dimension length**:
- For `coordinates` transformations, the length of the array along the `coordinate` dimension (last axis) MUST equal `M`,
the number of axes in the output coordinate system.
The coordinate dimension SHOULD be stored as the channel axis (`"name": "c"`) of the multiscale image group and have `"type": "coordinate"`.
- For `displacements` transformations, the length of the array along the `displacement` dimension (last axis) MUST equal `N`,
the number of axes in the input (and output) coordinate system. `displacements` require `M=N`.
The displacement dimension SHOULD be stored as the channel axis (`"name": "c"`) of the multiscale image group and have `"type": "displacement"`.
Comment thread
jo-mueller marked this conversation as resolved.
Outdated
- **Vector component mapping**: The `i`th value of the array along the `coordinate` or `displacement` axis refers to the `i`th output axis.
Comment thread
jo-mueller marked this conversation as resolved.

```{hint}
Expand All @@ -1132,168 +1113,30 @@ Applying the transformation to a point `x` in the input coordinate system amount
- a displacement to add to the input point `x` (`displacements`).
```
Comment thread
jo-mueller marked this conversation as resolved.

:::{dropdown} Example 1: 1D coordinate transformation
For example, in 1D, a coordinate field transformation mapping from an input coordinate system `input`
to an output coordinate system `output` would have metadata such as:
```json
{
"name" : "a coordinate field transform",
"type": "coordinates",
"path" : "i2xCoordinates",
"input" : {"name": "i"},
"output" : {"name": "x"},
"interpolation" : "nearest"
}
```
:::{dropdown} Example: 2D displacement transformation

where we assume input coordinate systems `input` and `output` are defined elsewhere.
Example metadata under the attributes of the zarr array at path `coordinateTransformations/i2xCoordinates` above:
In this example, a `displacements` transformation is defined between two 2D coordinate systems, `physical` and `output`.
The transformation references a displacement vector field located at `coordinateTransformations/displacementField`.

```json
{
"ome": {
"coordinateSystems" : [
{
"name" : "a coordinate field transform",
"axes" : [
{ "name": "i", "type": "space", "discrete": true },
{ "name": "c", "type": "coordinate", "discrete": true }
]
}
],
"coordinateTransformations" : [
{
"type" : "identity",
"output" : {"name": "a coordinate field transform"}
}
]
}
}
```

Here, the axis `i` refers to the input positions along the `i`-axis,
which equal array indices in this case as indicated by the `identity` transformation.
The `c` axis holds the corresponding output coordinates.

If the array in `coordinates` contains the data:
```
[
[-9], // Output coordinate for index=0, i=0
[9], // Output coordinate for index=1, i=1
[0] // Output coordinate for index=2, i=2
]
```

then this metadata defines the following function to map
the `i` axis (`input` coordinate system) to the `x` axis (`output` coordinate system):

```
x =
if ( i < 0.5 ) -9
else if ( i >= 0.5 and i < 1.5 ) 9
else if ( i >= 1.5 ) 0
```
:::

:::{dropdown} Example 2: 1D displacement transformation
A 1D example displacement field:
```json
{
"name" : "a displacement field transform",
"type": "displacements",
"path" : "displacements",
"input" : {"name": "i"},
"output" : {"name": "x"},
"interpolation" : "linear"
}
```

where we assume input coordinate systems `input` and `output` are defined elsewhere.
Example metadata under the attributes of the zarr array at path `displacements` above:

```json
{
"ome": {
"coordinateSystems" : [
{
"name" : "a displacement field transform",
"axes" : [
{ "name": "x", "type": "space", "unit" : "nanometer" },
{ "name": "d", "type": "displacement", "discrete": true }
]
}
],
"coordinateTransformations" : [
{
"type" : "scale",
"scale" : [2, 1],
"output" : {"name": "a displacement field transform"}
}
]
}
}
```

Since the input and output coordinate system may be defined in physical units,
a scale transformation is needed to map the displacement vectors into the same physical units as the input point.

If the array in `displacements` contains the data:

```
[
[-1], // Displacement for index=0, x=0
[0], // Displacement for index=1, x=2
[1] // Displacement for index=2, x=4
]
```

this transformation maps the point `[1.0]` to the point `[0.5]`.
A scale transformation maps the array coordinates to the `x` axis.
Using the inverse of the scale transform, we see that we need the position `0.5` in array coordinates.
The transformation specifies linear interpolation,
which in this case yields `(0.5 * -1) + (0.5 * 0) = -0.5`.
That value gives us the displacement of the input point,
hence the output is `1.0 + (-0.5) = 0.5`.
:::

:::{dropdown} Example 3: 2D displacement transformation

In this example, the array located at `displacementField` MUST have three dimensions.
One dimension MUST correspond to an axis with `type : displacement` (in this example, the last dimension),
the other two dimensions MUST be axes that are identical to the axes of the `in` coordinate system.

```json
"coordinateSystems" : [
{ "name" : "in", "axes" : [{"name" : "y"}, {"name":"x"}] },
{ "name" : "out", "axes" : [{"name" : "y"}, {"name":"x"}] }
],
"coordinateTransformations" : [
{
"type": "displacements",
"input" : {"name": "in"},
"output" : {"name": "out"},
"path" : "displacementField"
}
]
```{literalinclude} examples/transformations/displacements/multiscales.json
:language: json
```

The metadata at location `displacementField` should have a coordinate system such as:
This multiscales (containing the vector field) MUST have three dimensions,
i.e., `x` and `y` as mandated by the input coordinate system and an additional channel dimension `d` to hold the x- and y-displacements.
The metadata for the multiscale group at location `coordinateTransformations/displacementField` would look as follows;
Comment thread
jo-mueller marked this conversation as resolved.
Outdated
The channel dimension corresponds to an axis with `type : displacement`,
the other two dimensions MUST be axes that are identical to the axes of the input coordinate system (`"name": "physical"`) to which the transformation is applied.

```json
"coordinateSystems" : [
{ "name" : "in", "axes" : [
{"name":"y"},
{"name":"x"},
{"name":"d", "type":"displacement", "discrete":true} ]
}
]
```{literalinclude} examples/transformations/displacements/displacement_field.json
:language: json
```

Indexing into this array using c-order, for spatial positions `y` and `x`, the y- and x-displacements would be given by:

```
y_displacement = displacementField[y][x][0]
x_displacement = displacementField[y][x][1]
y_displacement = displacementField[0][y][x]
x_displacement = displacementField[1][y][x]
```

I.e. the y-displacement is first, because the y-axis is the first element of the input and output coordinate systems.
Comment thread
jo-mueller marked this conversation as resolved.
Expand Down
6 changes: 6 additions & 0 deletions version_history.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,12 @@ All notable changes to this project will be documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).

## {0.6rc0} - 2026-06-12
Comment thread
jo-mueller marked this conversation as resolved.
Outdated

### Changed
- Displacements and coordinate transformations are now required
to store the vector field as a normal multiscale group with the same metadata as other multiscales. Decided at EMBL Hackathon 06/2026 by RFC5 developers.

## [0.6.dev4] - 2026-04-22

### Changed
Expand Down
Loading