WIP: Update rfc5 schemas by jo-mueller · Pull Request #1 · jo-mueller/ngff-spec

jo-mueller · 2025-10-24T11:24:21Z

cc @will-moore @dstansby if relevant for you.

Copilot description

This pull request introduces significant updates to the OME-NGFF (Next-Generation File Formats) metadata specification, schemas, and example files, primarily advancing the version to 0.6dev2. The changes focus on enhancing the flexibility and clarity of coordinate systems and transformations, updating schema references, and improving validation. Additionally, a new GitHub Actions workflow is added for automated validation.

Schema and Metadata Version Updates

Updated all schema $id references and metadata version strings from previous development versions (e.g., 0.6.dev1) to 0.6dev2 for consistency across the codebase. [1] [2] [3] [4] [5] [6]

Coordinate Systems and Transformations Enhancements

Refactored coordinate system and transformation schemas to:
- Require unique, non-empty names for coordinate systems and axes.
- Add descriptions and stricter requirements for axes (e.g., min/max items, uniqueness, minLength). [1] [2]
- Replace input/output with input_axes/output_axes for improved clarity and flexibility in transformation definitions.
- Update transformation types (e.g., mapAxis now uses an array of integers for axis order, and additional descriptions for transformation fields).
- Add new transformation types (displacements, coordinates) supporting interpolation and zarr array paths.
- Remove the obsolete path_w_url definition in favor of direct path specifications.

Example Metadata Updates

Updated example JSON files to reflect the new schema:
- Changed coordinate system and transformation structures to match new requirements, including the use of named coordinate systems, axes as objects, and explicit transformation inputs/outputs. [1] [2] [3] [4] [5]
- Updated version fields in examples to 0.6dev2. [1] [2]

Documentation and Validation Improvements

Added descriptions to schema properties for better documentation and developer guidance. [1] [2] [3]
Introduced a GitHub Actions workflow (.github/workflows/validation.yml) for automated validation using Python 3.12 and tox.

Backward Compatibility and Clean-up

Removed legacy or redundant schema definitions and updated required fields to enforce stricter validation and future-proofing. [1] [2] [3]

These changes collectively modernize the NGFF metadata specification, improve schema rigor, and set the foundation for more robust and interoperable OME-Zarr datasets.

The displacements object now includes a required 'path' property for specifying the zarr array location and an 'interpolation' property with supported methods. This enhances the schema's ability to describe displacement fields and their application.

Introduces 'path' and 'interpolation' properties to the 'coordinates' object in the schema, specifying the location of the coordinate field and the interpolation method to use. The 'path' property is now required.

github-actions · 2025-10-24T11:24:36Z

Automated Review URLs

Readthedocs

will-moore · 2025-10-24T11:56:13Z

Ah, with the copilot description and diff, I initially missed it was byDimensionTransformation attributes of input/outputrenamed to input_axes/output_axes, rather than for coordinateTransformation itself.

This reverts commit ff3fedd.

Set a maximum of 5 items for the axes array in the schema to enforce limits on the number of axes allowed.

Refactors the definition of coordinateTransformations and coordinateSystems to use inline array schemas with stricter constraints. Adds a new multiscale_coordinateTransformations definition to support scale and translate transformations for multiscale datasets.

Updated example and test JSON files to use the NGFF 0.6dev2 specification. Replaced 'axes' with 'coordinateSystems', added explicit 'input' and 'output' fields to coordinateTransformations, and restructured transformation chains for clarity and compliance with the new spec.

jo-mueller · 2025-11-03T09:44:40Z

@dstansby, could I ask you to give this one a quick look? My first complete-ish take on getting the schemas and examples up to speed with the latest version of rfc5

dstansby · 2025-11-03T16:21:50Z

sorry, don't think I'll have the time to look into this in detail. What I will do at some point is try and validate metadata produced by ome-zarr-models against these as a way of checking them (and checking ome-zarr-models!)

will-moore · 2025-11-03T16:31:21Z

The schemas on this branch are currently being used by ome-ngff-validator at ome/ome-ngff-validator#48

E.g. https://deploy-preview-48--ome-ngff-validator.netlify.app/?source=https://uk1s3.embassy.ebi.ac.uk/idr/zarr/test-data/v0.6dev2/idr0115/13966272_tiles.zarr

* upgrade to Jupyter book 2 * correctly mark examples * bump mystmd dependency * pin mystmd * hashpin template * remove legacy dependencies * Create README.md for OME-NGFF specification * remove legacy files * use auto-built footer for copyright * Update `coordinateSystems` metadata * Updated `axes` metadata and moved under `coordinateSystems` header level * Update array coordinate systems metadata and merge existing examples * Update coordinate convention metadata * Updated coordinate transformations metadata * updated matrix transformations * update transformation types * update transformation types metadata * harmonize link syntax * removed deprecated statement about `byDimension` transform * harmonized indendations and json style * fixed `byDimension` metadata * make examples collapsible * renamed example folder * add orcid logo to editor info * fix reference * change precedence See discussion [here](bogovicj/ngff-rfc5-coordinate-transformation-examples#11 (comment)) * added examples for transformations with discrete axes * fix link to example * untrack autogenerated files * update affine examples and fix variable ordering * Fixed scale examples * removed example from additional details * specify affines/rotations as 2D matrices in parameter table * renamed coordinate transformations schema * Refactor rotation property to use `mtxFlatOrNested` * Delete schemas.md * remove `mtxFloatOrNested` affine/rotation matrices should always be 2D * "zarr array" instead of "binary data" * WIP: Update rfc5 schemas (#1) I'm merging this so that all necessary changes regarding the addition of rfc5-stuff to the main branch of the ngff-spec repo can be in one place. This branch was supposed to be a break-out to keep the commit-history of ome#17 clean. * Update input/output to input_axes/output_axes in schema * Add description and required field to byDimension * Update mapAxis schema to use integer array * Add path and interpolation to displacements schema The displacements object now includes a required 'path' property for specifying the zarr array location and an 'interpolation' property with supported methods. This enhances the schema's ability to describe displacement fields and their application. * Add path and interpolation to coordinates schema Introduces 'path' and 'interpolation' properties to the 'coordinates' object in the schema, specifying the location of the coordinate field and the interpolation method to use. The 'path' property is now required. * only allow paths, no URLs * move required fields to correct places * name musnt't be empty * added axis types to schema * Added descriptions to schemas * updated versions to 0.6dev2 * fix paths and versions * pull schema from correct location * Added action to run the tests * update config reference * update versions in test suite * Revert "update config reference" This reverts commit ff3fedd. * update all version references to "0.6dev2" * update version reference * at least two spatial axes * allow any axis type * update version reference * Add maxItems constraint to axes schema Set a maximum of 5 items for the axes array in the schema to enforce limits on the number of axes allowed. * Refactor image schema for coordinate transformations Refactors the definition of coordinateTransformations and coordinateSystems to use inline array schemas with stricter constraints. Adds a new multiscale_coordinateTransformations definition to support scale and translate transformations for multiscale datasets. * Update to NGFF 0.6dev2 and refactor coordinate systems Updated example and test JSON files to use the NGFF 0.6dev2 specification. Replaced 'axes' with 'coordinateSystems', added explicit 'input' and 'output' fields to coordinateTransformations, and restructured transformation chains for clarity and compliance with the new spec. * fix schema resolution * sync PR with latest RFC5 proposal (#2) Co-authored-by: Will Moore <900055+will-moore@users.noreply.github.com> Co-authored-by: David Stansby <d.stansby@ucl.ac.uk> Co-authored-by: Davis Bennett <davis.v.bennett@gmail.com> * fix links * fix schema resolution * move example/schema index pages in _generated folder and fix encoding error * removed unnecessary code and added traceback print * Comment out example tests * correct versioning semantic Previously `0.6dev2`, now `0.6.dev2` * typo --------- Co-authored-by: Will Moore <900055+will-moore@users.noreply.github.com> Co-authored-by: David Stansby <d.stansby@ucl.ac.uk> Co-authored-by: Davis Bennett <davis.v.bennett@gmail.com>

* upgrade to Jupyter book 2 * correctly mark examples * bump mystmd dependency * pin mystmd * hashpin template * remove legacy dependencies * Create README.md for OME-NGFF specification * remove legacy files * use auto-built footer for copyright * Update `coordinateSystems` metadata * Updated `axes` metadata and moved under `coordinateSystems` header level * Update array coordinate systems metadata and merge existing examples * Update coordinate convention metadata * Updated coordinate transformations metadata * updated matrix transformations * update transformation types * update transformation types metadata * harmonize link syntax * removed deprecated statement about `byDimension` transform * harmonized indendations and json style * fixed `byDimension` metadata * make examples collapsible * renamed example folder * add orcid logo to editor info * fix reference * change precedence See discussion [here](bogovicj/ngff-rfc5-coordinate-transformation-examples#11 (comment)) * added examples for transformations with discrete axes * fix link to example * untrack autogenerated files * update affine examples and fix variable ordering * Fixed scale examples * removed example from additional details * specify affines/rotations as 2D matrices in parameter table * renamed coordinate transformations schema * Refactor rotation property to use `mtxFlatOrNested` * Delete schemas.md * remove `mtxFloatOrNested` affine/rotation matrices should always be 2D * "zarr array" instead of "binary data" * WIP: Update rfc5 schemas (#1) I'm merging this so that all necessary changes regarding the addition of rfc5-stuff to the main branch of the ngff-spec repo can be in one place. This branch was supposed to be a break-out to keep the commit-history of ome#17 clean. * Update input/output to input_axes/output_axes in schema * Add description and required field to byDimension * Update mapAxis schema to use integer array * Add path and interpolation to displacements schema The displacements object now includes a required 'path' property for specifying the zarr array location and an 'interpolation' property with supported methods. This enhances the schema's ability to describe displacement fields and their application. * Add path and interpolation to coordinates schema Introduces 'path' and 'interpolation' properties to the 'coordinates' object in the schema, specifying the location of the coordinate field and the interpolation method to use. The 'path' property is now required. * only allow paths, no URLs * move required fields to correct places * name musnt't be empty * added axis types to schema * Added descriptions to schemas * updated versions to 0.6dev2 * fix paths and versions * pull schema from correct location * Added action to run the tests * update config reference * update versions in test suite * Revert "update config reference" This reverts commit ff3fedd. * update all version references to "0.6dev2" * update version reference * at least two spatial axes * allow any axis type * update version reference * Add maxItems constraint to axes schema Set a maximum of 5 items for the axes array in the schema to enforce limits on the number of axes allowed. * Refactor image schema for coordinate transformations Refactors the definition of coordinateTransformations and coordinateSystems to use inline array schemas with stricter constraints. Adds a new multiscale_coordinateTransformations definition to support scale and translate transformations for multiscale datasets. * Update to NGFF 0.6dev2 and refactor coordinate systems Updated example and test JSON files to use the NGFF 0.6dev2 specification. Replaced 'axes' with 'coordinateSystems', added explicit 'input' and 'output' fields to coordinateTransformations, and restructured transformation chains for clarity and compliance with the new spec. * fix schema resolution * sync PR with latest RFC5 proposal (#2) Co-authored-by: Will Moore <900055+will-moore@users.noreply.github.com> Co-authored-by: David Stansby <d.stansby@ucl.ac.uk> Co-authored-by: Davis Bennett <davis.v.bennett@gmail.com> * fix links * fix schema resolution * split tests into separate files; add conformance test script * Conformance testing docs Also minor refactor test_validation * Add $schema to strict schemas * Add jsonschema_dingus * replaced `0.6dev2` by `0.6.dev2` --------- Co-authored-by: Will Moore <900055+will-moore@users.noreply.github.com> Co-authored-by: David Stansby <d.stansby@ucl.ac.uk> Co-authored-by: Davis Bennett <davis.v.bennett@gmail.com> Co-authored-by: Chris Barnes <chris.barnes@gerbi-gmb.de>

* upgrade to Jupyter book 2 * correctly mark examples * bump mystmd dependency * pin mystmd * hashpin template * remove legacy dependencies * Create README.md for OME-NGFF specification * remove legacy files * use auto-built footer for copyright * Update `coordinateSystems` metadata * Updated `axes` metadata and moved under `coordinateSystems` header level * Update array coordinate systems metadata and merge existing examples * Update coordinate convention metadata * Updated coordinate transformations metadata * updated matrix transformations * update transformation types * update transformation types metadata * harmonize link syntax * removed deprecated statement about `byDimension` transform * harmonized indendations and json style * fixed `byDimension` metadata * make examples collapsible * renamed example folder * add orcid logo to editor info * fix reference * change precedence See discussion [here](bogovicj/ngff-rfc5-coordinate-transformation-examples#11 (comment)) * added examples for transformations with discrete axes * fix link to example * untrack autogenerated files * update affine examples and fix variable ordering * Fixed scale examples * removed example from additional details * specify affines/rotations as 2D matrices in parameter table * renamed coordinate transformations schema * Refactor rotation property to use `mtxFlatOrNested` * Delete schemas.md * remove `mtxFloatOrNested` affine/rotation matrices should always be 2D * "zarr array" instead of "binary data" * WIP: Update rfc5 schemas (#1) I'm merging this so that all necessary changes regarding the addition of rfc5-stuff to the main branch of the ngff-spec repo can be in one place. This branch was supposed to be a break-out to keep the commit-history of ome#17 clean. * Update input/output to input_axes/output_axes in schema * Add description and required field to byDimension * Update mapAxis schema to use integer array * Add path and interpolation to displacements schema The displacements object now includes a required 'path' property for specifying the zarr array location and an 'interpolation' property with supported methods. This enhances the schema's ability to describe displacement fields and their application. * Add path and interpolation to coordinates schema Introduces 'path' and 'interpolation' properties to the 'coordinates' object in the schema, specifying the location of the coordinate field and the interpolation method to use. The 'path' property is now required. * only allow paths, no URLs * move required fields to correct places * name musnt't be empty * added axis types to schema * Added descriptions to schemas * updated versions to 0.6dev2 * fix paths and versions * pull schema from correct location * Added action to run the tests * update config reference * update versions in test suite * Revert "update config reference" This reverts commit ff3fedd. * update all version references to "0.6dev2" * update version reference * at least two spatial axes * allow any axis type * update version reference * Add maxItems constraint to axes schema Set a maximum of 5 items for the axes array in the schema to enforce limits on the number of axes allowed. * Refactor image schema for coordinate transformations Refactors the definition of coordinateTransformations and coordinateSystems to use inline array schemas with stricter constraints. Adds a new multiscale_coordinateTransformations definition to support scale and translate transformations for multiscale datasets. * Update to NGFF 0.6dev2 and refactor coordinate systems Updated example and test JSON files to use the NGFF 0.6dev2 specification. Replaced 'axes' with 'coordinateSystems', added explicit 'input' and 'output' fields to coordinateTransformations, and restructured transformation chains for clarity and compliance with the new spec. * fix schema resolution * sync PR with latest RFC5 proposal (#2) Co-authored-by: Will Moore <900055+will-moore@users.noreply.github.com> Co-authored-by: David Stansby <d.stansby@ucl.ac.uk> Co-authored-by: Davis Bennett <davis.v.bennett@gmail.com> * fix links * fix schema resolution * split tests into separate files; add conformance test script * Conformance testing docs Also minor refactor test_validation * Add $schema to strict schemas * Add jsonschema_dingus * replaced `0.6dev2` by `0.6.dev2` --------- Co-authored-by: Johannes Soltwedel <38459088+jo-mueller@users.noreply.github.com> Co-authored-by: Will Moore <900055+will-moore@users.noreply.github.com> Co-authored-by: David Stansby <d.stansby@ucl.ac.uk> Co-authored-by: Davis Bennett <davis.v.bennett@gmail.com>

align schema and spec text

jo-mueller added 7 commits October 24, 2025 11:29

Update input/output to input_axes/output_axes in schema

11e46d2

Add description and required field to byDimension

0615a22

Update mapAxis schema to use integer array

90d49bc

Add path and interpolation to coordinates schema

53f2ae8

Introduces 'path' and 'interpolation' properties to the 'coordinates' object in the schema, specifying the location of the coordinate field and the interpolation method to use. The 'path' property is now required.

only allow paths, no URLs

da238c1

move required fields to correct places

9969e4d

jo-mueller changed the title ~~Update rfc5 schemas~~ WIP: Update rfc5 schemas Oct 24, 2025

jo-mueller added 10 commits October 24, 2025 15:48

name musnt't be empty

1440931

added axis types to schema

f0fbb02

Added descriptions to schemas

763cb91

updated versions to 0.6dev2

85afeea

fix paths and versions

10e8e9f

pull schema from correct location

d15301e

Added action to run the tests

40bf356

update config reference

ff3fedd

update versions in test suite

bad006b

Revert "update config reference"

7e97f37

This reverts commit ff3fedd.

jo-mueller mentioned this pull request Oct 27, 2025

rotation and affine should be 2D list of numbers? ome/ngff-spec#11

Closed

jo-mueller added 8 commits October 28, 2025 14:05

update all version references to "0.6dev2"

f0cd2ac

update version reference

5719082

at least two spatial axes

57bbfba

allow any axis type

7691cb7

update version reference

5c8bb7d

Add maxItems constraint to axes schema

fd09279

Set a maximum of 5 items for the axes array in the schema to enforce limits on the number of axes allowed.

jo-mueller mentioned this pull request Oct 29, 2025

RFC5: coordinate systems and transformations ome/ngff-spec#17

Merged

jo-mueller mentioned this pull request Oct 29, 2025

Rfc5 review response ome/ngff#350

Merged

fix schema resolution

6ef928e

jo-mueller merged commit fc7fef0 into RFC5 Nov 3, 2025
3 checks passed

jo-mueller deleted the update-rfc5-schemas branch November 3, 2025 12:06

jo-mueller pushed a commit that referenced this pull request Feb 13, 2026

Merge pull request #1 from jo-mueller/omero_metadata_optional

87b04ae

align schema and spec text

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

WIP: Update rfc5 schemas#1

WIP: Update rfc5 schemas#1
jo-mueller merged 26 commits into
RFC5from
update-rfc5-schemas

jo-mueller commented Oct 24, 2025 •

edited

Loading

Uh oh!

github-actions Bot commented Oct 24, 2025

Uh oh!

will-moore commented Oct 24, 2025

Uh oh!

jo-mueller commented Nov 3, 2025

Uh oh!

Uh oh!

dstansby commented Nov 3, 2025

Uh oh!

will-moore commented Nov 3, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

jo-mueller commented Oct 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Oct 24, 2025

Automated Review URLs

Uh oh!

will-moore commented Oct 24, 2025

Uh oh!

jo-mueller commented Nov 3, 2025

Uh oh!

Uh oh!

dstansby commented Nov 3, 2025

Uh oh!

will-moore commented Nov 3, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

jo-mueller commented Oct 24, 2025 •

edited

Loading