Skip to content

Imaris HDF: allow paths to be in a subdirectory#4421

Open
melissalinkert wants to merge 3 commits intoome:developfrom
melissalinkert:imaris-path-change
Open

Imaris HDF: allow paths to be in a subdirectory#4421
melissalinkert wants to merge 3 commits intoome:developfrom
melissalinkert:imaris-path-change

Conversation

@melissalinkert
Copy link
Copy Markdown
Member

See https://forum.image.sc/t/ims-files-cannot-open-in-bio-formats-again/118877/10.

Relevant data is https://zenodo.org/records/18852087 and https://zenodo.org/uploads/19090243.

Without this change, showinf -debug on either file should show:

loci.formats.FormatException: Unknown pixel type: null
        at loci.formats.in.ImarisHDFReader.initFile(ImarisHDFReader.java:339)
        at loci.formats.FormatReader.setId(FormatReader.java:1480)
        at loci.formats.ImageReader.setId(ImageReader.java:864)
        at loci.formats.ReaderWrapper.setId(ReaderWrapper.java:692)
        at loci.formats.tools.ImageInfo.testRead(ImageInfo.java:1054)
        at loci.formats.tools.ImageInfo.main(ImageInfo.java:1165)

h5ls on either file will show something like:

DataSet                  Soft Link {/Workflows/InitialImages/DataSet}
DataSetEvents            Soft Link {/Workflows/InitialImages/DataSetEvents}
DataSetInfo              Soft Link {/Workflows/InitialImages/DataSetInfo}
DataSetTimes             Soft Link {/Workflows/InitialImages/DataSetTimes}
Thumbnail                Soft Link {/Workflows/InitialImages/Thumbnail}
Workflows                Group

netcdf-java does not appear to automatically resolve soft links, so to get the correct path prefix (Workflows/InitialImages) this change looks for the ImarisVersion attribute and captures the parent path.

@imagesc-bot
Copy link
Copy Markdown

This pull request has been mentioned on Image.sc Forum. There might be relevant details there:

https://forum.image.sc/t/ims-files-cannot-open-in-bio-formats-again/118877/13

@melissalinkert
Copy link
Copy Markdown
Member Author

No build failures in https://bf-testing-results.s3.amazonaws.com/2026/2026-03-24/build-status.log, but #3645 still applies.

@melissalinkert melissalinkert requested a review from sbesson March 24, 2026 16:01
Copy link
Copy Markdown
Member

@sbesson sbesson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the absence of native support for symlink resolution in NetCDF-Java which is unfortunate, detecting the path prefix as proposed here might be our best workaround.

Testing using using the sample IMS file from https://zenodo.org/records/18852087, a few high-level questions/comments :

  • this proposal is using the ImarisVersion attribute. Interesting, this attribute is undefined in https://imaris.oxinst.com/support/imaris-file-format and the FormatVersion attribute expected as per this specification is missing from the sample. Would it make sense to use ImarisDataSet instead to have some consistency with the official specification document?
  • in the provided IMS samples, there are two ImarisVersion (or ImarisDataset) attributes: one at the top-level and one under the hierarchy. Is it correct that the current logic works parses the groups and attributes recursively starting with the root group so the last occurence of the attribute will be the one in the leaf group of interest?
  • extending the above, I assume in the hypothetical case of multiple group hierarchies containing an ImarisDataset, the last parsed group would take precedence independently of the top-level symlink.

@melissalinkert
Copy link
Copy Markdown
Member Author

  • this proposal is using the ImarisVersion attribute. Interesting, this attribute is undefined in https://imaris.oxinst.com/support/imaris-file-format and the FormatVersion attribute expected as per this specification is missing from the sample. Would it make sense to use ImarisDataSet instead to have some consistency with the official specification document?

Done in b417b7b. That seems to work locally, so hopefully tests continue to pass overnight.

  • in the provided IMS samples, there are two ImarisVersion (or ImarisDataset) attributes: one at the top-level and one under the hierarchy. Is it correct that the current logic works parses the groups and attributes recursively starting with the root group so the last occurence of the attribute will be the one in the leaf group of interest?

Yes, the reader is iterating over the list of attributes recursively parsed from https://github.com/ome/bioformats/blob/develop/components/formats-gpl/src/loci/formats/services/NetCDFServiceImpl.java#L113 and https://github.com/ome/bioformats/blob/develop/components/formats-gpl/src/loci/formats/services/NetCDFServiceImpl.java#L243.

  • extending the above, I assume in the hypothetical case of multiple group hierarchies containing an ImarisDataset, the last parsed group would take precedence independently of the top-level symlink.

Exactly, yes. I could see a case for only setting the pathPrefix if it is empty, which means that if a file had something like:

  • /ImarisDataSet
  • /Workflows/InitialImages/ImarisDataSet
  • /Workflows/Workflow0/ImarisDataSet

then /Workflows/InitialImages would be chosen (not /Workflows/Workflow0 as is the case now). We don't have data that demonstrates this case though, so that might be premature.

Copy link
Copy Markdown
Member

@sbesson sbesson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Retested with https://zenodo.org/records/18852087 and I can confirm that Bio-Formats 8.5.0 fails to open this file while the latest version of this PR successfully opens an image with two resolutions and 330x330x12x2x121 dimensions. When setting the log level to DEBUG, the initialization also indicates the path prefix is updated

Set path prefix to 
Set path prefix to Workflows/InitialImages

Looking at the image metadata, the IMS file was derived from a Slidebook file and processed using Imaris 11

OriginalFormat: Intelligent Imaging Innovations: SlideBook
OriginalFormatFileIOVersion: ImarisFileIO x64 11.0.0

The workflows features introduced in https://imaris.oxinst.com/support/imaris-release-notes/11-0-0 are consistent with the HDF layout of the representative sample. These changes should allow the reader to support the scenario where a single dataset has been processed via these workflows. As discussed during the review, supporting HDF files will multiple groups under Workflows is outside the scope of this PR and we would need representative samples as well as upstream information on how to resolve this scenario anyways.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants