Slidebook7Reader: missing files reported by getUsedFiles

[sbesson]

The new Slidebook7 reader introduced in #3731 works fine when being used by Bio-Formats command-line tools and the provided sample file is being tested daily by the repository tests.

However, importing the sample file into OMERO fails with a server-side exception:

2022-09-19 12:55:59,350 673        [      main] INFO          ome.formats.importer.ImportConfig - OMERO.blitz Version: 5.5.13-SNAPSHOT
2022-09-19 12:55:59,388 711        [      main] INFO          ome.formats.importer.ImportConfig - Bioformats version: 6.10.2-SNAPSHOT revision: dba89bd86ddb573efab182c1cb664862e44c8c8d date: 19 September 2022
2022-09-19 12:55:59,994 1317       [      main] INFO   formats.importer.cli.CommandLineImporter - Setting transfer to ln_s
2022-09-19 12:56:00,003 1326       [      main] INFO   formats.importer.cli.CommandLineImporter - Log levels -- Bio-Formats: ERROR OMERO.importer: INFO
2022-09-19 12:56:01,123 2446       [      main] INFO      ome.formats.importer.ImportCandidates - Depth: 4 Metadata Level: MINIMUM
2022-09-19 12:56:01,613 2936       [      main] INFO      ome.formats.importer.ImportCandidates - 1 file(s) parsed into 1 group(s) with 1 call(s) to setId in 483ms. (490ms total) [0 unknowns]
2022-09-19 12:56:01,833 3156       [      main] INFO       ome.formats.OMEROMetadataStoreClient - Attempting initial SSL connection to localhost:4064
2022-09-19 12:56:02,909 4232       [      main] INFO       ome.formats.OMEROMetadataStoreClient - Insecure connection requested, falling back
2022-09-19 12:56:03,494 4817       [      main] INFO       ome.formats.OMEROMetadataStoreClient - Pinging session every 300s.
2022-09-19 12:56:03,503 4826       [      main] INFO       ome.formats.OMEROMetadataStoreClient - Server: 5.6.3
2022-09-19 12:56:03,503 4826       [      main] INFO       ome.formats.OMEROMetadataStoreClient - Client: 5.5.13-SNAPSHOT
2022-09-19 12:56:03,503 4826       [      main] INFO       ome.formats.OMEROMetadataStoreClient - Java Version: 1.8.0_212
2022-09-19 12:56:03,503 4826       [      main] INFO       ome.formats.OMEROMetadataStoreClient - OS Name: Linux
2022-09-19 12:56:03,504 4827       [      main] INFO       ome.formats.OMEROMetadataStoreClient - OS Arch: amd64
2022-09-19 12:56:03,504 4827       [      main] INFO       ome.formats.OMEROMetadataStoreClient - OS Version: 3.10.0-1160.15.2.el7.x86_64
2022-09-19 12:56:03,827 5150       [2-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILESET_UPLOAD_PREPARATION
2022-09-19 12:56:08,518 9841       [2-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILESET_UPLOAD_START
2022-09-19 12:56:08,611 9934       [3-thread-1] INFO   s.importer.transfers.SymlinkFileTransfer - Transferring /uod/idr/repos/curated/slidebook7/QwickTour/QweekTour.sldy...
2022-09-19 12:56:08,791 10114      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_STARTED: /uod/idr/repos/curated/slidebook7/QwickTour/QweekTour.sldy
2022-09-19 12:56:09,262 10585      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_COMPLETE: /uod/idr/repos/curated/slidebook7/QwickTour/QweekTour.sldy
2022-09-19 12:56:09,385 10708      [2-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILESET_UPLOAD_END
2022-09-19 12:56:09,728 11051      [2-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - IMPORT_STARTED Logfile: 1512302
2022-09-19 12:56:09,739 11062      [l.Client-0] ERROR     ome.formats.importer.cli.ErrorHandler - INTERNAL_EXCEPTION: /uod/idr/repos/curated/slidebook7/QwickTour/QweekTour.sldy
java.lang.RuntimeException: Failure response on import!
Category: ::omero::grid::ImportRequest
Name: error-on-init
Parameters: {stacktrace=java.lang.IllegalArgumentException: 0 must be non-null and strictly positive.
	at ome.xml.model.primitives.PositiveInteger.<init>(PositiveInteger.java:48)
	at loci.formats.MetadataTools.populatePixelsOnly(MetadataTools.java:310)
	at loci.formats.MetadataTools.populatePixelsOnly(MetadataTools.java:273)
	at loci.formats.ChannelFiller.setId(ChannelFiller.java:238)
	at loci.formats.ReaderWrapper.setId(ReaderWrapper.java:662)
	at loci.formats.ChannelSeparator.setId(ChannelSeparator.java:293)
	at loci.formats.ReaderWrapper.setId(ReaderWrapper.java:662)
	at loci.formats.Memoizer.setId(Memoizer.java:690)
	at loci.formats.ReaderWrapper.setId(ReaderWrapper.java:662)
	at ome.services.blitz.repo.CheckedPath.bfSetId(CheckedPath.java:412)
	at ome.services.blitz.repo.ManagedImportRequestI.open(ManagedImportRequestI.java:763)
	at ome.services.blitz.repo.ManagedImportRequestI.init(ManagedImportRequestI.java:245)
	at omero.cmd.HandleI.steps(HandleI.java:438)
...

From the output of showinf, I think the issue is that only the top-level .sldy file is reported by the getUsedFiles API. So effectively the import only transmit a partial copy of the fileset:

Checking file format [SlideBook 7 SLD (native)]
Initializing reader
SlideBook7Reader initializing /uod/idr/repos/curated/slidebook7/QwickTour/QweekTour.sldy
Initialization took 0.512s

Reading core metadata
filename = /uod/idr/repos/curated/slidebook7/QwickTour/QweekTour.sldy
Series count = 2
...

/cc @nicolapapp

[nicolapapp]

So I have to add the getUsedFiles method. Looks like KLBReader.java is a good example.

[dgault]

Yeah, the getUsedFiles should return the full list of files required for the dataset, so including the full list of associated files. The KLBReader looks like it would be fine as an example but any reader that has multiple files in the dataset should have similar logic. If the files are related in a particular series rather than the full dataset then you can implement getSeriesUsedFiles (see https://github.com/ome/bioformats/blob/develop/components/formats-bsd/src/loci/formats/in/OMETiffReader.java#L405).

[melissalinkert]

Fixed by #4294.