Use dask or bioio for extra image format support in `igneous image create`

[chourroutm]

Are you okay to add dependencies such as dask or bioio to igneous? I've noticed that CloudVolume has an optional flag for dask.

bioio has the OME-Zarr and Bioformats plugins for additional input support.

If so, I'll work on a PR.

[william-silversmith]

I think they are fine as extras.

[chourroutm]

I'm now working on this. I've added bioio as an extra:

BIOIO = [ "bioio",
          "bioio-ome-zarr",
          "bioio-bioformats",
        ]

setuptools.setup(
  setup_requires=['pbr', 'numpy'],
  entry_points={
    "console_scripts": [
      "igneous=igneous_cli:main"
    ],
  },
  extras_require={
    "bioio": BIOIO,
    "mysql": MYSQL,
    "nrrd": NRRD,
    "nii": NII,
    "all": MYSQL + NII + NRRD,
  },
  long_description_content_type="text/markdown",
  pbr=True,
)

but the extras are not recognized because of the pbr=True, is it meant to be this way?

---

Also, bioio-bioformats has this handy method bioio_bioformats.ReaderMetadata().get_supported_extensions() (link) which lists all the extension, but the current logic in

igneous/igneous_cli/cli.py

Lines 1867 to 1925 in 597cdb1

	def create(
	ctx, src, dest,
	resolution, offset,
	seg,
	encoding, encoding_level, encoding_effort,
	compress, chunk_size,
	h5_dataset
	):
	"""Create a Precomputed volume from another data source.
	Supports: .npy, .h5/.hdf5, .nii/.nii.gz, .nrrd, and .ckl files
	Hopefully will support others such as TIFF in the future.
	"""
	src = src.replace("file://", "")
	ext = normalize_file_ext(src)
	if ext == ".npy":
	with open(src, "rb") as f:
	shape, forder, dtype = read_array_header(f)
	arr = np.lib.format.open_memmap(src, dtype=dtype, shape=shape, fortran_order=forder, mode="r")
	elif ext == ".ckl":
	import crackle
	arr = crackle.util.aload(src)
	if arr.nbytes < int(1e9):
	arr = arr.decompress()
	elif ext == ".nrrd":
	import nrrd
	arr, header = nrrd.read(src)
	if arr.shape[0] == 3 and arr.ndim == 3:
	arr = arr[..., np.newaxis]
	arr = np.transpose(arr, axes=[1,2,3,0])
	elif ext == ".nii":
	import nibabel as nib
	arr = nib.load(src)
	arr = np.array(arr.dataobj)
	elif ext in (".h5", ".hdf5"):
	import h5py
	file = h5py.File(src, 'r')
	arr = np.array(file[h5_dataset])
	else:
	print(f"Format not supported: {ext}")
	return
	while arr.ndim < 3:
	arr = arr[..., np.newaxis]
	CloudVolume.from_numpy(
	arr, dest,
	resolution=resolution,
	voxel_offset=offset,
	chunk_size=chunk_size,
	layer_type=("segmentation" if seg else "image"),
	encoding=encoding,
	compress=compress,
	progress=True,
	encoding_level=encoding_level,
	encoding_effort=encoding_effort,
	)

is to import after reading the ext. Do you have a recommendation?

[william-silversmith]

Huh. If pbr is blocking extras, we should just get rid of pbr and specify versions manually.

As for detecting formats something like this could work:

try:
   import bioio_bioformats
   bioio_exts = bioio_bioformats.ReaderMetadata().get_supported_extensions()
except ImportError:
   bioio_exts = []

If you put it after the explicitly supported list, you can even save the possible latency of searching for a non-existent bioio module if an explicit extension matches first.