Investigate jupyter itk-widgets for multi-volume rendering

[GenevieveBuckley]

IT's not clear what the best way to

Currently we've been using ipyvolume

• [WIP] FEAT: Support for 3d image data #103
• [WIP] FEATURE: Multivolume rendering for multi-dimensional data widgetti/ipyvolume#186

But itk jupyter widgets has also been improving their support in this area too.
Recently multivolume support was merged into the vtk-js master branch

• https://kitware.github.io/vtk-js/examples/TestVolumeTypes.html
• Add multi-component volume rendering support Kitware/itk-vtk-viewer#122
• Add a multi-component volume rendering example Kitware/vtk-js#936 (comment)

Later on it might be possible to view multi-dimensional data with the volume renderer widget

• Binder: 3D ITK Images
• Binder: NumPy array images (processed with SciPy)

[GenevieveBuckley]

Just tried it with a 4D array

from itkwidgets import view
view(arr_4d)

Error traceback::

---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
/srv/conda/envs/notebook/lib/python3.6/site-packages/itkTemplate.py in __getitem__(self, parameters)
    265         try:
--> 266             return(self.__template__[tuple(cleanParameters)])
    267         except:

KeyError: (<itkCType signed short>, 4)

During handling of the above exception, another exception occurred:

KeyError                                  Traceback (most recent call last)
/srv/conda/envs/notebook/lib/python3.6/site-packages/itkTemplate.py in __getitem__(self, parameters)
    269             try:
--> 270                 return(self.__template__[tuple(cleanParameters)])
    271             except:

KeyError: (<itkCType signed short>, 4)

During handling of the above exception, another exception occurred:

KeyError                                  Traceback (most recent call last)
<ipython-input-37-f9ba020a412d> in <module>
----> 1 view(multidim_arr)

/srv/conda/envs/notebook/lib/python3.6/site-packages/itkwidgets/widget_viewer.py in view(image, ui_collapsed, annotations, interpolation, cmap, mode, shadow, slicing_planes, gradient_opacity, **kwargs)
    448             annotations=annotations, interpolation=interpolation, cmap=cmap,
    449             mode=mode, shadow=shadow, slicing_planes=slicing_planes,
--> 450             gradient_opacity=gradient_opacity, **kwargs)
    451     return viewer

/srv/conda/envs/notebook/lib/python3.6/site-packages/itkwidgets/widget_viewer.py in __init__(self, **kwargs)
    206 
    207     def __init__(self, **kwargs):
--> 208         super(Viewer, self).__init__(**kwargs)
    209         dimension = self.image.GetImageDimension()
    210         largest_region = self.image.GetLargestPossibleRegion()

/srv/conda/envs/notebook/lib/python3.6/site-packages/ipywidgets/widgets/widget.py in __init__(self, **kwargs)
    409         """Public constructor"""
    410         self._model_id = kwargs.pop('model_id', None)
--> 411         super(Widget, self).__init__(**kwargs)
    412 
    413         Widget._call_widget_constructed(self)

/srv/conda/envs/notebook/lib/python3.6/site-packages/traitlets/traitlets.py in __init__(self, *args, **kwargs)
    995             for key, value in kwargs.items():
    996                 if self.has_trait(key):
--> 997                     setattr(self, key, value)
    998                 else:
    999                     # passthrough args that don't set traits to super

/srv/conda/envs/notebook/lib/python3.6/site-packages/traitlets/traitlets.py in __set__(self, obj, value)
    583             raise TraitError('The "%s" trait is read-only.' % self.name)
    584         else:
--> 585             self.set(obj, value)
    586 
    587     def _validate(self, obj, value):

/srv/conda/envs/notebook/lib/python3.6/site-packages/traitlets/traitlets.py in set(self, obj, value)
    557 
    558     def set(self, obj, value):
--> 559         new_value = self._validate(obj, value)
    560         try:
    561             old_value = obj._trait_values[self.name]

/srv/conda/envs/notebook/lib/python3.6/site-packages/traitlets/traitlets.py in _validate(self, obj, value)
    589             return value
    590         if hasattr(self, 'validate'):
--> 591             value = self.validate(obj, value)
    592         if obj._cross_validation_lock is False:
    593             value = self._cross_validate(obj, value)

/srv/conda/envs/notebook/lib/python3.6/site-packages/itkwidgets/trait_types.py in validate(self, obj, value)
     28         self._source_object = value
     29 
---> 30         image_from_array = to_itk_image(value)
     31         if image_from_array:
     32             return image_from_array

/srv/conda/envs/notebook/lib/python3.6/site-packages/itkwidgets/_to_itk.py in to_itk_image(other_image_datatype)
     36             case_use_view = False
     37         if case_use_view:
---> 38             image_from_array = itk.GetImageViewFromArray(array)
     39         else:
     40             image_from_array = itk.GetImageFromArray(array)

/srv/conda/envs/notebook/lib/python3.6/site-packages/itkExtras.py in GetImageViewFromArray(arr, isVector)
    284     """Get an ITK image view from a Python array.
    285     """
--> 286     return _GetImageFromArray(arr, "GetImageViewFromArray", isVector)
    287 
    288 def _GetArrayFromVnlObject(vnlObject, function):

/srv/conda/envs/notebook/lib/python3.6/site-packages/itkExtras.py in _GetImageFromArray(arr, function, isVector)
    272     else:
    273         Dimension = arr.ndim
--> 274     ImageType = itk.Image[PixelType, Dimension]
    275     templatedFunction = getattr(itk.PyBuffer[ImageType], function)
    276     return templatedFunction(arr, isVector)

/srv/conda/envs/notebook/lib/python3.6/site-packages/itkTemplate.py in __getitem__(self, parameters)
    272                 raise KeyError(
    273                     'itkTemplate : No template %s for the %s class' %
--> 274                     (str(parameters), self.__name__))
    275 
    276     def __repr__(self):

KeyError: 'itkTemplate : No template (<itkCType signed short>, 4) for the itk::Image class'

In contrast, this is the output for a grayscale 3D array

from itkwidgets import view
view(arr_3d)

[GenevieveBuckley]

How to reproduce:

# Imports
import os
from urllib.request import urlretrieve

import scipy.misc
import numpy as np

import itk
from itkwidgets import view

# Download data
file_name = '005_32months_T2_RegT1_Reg2Atlas_ManualBrainMask_Stripped.nrrd'
if not os.path.exists(file_name):
    url = 'https://data.kitware.com/api/v1/file/564a5b078d777f7522dbfaa6/download'
    urlretrieve(url, file_name)

# Create image arrays
itk_image = itk.imread(file_name)
arr_3d = itk.GetArrayViewFromImage(itk_image)
arr_4d = np.stack([arr_3d, np.flip(arr_3d), np.flip(arr_3d, axis=1)], axis=-1)

# Visualize
view(arr_3d)
view(arr_4d)

[jwitos]

Hi @GenevieveBuckley, I was wondering if you ever figured out a nice way to display multiple volumes / 4d data? :)

[GenevieveBuckley]

I didn't pursue this further, I switched teams at work & moved focus into other projects. It's worth seeing whether there's been changes to ITK in the meantime.

Maybe @thewtex has more insight?