Encountering issues when rendering vtkImageViewer2 with VtkLocalView

[pengwuke]

When I run the following program and modify the rendering from VtkRemoteView to VtkLocalView, I do not get the correct results.

The rendering results are correct when using VtkRemoteView：

"""
Need trame to work: `pip install trame`
"""
import os, vtk

from trame.app import get_server
from trame.widgets import vuetify, vtk as vtk_widgets
from trame.ui.vuetify import SinglePageLayout

# Attempt with https://kitware.github.io/vtk-examples/site/Java/IO/JPEGReader/
# vtk.vtkNativeLibrary # Didn't work, but maybe we don't need
vtk.vtkRenderWindowInteractor
# vtk.vtkImageReader2 # Need but we have already from vtk.vtkJPEGReader via inheritance
vtk.vtkJPEGReader  # inherits from vtk.vtkImageReader2
vtk.vtkNamedColors
vtk.vtkImageViewer2

images_path = "images"
os.listdir(images_path)
image = os.path.join(images_path, "MRN 803567 2_aff_registered.jpg")


# The following is python transcribed from java code almost exactly
# Colors
colors = vtk.vtkNamedColors()

# Read Image
jpegReader = vtk.vtkJPEGReader()
jpegReader.SetFileName(image)

# Visualize
imageViewer2 = vtk.vtkImageViewer2()
imageViewer2.SetInputConnection(jpegReader.GetOutputPort())
renderWindowInteractor = vtk.vtkRenderWindowInteractor()
imageViewer2.SetupInteractor(renderWindowInteractor)
imageViewer2.Render()
imageViewer2.GetRenderer().ResetCamera()
imageViewer2.GetRenderer().SetBackground(colors.GetColor3d("DarkSlateGray"))
imageViewer2.GetRenderWindow().SetWindowName("JPEGReader")

# ---------------------------------------
# Disable regular VTK event handling
# ---------------------------------------
# imageViewer2.Render()
# renderWindowInteractor.Start()
# ---------------------------------------

# Setup trame server
server = get_server()
ctrl = server.controller

# Build UI
with SinglePageLayout(server) as layout:
    with layout.toolbar as toolbar:
        layout.title.set_text("JPEG Viewer")
        vuetify.VSpacer()
        with vuetify.VBtn(icon=True, click=ctrl.view_reset_camera):
            vuetify.VIcon("mdi-crop-free")

    with layout.content:
        with vuetify.VContainer(classes="fluid fill-height pa-0"):
            #with vtk_widgets.VtkRemoteView(imageViewer2.GetRenderWindow()) as view:
            with vtk_widgets.VtkLocalView(imageViewer2.GetRenderWindow()) as view:
                ctrl.view_update = view.update
                ctrl.view_reset_camera = view.reset_camera

# Start server
server.start()

[jourdain]

The VtkLocalView is not mapping 100% of what VTK/C++ has to offer. I'm not sure what vtkImageViewer2 actually build as pipeline internally, but it could be that vtk.js is missing classes compared to its C++ counterpart or more simply we could be missing some serializer to do the mapping between C++ to JS data structure.

[pengwuke]

Writing code for local rendering of the MPR (Multi-Planar Reconstruction) feature in Trame seems not straightforward; however, ParaView Glance has already implemented this functionality. Can ParaView Glance be used directly in Trame?

[jourdain]

Glance can not be used directly but can be adapted to leverage a server-side like trame. But it really boils down to what you aim to do. You can "easily" extract the MPR part of Glance into a trame component that just needs to get the full data to work. But is that what you want trame to do? (send big data around?) If that is the case, trame might not be what you are looking for since a regular web server can do that in a simpler fashion.

HTH