v2.4.0

Reworked data input/loading in #187 by @AnthonyAndroulakis

• add url and data+name input options for Volumes
• add url and data+name input options for Meshes
• add url and data+name input options for MeshLayers
• save volume hdr info and dim info on image load
• organize serializers/deserializers into their own file
• save volume img data on image load (chunked)
• save mesh data (pts and tris) on mesh load (chunked)
• be able to change binary data (ex: volume.img) from backend/python and have it update on the frontend

    View changes on GitHub

In-depth details / Design decisions

Volume, Mesh, and MeshLayer Inputs

The Volume, Mesh, and MeshLayer classes have been updated to support loading from one of three sources:

1. Path: Load from a local file path.
2. URL: Load from a remote URL.
3. Data + Name: Load from binary data with a specified name.

Examples

Loading a Volume from Path, URL, or Data

from ipyniivue import NiiVue, Volume

# Load from a local file path
volume1 = Volume(path="path/to/volume.nii.gz")

# Load from a URL
volume2 = Volume(url="https://example.com/volume.nii.gz")

# Load from binary data with a name
with open("path/to/volume.nii.gz", "rb") as f:
    data = f.read()
volume3 = Volume(data=data, name="volume.nii.gz")

# Load volumes into NiiVue
nv = NiiVue()
nv.load_volumes([volume1, volume2, volume3])

The same approach can be applied to Mesh and MeshLayer objects.

Using Dictionaries or Objects

In addition to using dictionaries, you can now use Volume, Mesh, or MeshLayer objects directly when loading or adding to NiiVue. This is done for 2 reasons:

1. can add on-change handlers or traitlets-specific functions to volumes, meshes, and mesh layer objects before adding them onto the main widget
2. can feel more pythonic than interacting with dictionaries. You can still use dictionaries since that mirrors NiiVue JS usage.

# Use Volume objects directly
nv.load_volumes([volume1, volume2, volume3])

# Or use dictionaries
nv.load_volumes([
    {"path": "path/to/volume.nii.gz"},
    {"url": "https://example.com/volume.nii.gz"},
    {"data": data, "name": "volume.nii.gz"}
])

# You can also have a list that contains a mix of dictionaries and Volume objects.

Communication Updates

Chunked Communication from Frontend to Backend

ipywidgets uses tornado for websocket communication. Once major limitation is that tornado limits incoming messages to 10 mb. This is a known issue without an official implementation. See:

• jupyter-widgets/ipywidgets#2522
• jupyter/notebook#3468

Of course, one solution would be to just ask users to update their jupyterlab config when using ipyniivue, but this isn't ideal since it requires:

1. the user to do extra work
2. you can already send data from backend -> frontend that's way over 10 mb, why not allow the opposite (frontend -> backend)

And so, I've made the decision to implement chunking (specifically, thru the state-change communication line). I've tested my implementation in jupyter lab, jupyter notebook, google colab, and marimo. Marimo uses POST requests instead of WebSockets for frontend -> backend communication, so special handling has been implemented for that.

Visualization

Chunked Data Transfer from Frontend to Backend:

Frontend                             Backend
+------------+                       +------------+
|            | -- Chunk 1 ----------> |            |
|            | -- Chunk 2 ----------> |            |
| Large Data | -- Chunk 3 ----------> |            |
|            |          ...           | Reassemble |
|            | -- Chunk N ----------> |            |
+------------+                       +------------+

• Frontend breaks the large data into smaller chunks.
• Chunks are sent to the backend with index and type information.
• Backend receives chunks and reassembles them into the complete data.

Numpy arrays are used in the backend so that array type information can be properly communicated.

Updates from Backend to Frontend

Instead of sending the entire array, only the differences are sent:

Backend                                Frontend
+------------+        Only Changes       +------------+
|            | -- Indices & Values --->  |            |
| Large Data |                           | Large Data |
| (Modified) |                           | (Updated)  |
+------------+                           +------------+

• Backend identifies changes (modified indices and new values).
• Changes are sent to the frontend.
• Frontend applies updates to its local copy based on received indices and values.

If the array type is changed, then the entire array along with the new type is sent to the frontend.

    View changes on GitHub