Replace papaya with niivue

[CloseChoice]

I was contacted by Chris Rorden whose group is developing NiiVue (see https://github.com/niivue/niivue), which leverages WebGL2 (in contrast to Papaya which is WebGL1 based). He also offered support in the implementation, which might come in handy in case of any questions later on (see DICOM implemenation). I completely overlooked NiiVue when searching for frameworks.
Development speed or lack thereof was already mentioned as a potential risk with Papaya. NiiVue is well and actively maintained, simply compare these two contribution charts:
NiiVue:

Papaya:

I gave NiiVue a try and it supports all features Papaya does, though I find Papaya's UI slightly more appealing but that is just personal taste. There is also a 3D image of the scanned object included in the NiiVue UI, but that is possible for Papaya aswell (at least in some way, check the image in their github repo README.md).

from datasets import load_dataset

# new dataset compared to papaya PR, this has more interesting images
ds = load_dataset("TobiasPitters/nifti-papaya-testdata",
              split="train")
ds[1]['nifti']  # ds[2]['nifti'] is also interesting

Here's a brief video how this looks with NiiVue: https://github.com/user-attachments/assets/3f2a52d4-2109-45e2-aca8-e4a4b1e46b32

NOTE: I explicitly created this as draft PR since I suspect the DICOM support to be a crucial factor to decide which of these two is better suited for our needs. DICOM is supported by Papaya, and for NiiVue as well using a plugin, but as far as I understand one DICOM file contains one 2D image, therefore support for loading a whole folder, containing all 2D layers for a complete 3D image is desired. NiiVue supports this according to their docs, I am unsure about Papaya.

[neurolabusc]

@CloseChoice thanks for your work on this. As you mentioned, the prime developers for Papaya have moved on, so it is in maintenance mode, albeit it is mature and may fill all your requirements.

Papaya does reflect the era of its creation, so it uses WebGL1 (which only supports 2D textures) for display and pako for decompression. In contrast, NiiVue uses WebGL2 (where 3D textures provide a native representation for volumes) and compression streams (x4 decoding speed). A major benefit of 3D textures is simple support for 3D volume rendering using ray casting. Note the Papaya README shows an isosurface rendering based on a triangulated mesh. In contrast, NiiVue can show both volume rendering (good for data with fuzzy boundaries) as well as surface rendering (good when a clean isosurface can be defined). I think the gallery provides a nice example of NiiVue capabilities as well as minimal recipes.

I do agree that Papaya UI is more advanced: by design NiiVue is a graphic widget that can be embedded into a container that provides your preferred user interface (React, Angular, Vue, pure html, or even jupyter notebooks).

I think DICOM support is a challenge for any tool for several reasons: the diversity of the implementations and compression methods (transfer syntaxes), the fact that in classic DICOM each 2D slice is saved as a separate file (though note modern enhanced DICOM can save an entire 3D volume or even 4D timeseries in a single file), and the rate that this format has evolved over time. Papaya uses Daikon to handle DICOM images, and I think it is only one file at a time. In contrast, NiiVue provides plugins for complex image formats, so you can choose your desired tool. We do provide illustrate how to use dcm2niix WASM as a DICOM loader, and it can extract coherent volumes from a random assortment of files or a manifest of files - see the live demo. Note that diakon development has halted, while dcm2niix has halted, which impacts support for emerging compression methods (e.g. JPEG2000-HT). Having said that, if your primary focus is DICOM, cornerstonejs is probably a better choice than NiiVue or Papaya.

Another feature that may or may not be worth noting is that NiiVue has a plugin model that allows you to use a lot of mature image processing tools. So you can do image conversion, image processing (itk-wasm, niimath), image registration (flirt, elastix) and edge-based AI models. brainchop illustrates edge-based AI model inference for brain segmentation, extraction and parcellation, though we provide minimal examples for ONNX, tensorflowjs and tinygrad. This would provide a convenient way for huggingface inference models to be shared. After training, the models could be converted to ONNX and deployed on a web page, allowing the user to drag-and-drop images and process them regardless of operating system or graphics card manufacturer. Since the AI model inference leverages the users own graphics card, the privacy issues and hardware scaling concerns of cloud distribution are mitigated.

[CloseChoice]

@neurolabusc thanks so much for the nuanced and informative reply.
I am convinced that niivue is the better option here, having 3D support is huge and Papaya's UI features are actually not necessary at all, and AFAIS we can get what we need and more with some additional configuration for niivue as well.
Thanks a lot for words about DICOM, though the focus of this PR is not NifTI and not DICOM, I think having one tool being able to load both (and potentially more formats) is best, I'll definitely test the live demo. My primary interest in your thoughts about DICOM is to enable visualization as a follow-up to this PR ##7835. Even for the DICOM case NiiVue seems like a great option using the dcm2niix webassembly plugin, I think the main challenge is here how we let the user organize files in an intuitive way (e.g. provide DICOM folder class, and a DICOM document class where one folder can contain multiple documents and 3d visualization is on the folder level).

Given that NiiVue is a modern neuroimaging viewer, well maintained and widely used and we have @neurolabusc attention in case of questions/problems I think we should go ahead with NiiVue.

@lhoestq your thoughts are highly appreciated.