processing/filters

[martinschorb]

I just saw that clij comes with bdv functionality.
Would it be possible to access its filters through the new interface?

[tischi]

@haesleinhuepf Can you comment on the request of Martin (see above)?

[haesleinhuepf]

Hey @martinschorb ,

so far CLIJx uses BDV just as viewer. This is an experimental feature scheduled for CLIJ3 or CLIJ4 🙄

I would love to implement block-wise GPU-acceleration using CLIJ on Cells as managed inside BDV. However, there is quite some work to do down that road. 😉

If you need GPU-accelerated image processing on super large images, which can only be processed block-wise, potentially on multiple GPUs, I recommend trying the Hydra Image Processor made by @ericwait 💪 :
https://www.hydraimageprocessor.com/

Cheers,
Robert

[martinschorb]

The question is, could BDV just push the block around the current view (FOV,plane,zoom=resolutionlevel) into the GPU for CLIJ to process? The block size could be defined by some settings.

If the blocks size is decent (small) given the processing speed on the GPU, this might even enable to apply filters on-the-fly with CLIJ just returning the processed view to BDV.

As an option, whatever chunk has been processed could be directly written into a sparse h5 so eventually the whole relevant volume would be processed by browsing through the views.

Is this completely insane?

[haesleinhuepf]

This makes a lot of sense - it might just be an insane amount of work. At least if you do it in a general fashion ;-)

Getting the current 2D view, processing it on the GPU and generating a new BDV source might be straight forward though...

A starting point could be the ScreenShotDemo

[NicoKiaru]

With cached cell disk image we could probably do a bunch of processing, if we forget about the generality.

It should be straigthforward if you deal with pixel to pixel operation, like squaring an image.

But if need to apply operation on blocks that need to access the values of neighboring blocks (like a gaussian blur), then as @haesleinhuepf wrote it's a lot more work. That's also the most interesting operations...

[haesleinhuepf]

I know that @tpietzsch and @maarzt thought about these things earlier. Curious about their thoughts :-)

[tpietzsch]

As @NicoKiaru said, this should be relatively straightforward with cached CellImg.
To be clear, this has nothing to do with BigDataViewer. This would be just a CellImg where the data for each block is "loaded" by doing an operation on another image.

Have a look at this example.
In particular, this line

final Img< UnsignedShortType > gauss1 = factory.create(
	dimensions,
	cell -> Gauss3.gauss( sigma1, source, cell, 1 ),
	options().initializeCellsAsDirty( true ) );

creates a cached image where blocks are computed by a Gauss filter.
All you need to do is create a CellLoader which knows the full source image (which could of course itself be very large and cached) and when asked to fill a Cell crops the appropriately padded region from the source image, pushes it through clij, crops the result (if necessary), and copies it to the provided Cell.

CellLoader only has a single method, in the above example, cell -> Gauss3.gauss( sigma1, source, cell, 1 ) is the CellLoader. This is a more verbose example:
https://github.com/imglib/imglib2-cache-examples/blob/6c9a891638f76e1760204bf106a7fbc0d31f9e68/src/main/java/net/imglib2/cache/example06/Example06.java#L29-L49

Let me know how it goes...

In principle, it would get much more interesting when this is pushed further, where the cached image knows which parts are already in opencl buffers and tracks modifications on both sides, so that unnecessary copying back and forth could be avoided.

[maarzt]

I will soon try to use CellImg, BDV and CLIJ together. And it will be something along the lines what @tpietzsch wrote.

[haesleinhuepf]

I will soon try to use CellImg, BDV and CLIJ together.

Let me know if you need any support! That sounds to me like a strategically interesting project! ;-)