Issue with using 2D images in tigre.utilities.im_3d_denoise

[MargaretDuff]

Expected Behavior

TV denoised images

Actual Behavior

Instability (but only sometimes) in 2D case. Sometimes it kills my kernel! Seems stable for 3D cases

Code to reproduce the problem (If applicable)

The problem code

##2d image example - 1 slices 
from tigre.utilities.im_3d_denoise import im3ddenoise
from cil.utilities import dataexample
import numpy as np
from tigre.utilities.gpu import GpuIds
from cil.utilities.display import show2D
from cil.processors import Slicer

gpuids = GpuIds()
image = dataexample.SIMULATED_SPHERE_VOLUME.get()

image = image.get_slice(vertical='centre').as_array()
image = np.expand_dims(image, axis=0) 
print("image shape", image.shape)
for i in range(10):

    denoise = im3ddenoise(image, 20, 1.0 / 0.1, gpuids)

    #show2D([denoise, image], title=['im3ddenoise(image, 20, 1.0 / 0.1, gpuids)', 'image'], cmap='inferno', origin='upper-left')
    print("Run", i+1)
    print('Check for Nan', np.isnan(np.sum(denoise)))

Sometimes it works! Sometimes it all goes wrong:

Sometimes it kills my kernel!

For 2 slices it seems to consistently work fine:

image = dataexample.SIMULATED_SPHERE_VOLUME.get()

image = Slicer(roi={'vertical':(63,65,1)})(image).as_array()
print("image shape", image.shape)
for i in range(10):
    denoise = im3ddenoise(image, 20, 1.0 / 0.1, gpuids)

   # show2D([denoise, image], title=['im3ddenoise(image, 20, 1.0 / 0.1, gpuids)', 'image'], cmap='inferno', origin='upper-left')
    print('Check for Nan', np.isnan(np.sum(denoise)))

Specifications

• MATLAB/python version: python
• OS: Linux
• CUDA version: 12.9
• Tigre version: pytigre==2.6.0

[AnderBiguri]

You mention inestabilty: This means that its quite likely an out-of bounds error read. CUDA does weird and stochastic things when you read out of bounds, sometimes error, sometimes read random values. I suspect there is some if condition missing in the gradient/divergence code calculation that does not account for 2D.

Finding this won't be too hard, but I guess I need to also test if the if condition significantly affects computational speed, which they tend to do in CUDA.

[AnderBiguri]

Out of curiosity: How many GPUs are you running this on?

[MargaretDuff]

Out of curiosity: How many GPUs are you running this on?

2 GPUs

[AnderBiguri]

@MargaretDuff if you run it in 1 GPU, does the error dissapear?

It could be that the logic to split the problems between GPUs does not consider the edge case of 1 slice (i.e. no splitting required)

[MargaretDuff]

@AnderBiguri That works!

##2d image example - 1 slices 
from tigre.utilities.im_3d_denoise import im3ddenoise
from cil.utilities import dataexample
import numpy as np
from tigre.utilities.gpu import GpuIds, getGpuIds, getGpuNames
from cil.utilities.display import show2D
from cil.processors import Slicer

gpuids = GpuIds()
gpuids.devices = [0]  # Specify the GPU device IDs you want to use
print("Using GPU ids:", gpuids)

Using GPU ids: {'name': '', 'devices': [0]}

image = dataexample.SIMULATED_SPHERE_VOLUME.get()

image = image.get_slice(vertical='centre').as_array()
image = np.expand_dims(image, axis=0) 
print("image shape", image.shape)

for i in range(10):

    denoise = im3ddenoise(image, 20, 1.0 / 0.1, gpuids)

    #show2D([denoise, image], title=['im3ddenoise(image, 20, 1.0 / 0.1, gpuids)', 'image'], cmap='inferno', origin='upper-left')
    print("Run", i+1)
    print('Check for Nan', np.isnan(np.sum(denoise)))

image shape (1, 128, 128)
Run 1
Check for Nan False
Run 2
Check for Nan False
Run 3
Check for Nan False
Run 4
Check for Nan False
Run 5
Check for Nan False
Run 6
Check for Nan False
Run 7
Check for Nan False
Run 8
Check for Nan False
Run 9
Check for Nan False
Run 10
Check for Nan False

[AnderBiguri]

Ah great! then I need to add an if either in python side or CUDA side to make sure the splits are not done in this case.

In CUDA it would be somewhere here:

TIGRE/Common/CUDA/tv_proximal.cu

Line 223 in 989615d

if(mem_GPU_global> 5*mem_size_image+5*mem_slice_image*buffer_length*2){

Where the number of available GPUs also would need to be modified.

In python somewhere here:

TIGRE/Python/tigre/utilities/im_3d_denoise.py

Line 16 in 989615d

img = tvdenoise(img, iter, lmbda, gpuids)

Just catch image size, modify gpuids to just 1 GPU if slice is 1