PERF: utils/cv/read_image_as_pil - 11x faster

[TristanBandat]

Hi,

I was line profiling the get_sliced_prediction() function and continued with the get_prediction() function.
In the get_prediction() function I was wondering why every image was converted to a PIL image in order to be then converted to a contiguous numpy array. As the get_sliced_prediction() function already provides numpy arrays and the contiguity is anyway given with np.ascontiguousarray(image_as_pil), I wanted to skip the np -> PIL -> np conversion.

So I looked into the read_image_as_pil() function.
BTW that function was taking about 30% of the get_sliced_prediction() computation time.

As it turns out, there is a very small BUG in the code, that I intended to fix:

elif isinstance(image, np.ndarray):
        # BUG: small images break the check
        if image.shape[0] < 5:  # image in CHW
            image = image[:, :, ::-1]
        image_pil = Image.fromarray(image)

If you provide the function with a very small image, you introduce a hard-to-find bug.
Additionally, the PIL convertion (last line from above) takes ~99% of the CPU time of this function.

So I wrote a function based on cv2 (which is already imported) to fix the bug:

def to_rgb_hwc(arr):
    a = np.asarray(arr)
    if a.ndim == 3 and a.shape[-1] == 3:     # HWC
        return cv2.cvtColor(a, cv2.COLOR_BGR2RGB)
    if a.ndim == 3 and a.shape[0] == 3:      # CHW
        a_hwc = np.transpose(a, (1,2,0))
        return cv2.cvtColor(a_hwc, cv2.COLOR_BGR2RGB)
    return a

Next, I added a flag called return_arr to the read_image_as_pil() function, with which one could skip that costly conversion:

def read_image_as_pil(image: Image.Image | str | np.ndarray, exif_fix: bool = True,
                      return_arr: bool = False) -> Image.Image:

Of course you have to add some if/else and some PIL <-> numpy conversions as well.
So now the call in the get_prediction() function looks like this:

# read image as pil
image_as_pil = read_image_as_pil(image, return_arr=True)
# get prediction
time_start = time.time()
detection_model.perform_inference(np.ascontiguousarray(image_as_pil))
time_end = time.time() - time_start
durations_in_seconds["prediction"] = time_end

if full_shape is None:
    if image_as_pil.ndim == 2:  # H x W (grayscale)
        h, w = image_as_pil.shape
    elif image_as_pil.ndim == 3:
        if image_as_pil.shape[-1] in (1, 3, 4):  # H x W x C  (channels-last)
            h, w = image_as_pil.shape[:2]
    full_shape = [h, w]
    # full_shape = [image_as_pil.height, image_as_pil.width]

With those changes I was able to get a reliable performance improvement of ~11x of the read_image_as_pil() function.
This concluded to a reliable performance improvement of ~26% of the get_sliced_prediction() function.
And a small bug fixed.

NOTE:
I run the tests on my notebook with a Intel Core Ultra 7 155H and using a RTX 500 Ada GPU for the prediction.
Model is a torchvision FasterRCNN and I performed my tests on 108 slices.
The function read_image_as_pil() was called twice in my setup, once with PIL.Image.Image output and once with a np.array as output.
I'm running SAHI 0.11.36 with Python 3.13.9.

I don't provide the line profiles on purpose (unless needed), one has to implement proper testing scripts and line profiling is not well suited for comparable numbers across machines...

[onuralpszr]

I was able to test and added via this PR #1353

Thank you for detailed report and test.