Performance degrades for multi-image encode, is there support for multi-threaded encoder?

[zhenbuxianggaimingzi]

Describe the bug

When I want to compress a large resolution image, I want to cut the large resolution image into multiple sub-images of small resolution, such as 512*512, pillow_heif supports the compression of multiple sub-images, but the performance will be greatly reduced compared with the direct compression of large-resolution images. I noticed that multi-graph compression in the code seems to be serial for encode one by one,

pillow_heif/pillow_heif/heif.py

Line 604 in 55a3d39

for i, img in enumerate(images_to_save):

is there a multithreaded parallel encoder for multiple sub-images? Or is there something wrong with my Settings?

Steps/Code to Reproduce

rgb_img = Image.open("1-input.jpeg")
heif_file = HeifFile()
tile_size = 512
for i in range(0, rgb_img.height, tile_size):
    h_start = i
    h_end = i + tile_size
    for j in range(0, rgb_img.width, tile_size):
        w_start = j
        w_end = j + tile_size

        patch = rgb_img.crop((w_start, h_start, w_end, h_end))
        heif_file.add_from_pillow(patch)

heif_file.save(output_heif_path, quality=50)

and here is my test image
1-input.zip

Expected Results

The performance of multiple sub-images encode should be no lower than that of a single full-resolution graph directly encode

Actual Results

The performance of multi-subimage encode is more than 4 times lower than that of direct encode for a single full-resolution image
multi-subimage encode cost: 1.2968940734863281s
single full-resolution encode cost: 5.515080690383911s

Versions

3.10.15 | packaged by conda-forge | (main, Oct 16 2024, 01:24:24) [GCC 13.3.0]
Linux-6.8.0-49-generic-x86_64-with-glibc2.35
0.20.0
{'libheif': '1.18.2', 'HEIF': 'x265 HEVC encoder (3.5+1-f0c1022b6)', 'AVIF': 'AOMedia Project AV1 Encoder v3.6.1', 'encoders': {'x265': 'x265 HEVC encoder (3.5+1-f0c1022b6)', 'aom': 'AOMedia Project AV1 Encoder v3.6.1', 'mask': 'mask'}, 'decoders': {'libde265': 'libde265 HEVC decoder, version 1.0.15', 'aom': 'AOMedia Project AV1 Decoder v3.6.1'}}

[bigcat88]

From your code I see it's not a tiled image, you're just creating a file with 4 images, right?

Support for tiled image encoding was only added in the latest libheif version 1.19, but I haven't added tiled image support to pillow-heif yet.

[zhenbuxianggaimingzi]

Yes, because I don't know how to create a tiled image from a large resolution image, I did not find the parameter Settings related to tiled in the code, so I manually cut the large resolution image into several 512*512 small pictures, and then stack these small pictures together and use encode_image interface for encode.

In addition, In the latest 0.21.0 update log, libheif version was updated to 1.19.5, which is also the latest version of libheif. Is it because some features in libheif have not been integrated?

[bigcat88]

Even though the latest version is with libheif 1.19, these APIs are still not used.

I'll take a look in a few days to see how hard it is to add (how much code is needed for this) support for tiled images.

If anyone want implement this himself, I'll gladly accept PR for this

[MimoKar]

Even though the latest version is with libheif 1.19, these APIs are still not used.

I'll take a look in a few days to see how hard it is to add (how much code is needed for this) support for tiled images.

If anyone want implement this himself, I'll gladly accept PR for this

I think it will be enought if you set the autotile param for encoder to true as default. Then it will decide tilling automatically based on image resolution.

[bigcat88]

To do this, we need to completely rework image reading, not just writing.

To read images, use the following functions:

https://github.com/strukturag/libheif/blob/2b736bf786292da16b34a73bf82dad745f3c3115/libheif/api/libheif/heif.h#L1282-L1309

After that, we can implement only tile encoding into an image.

Since Pillow itself supports tiles, with a total rewrite of the pillow-heif code, it is easier to abandon native functions and support only the work as a Pillow plugin.

But I would still wait until Pillow starts supporting more than 8 bits per channel in rgb/rgba modes, so as not to rewrite everything completely twice, since this is a very large amount of work.