deepzoom_tile.py creates tiles with inconsistent width or height

[ToshitakaK]

Context

Issue type (bug report or feature request): bug report
Operating system (e.g. Fedora 24, Mac OS 10.11, Windows 10): Windows 10
Platform (e.g. 64-bit x86, 32-bit ARM): 64-bit x86
OpenSlide Python version (openslide.__version__): 1.1.2
OpenSlide version (openslide.__library_version__): 3.4.1
Slide format (e.g. SVS, NDPI, MRXS): TIFF

Details

I have some tiles converted using OpenSlide's deepzoom_tile.py from TIFF, but I noticed that they have inconsistent width or height.

For example, I convert the CMU-1.tiff (from here) to DZI format using libvips, the tiles with level=10 will have the following size:

$ vips dzsave --tile-size 254 --overlap 1 CMU-1.tiff CMU-1
$ magick identify .\CMU-1_files\10\*.jpeg
.\CMU-1_files\10\0_0.jpeg JPEG 255x255 255x255+0+0 8-bit sRGB 3747B 0.000u 0:00.000
.\CMU-1_files\10\0_1.jpeg JPEG 255x256 255x256+0+0 8-bit sRGB 9657B 0.000u 0:00.000
.\CMU-1_files\10\0_2.jpeg JPEG 255x8 255x8+0+0 8-bit sRGB 756B 0.000u 0:00.000
.\CMU-1_files\10\1_0.jpeg JPEG 256x255 256x255+0+0 8-bit sRGB 6745B 0.000u 0:00.000
.\CMU-1_files\10\1_1.jpeg JPEG 256x256 256x256+0+0 8-bit sRGB 4395B 0.000u 0:00.000
.\CMU-1_files\10\1_2.jpeg JPEG 256x8 256x8+0+0 8-bit sRGB 754B 0.000u 0:00.000
.\CMU-1_files\10\2_0.jpeg JPEG 212x255 212x255+0+0 8-bit sRGB 8213B 0.000u 0:00.000
.\CMU-1_files\10\2_1.jpeg JPEG 212x256 212x256+0+0 8-bit sRGB 2354B 0.016u 0:00.000
.\CMU-1_files\10\2_2.jpeg JPEG 212x8 212x8+0+0 8-bit sRGB 738B 0.000u 0:00.000

In particular, the tiles in the last column have a width of 212px and the tiles in the last row have a height of 8px.

On the other hand, when converted using deepzoom_tile.py, the tiles in the last row and the last column are generated with different widths and heights. In particular, the last column contains tiles of different widths.

$ python .\examples\deepzoom\deepzoom_tile.py --size 254 --overlap 1 .\CMU-1.tiff
Tiling slide: wrote 31644/31644 tiles
$ magick identify .\CMU-1_files\10\*.jpeg
.\CMU-1_files\10\0_0.jpeg JPEG 255x255 255x255+0+0 8-bit sRGB 6734B 0.000u 0:00.000
.\CMU-1_files\10\0_1.jpeg JPEG 255x256 255x256+0+0 8-bit sRGB 16262B 0.000u 0:00.000
.\CMU-1_files\10\0_2.jpeg JPEG 255x7 255x7+0+0 8-bit sRGB 809B 0.000u 0:00.000
.\CMU-1_files\10\1_0.jpeg JPEG 256x255 256x255+0+0 8-bit sRGB 11741B 0.000u 0:00.000
.\CMU-1_files\10\1_1.jpeg JPEG 256x256 256x256+0+0 8-bit sRGB 7340B 0.000u 0:00.000
.\CMU-1_files\10\1_2.jpeg JPEG 256x7 256x7+0+0 8-bit sRGB 765B 0.000u 0:00.000
.\CMU-1_files\10\2_0.jpeg JPEG 212x255 212x255+0+0 8-bit sRGB 13246B 0.000u 0:00.000
.\CMU-1_files\10\2_1.jpeg JPEG 211x256 211x256+0+0 8-bit sRGB 3904B 0.000u 0:00.000
.\CMU-1_files\10\2_2.jpeg JPEG 212x7 212x7+0+0 8-bit sRGB 784B 0.000u 0:00.000

The tiles in the last column have both 211px and 212px tile widths. (Also, the height of the tiles in the last row seems to be different from libvips.)

I'm confused by this behavior; what tile size should OpenSlide return in this case?

[ToshitakaK]

Update:

I found DeepZoomGenerator.get_tile() returns tiles which have incorrect shape.
https://github.com/openslide/openslide-python/blob/v1.1.2/openslide/deepzoom.py#L141-L160

It seems that PIL.Image.thumbnail() is the cause. This function is used to resize the image while keeping the aspect ratio, but sometimes it becomes smaller than z_size.

    def get_tile(self, level, address):
        """Return an RGB PIL.Image for a tile.
        level:     the Deep Zoom level.
        address:   the address of the tile within the level as a (col, row)
                   tuple."""

        # Read tile
        args, z_size = self._get_tile_info(level, address)  # z_size=(212, 8) 
        tile = self._osr.read_region(*args)  # tile.size=(423, 14)

        # Apply on solid background
        bg = Image.new('RGB', tile.size, self._bg_color)
        tile = Image.composite(tile, bg, tile)

        # Scale to the correct size
        if tile.size != z_size:
            tile.thumbnail(z_size, Image.ANTIALIAS)  # tile.size=(212, 7), it's smaller than z_size
        
        return tile

DeepZoom users would expect tiles in the same row to have the same height, and tiles in the same column to have the same width.
I think we need to do something like using PIL.ImageOps.fit(), which is a function that "generates the exact tile size while maintaining the aspect ratio", seems like a good idea.

[lambda-science]

Update:

I found DeepZoomGenerator.get_tile() returns tiles which have incorrect shape.
https://github.com/openslide/openslide-python/blob/v1.1.2/openslide/deepzoom.py#L141-L160

It seems that PIL.Image.thumbnail() is the cause. This function is used to resize the image while keeping the aspect ratio, but sometimes it becomes smaller than z_size.

    def get_tile(self, level, address):
        """Return an RGB PIL.Image for a tile.
        level:     the Deep Zoom level.
        address:   the address of the tile within the level as a (col, row)
                   tuple."""

        # Read tile
        args, z_size = self._get_tile_info(level, address)  # z_size=(212, 8) 
        tile = self._osr.read_region(*args)  # tile.size=(423, 14)

        # Apply on solid background
        bg = Image.new('RGB', tile.size, self._bg_color)
        tile = Image.composite(tile, bg, tile)

        # Scale to the correct size
        if tile.size != z_size:
            tile.thumbnail(z_size, Image.ANTIALIAS)  # tile.size=(212, 7), it's smaller than z_size
        
        return tile

DeepZoom users would expect tiles in the same row to have the same height, and tiles in the same column to have the same width.
I think we need to do something like using PIL.ImageOps.fit(), which is a function that "generates the exact tile size while maintaining the aspect ratio", seems like a good idea.

Is there any update or fix for this ?
Some tiles that are generated doesn't make sens and are almost blank. It's weird

Some other slide are just weirdly-sized rectangle (and not 255x255).
Some tiles are generated while not existing in original image

[bkf15]

I am having the same issue with viewing tiff files. Some generated tiles are white with checker lines across it, like shown above. Weirdly, this only seems to happen on certain 'levels' of the image, while others are unaffected.

[Brysch]

I solved it (for my needs) by modifying /Lib/site-packages/openslide/deepzoom.py

    def get_tile(self, level, address, method="fill"):
        """Return an RGB PIL.Image for a tile.

        level:     the Deep Zoom level.
        address:   the address of the tile within the level as a (col, row)
                   tuple.
        method:    "fill" - fills the space of partial images and the target image size
                   "rescale" - scales partial images to the target size
        """
        # Read tile
        args, z_size = self._get_tile_info(level, address)
        tile = self._osr.read_region(*args)

        # Apply on solid background
        bg = Image.new(mode='RGBA', size=(self._z_t_downsample, self._z_t_downsample), color=self._bg_color)

        if method=="fill":
            tile = Image.composite(tile, bg, tile)
            # Scale to the correct size
            if tile.size != z_size:
                # Image.Resampling added in Pillow 9.1.0
                # Image.LANCZOS removed in Pillow 10
                # tile.thumbnail(z_size, getattr(Image, 'Resampling', Image).LANCZOS)            
                tile.thumbnail(tile.size, getattr(Image, 'Resampling', Image).LANCZOS) # fill
                
        if method=="rescale":           
            tile_new = Image.composite(tile, bg, tile)
            # Scale to the correct size
            if tile_new.size != z_size:
                tile = tile.resize(tile_new.size, Image.ANTIALIAS) # scale

        return tile

now one can choose to fill the space or to rescale the partial image by

image_tiles = DeepZoomGenerator(image_slides, tile_size=tile_size, overlap=0, limit_bounds=False)
image_tiles .get_tile(level, (col, row), method="fill")

or

image_tiles = DeepZoomGenerator(image_slides, tile_size=tile_size, overlap=0, limit_bounds=False)
image_tiles .get_tile(level, (col, row), method="rescale")