Skip to content

Speed up uncompressed DDS reader - #9943

Open
akx wants to merge 3 commits into
python-pillow:mainfrom
akx:dds-speedup
Open

akx wants to merge 3 commits into
python-pillow:mainfrom
akx:dds-speedup

Conversation

@akx

@akx akx commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Follows up on #7589. I noticed this looking at pytest --durations; a fuzzer test that just loads a DDS was surprisingly slow, and turns out doing 1-byte reads is not very efficient...

Locally:

$ pytest-benchmark compare --between=min .benchmarks/Darwin-CPython-3.14-64bit/*.json
Name (time in us)                   0001_orig Min  0002_new Min      ΔMin
-------------------------------------------------------------------------
test_load[uncompressed_rgb.dds]      405,475.7920    1,589.0000    -99.6%
-------------------------------------------------------------------------

or 255.17 times lower minimum, or in average OPS terms, +22351.7% faster. 😂

Comment thread src/PIL/DdsImagePlugin.py Outdated
Comment thread src/PIL/DdsImagePlugin.py
Comment thread Tests/benchmarks.py
LOAD_PATHS = [
*SAVE_PATHS,
IMAGES_PATH / "uncompressed_rgb.dds",
]

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

An alternative to splitting up PATHS would be to change test_save_jpeg to test_save, and forgo the quality argument.

Is there any reason not to do that?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Um... test_save_jpeg always saves a JPEG (into memory).

That would make it run the same thing for more than one image - is there a reason to do that?

@radarhere radarhere Sep 15, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, I meant saving both JPEG and DDS images - akx#31

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Then we could be instead testing saving with the default benchmark image, into different formats?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At this stage, I'm just wondering why it isn't helpful to test saving DDS images, since it would seem simpler to do so.

Let me ask a different question though - why use flower2.jpg, rather than hopper.jpg?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Of course it can be helpful to also benchmark writing DDS images, it just didn't feel in scope to add that in a PR that's speeding up DDS reading.
I can whip up another PR that does that more comprehensively (e.g. benchmark all common codecs, both reading and writing)?

As for why flower2.jpg, I can't recall I had a particular reason for choosing it over hopper.jpg in #9654.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There: #10001

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you'd like to go forth with #10001, then sure, but don't do it for my sake. I was only trying to understand why there was a distinction being made between formats.

@radarhere

radarhere commented Sep 11, 2026

Copy link
Copy Markdown
Member

As far as uncompressed_rgb.dds is concerned, I think I can do you one better. #7589 added DdsRgbDecoder in order to generalise this decoding path, but before that, this file was read with the raw decoder.

So if apply the following change to main to use the raw decoder for this specific scenario, you will find it faster than the current state of this PR.

diff --git a/src/PIL/DdsImagePlugin.py b/src/PIL/DdsImagePlugin.py
index 40012bc27..a6cec0557 100644
--- a/src/PIL/DdsImagePlugin.py
+++ b/src/PIL/DdsImagePlugin.py
@@ -371,8 +371,11 @@ class DdsImageFile(ImageFile.ImageFile):
                 mask_count = 3
 
             masks = struct.unpack(f"<{mask_count}I", header[84 : 84 + mask_count * 4])
-            self.tile = [ImageFile._Tile("dds_rgb", extents, 0, (bitcount, masks))]
-            return
+            if masks == (0xFF0000, 0x00FF00, 0x0000FF):
+                rawmode = "BGR"
+            else:
+                self.tile = [ImageFile._Tile("dds_rgb", extents, 0, (bitcount, masks))]
+                return
         elif pfflags & DDPF.LUMINANCE:
             if bitcount == 8:
                 self._mode = "L"

@akx

akx commented Sep 15, 2026

Copy link
Copy Markdown
Contributor Author

@radarhere Good stuff! 🎉 Applied that on here too. I checked pytest-cov, and with that applied the speedups from dadbf4f are still tested and used, so this is a double win :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants