V3: Gracefully handle LZW overflows #2880

JimBobSquarePants · 2025-02-04T08:48:15Z

Prerequisites

I have written a descriptive pull-request title
I have verified that there are no overlapping pull-requests open
I have verified that I am following the existing coding patterns and practice as demonstrated in the repository. These follow strict Stylecop rules 👮.
I have provided test coverage for my change (where applicable)

Description

The issue is caused by using the full capacity of the pixel stack for intermediate pushes without reserving a slot for the final push, which leads to an overflow. By capping out the max index we can skip corrupt blocks.

Fixes #2859

Copilot

Copilot reviewed 3 out of 8 changed files in this pull request and generated no comments.

Files not reviewed (5)

tests/Images/External/ReferenceOutput/GifDecoderTests/Issue2859_LZWPixelStackOverflow_Rgba32_issue_2859_A.gif/00.png: Language not supported
tests/Images/External/ReferenceOutput/GifDecoderTests/Issue2859_LZWPixelStackOverflow_Rgba32_issue_2859_B.gif/00.png: Language not supported
tests/Images/External/ReferenceOutput/GifDecoderTests/Issue2859_LZWPixelStackOverflow_Rgba32_issue_2859_B.gif/01.png: Language not supported
tests/Images/Input/Gif/issues/issue_2859_A.gif: Language not supported
tests/Images/Input/Gif/issues/issue_2859_B.gif: Language not supported

antonfirsov · 2025-02-04T13:23:53Z

src/ImageSharp/Formats/Gif/LzwDecoder.cs

@@ -204,7 +205,7 @@ public void DecodePixelRow(Span<byte> indices)
                    this.code = this.oldCode;
                }

-                while (this.code > this.clearCode)
+                while (this.code > this.clearCode && this.top < maxTop)


Since we are here, can we also un-Unsafe the methods touched?

Only if we can get the same performance out. We're hitting two separate spans inside a loop which is will definitely incur bounds checks costs.

We can benchmark that. IMO it was a big mistake to start using Unsafe as default way of indexing things, since it opens us up to security risks and it should be our goal to significantly reduce the amount of Unsafe usage across the codebase. A gain of few percents is not worth the risk, it's better to find algorithmic optimizations.

@antonfirsov I've updated the LzwDecoder with the best I could do performance-wise using safe code.

I'm OK with the performance hit given the much-improved security. Given that System.Drawing doesn't actually fully decode the image I'm still happy with the the performance of the decoder.

Target Branch

BenchmarkDotNet=v0.13.0, OS=Windows 10.0.26100 11th Gen Intel Core i7-11370H 3.30GHz, 1 CPU, 8 logical and 4 physical cores .NET SDK=9.0.200-preview.0.25057.12 [Host] : .NET 7.0.20 (7.0.2024.26716), X64 RyuJIT Job-AUXYJW : .NET 6.0.36 (6.0.3624.51421), X64 RyuJIT Runtime=.NET 6.0 Arguments=/p:DebugType=portable IterationCount=3 LaunchCount=1 WarmupCount=3

Method TestImage Mean Error StdDev Ratio RatioSD Gen 0 Gen 1 Gen 2 Allocated

'System.Drawing Gif' Gif/rings.gif 294.7 us 100.98 us 5.53 us 1.00 0.00 - - - 168 B

'ImageSharp Gif' Gif/rings.gif 410.6 us 87.95 us 4.82 us 1.39 0.03 0.9766 - - 7,557 B

This PR

Method TestImage Mean Error StdDev Ratio RatioSD Gen 0 Gen 1 Gen 2 Allocated

'System.Drawing Gif' Gif/rings.gif 299.0 us 189.8 us 10.41 us 1.00 0.00 - - - 168 B

'ImageSharp Gif' Gif/rings.gif 486.9 us 153.3 us 8.40 us 1.63 0.06 0.9766 - - 7,715 B

antonfirsov

LGTM. The ~15-20% regression seems somewhat worse than what I expected, but I think we can improve on that by micro-optimizations.

Mentioning a few suggestions I found, but they might be not the most efficient ideas, I'm OK adding them in another PR if you prefer so.

The ideal approach would be to profile the code anyways. If that detects some indexing bottlenecks which can't be eliminated by other means, I wouldn't be against limited surgical applications of Unsafe, but those should come with a very mindful, well-documented analysis of the code arguing it's safe.

antonfirsov · 2025-02-08T00:53:35Z

src/ImageSharp/Formats/Gif/LzwDecoder.cs

+                while (code > clearCode && top < maxTop)
                {
-                    Unsafe.Add(ref pixelStackRef, (uint)this.top++) = Unsafe.Add(ref suffixRef, (uint)this.code);
-                    this.code = Unsafe.Add(ref prefixRef, (uint)this.code);
+                    pixelStack[top++] = suffix[code];


pixelStack can be sliced down here to turn this into a for loop that runs until pixelStackSlice.Length.

antonfirsov · 2025-02-08T00:55:08Z

src/ImageSharp/Formats/Gif/LzwDecoder.cs

+                    prefix[availableCode] = oldCode;
+                    suffix[availableCode] = first;


We can slice down one of these spans.

I think I understand you and had a look. It seems like this would make the code a little more difficult to follow so have left it.

antonfirsov · 2025-02-08T01:05:56Z

src/ImageSharp/Formats/Gif/LzwDecoder.cs

-            xyz++;
-            Unsafe.Add(ref pixelsRowRef, (uint)x++) = (byte)Unsafe.Add(ref pixelStackRef, (uint)this.top);
+            // Clear missing pixels.
+            indices[xyz++] = (byte)pixelStack[top];


Since xyz seems to run till indices.Length this can be a straightforward for-loop.

We don't always increment on each iteration so that's not possible.

antonfirsov · 2025-02-08T01:06:57Z

src/ImageSharp/Formats/Gif/LzwDecoder.cs

            }

            // Pop a pixel off the pixel stack.
-            this.top--;
+            top--;


Is there a some logical guarantee that top doesn't run below zero?

As soon as we hit zero we enter the condition above. In there I believe there's at least one condition that increments top by one. The worst thing that could ever happen if top fals below zero is that we skip to the end of the pixel indices.

antonfirsov · 2025-02-08T01:15:45Z

src/ImageSharp/Formats/Gif/LzwDecoder.cs

+        int max = Math.Min(this.clearCode, suffix.Length);
+        for (this.code = 0; this.code < max; this.code++)
        {
-            Unsafe.Add(ref suffixRef, (uint)this.code) = (byte)this.code;
+            suffix[this.code] = (byte)this.code;


This could run on a slice of suffix.

Or if there is an upper limit to Math.Min(this.clearCode, suffix.Length), it can be even copied out from a statically initialized buffer.

Clear code maxes out at 256. We can certainly slice here.

JimBobSquarePants · 2025-02-10T01:49:48Z

@antonfirsov Managed to squeeze a little more performance out of this. Still haven't done any profiling though. Will merge once built.

BenchmarkDotNet=v0.13.0, OS=Windows 10.0.26100
11th Gen Intel Core i7-11370H 3.30GHz, 1 CPU, 8 logical and 4 physical cores
.NET SDK=9.0.200-preview.0.25057.12
  [Host]     : .NET 7.0.20 (7.0.2024.26716), X64 RyuJIT
  Job-JWXLPI : .NET 6.0.36 (6.0.3624.51421), X64 RyuJIT

Runtime=.NET 6.0  Arguments=/p:DebugType=portable  IterationCount=3
LaunchCount=1  WarmupCount=3

Method	TestImage	Mean	Error	StdDev	Ratio	RatioSD	Gen 0	Gen 1	Gen 2	Allocated
'System.Drawing Gif'	Gif/rings.gif	281.1 us	156.76 us	8.59 us	1.00	0.00	-	-	-	173 B
'ImageSharp Gif'	Gif/rings.gif	434.7 us	66.17 us	3.63 us	1.55	0.06	0.9766	-	-	7,401 B

JimBobSquarePants added 2 commits February 4, 2025 18:41

Gracefully handle LZW overflows

b99bf32

Add reference output

1709797

JimBobSquarePants added bug formats:gif security labels Feb 4, 2025

JimBobSquarePants requested review from tocsoft, antonfirsov, dlemstra, brianpopow and Copilot February 4, 2025 08:48

Copilot AI reviewed Feb 4, 2025

View reviewed changes

Update build-and-test.yml

9cb022b

antonfirsov reviewed Feb 4, 2025

View reviewed changes

Use safe code.

ca07754

Erik-White mentioned this pull request Feb 7, 2025

Handle GIF images with missing clear code #2884

Closed

antonfirsov approved these changes Feb 8, 2025

View reviewed changes

antonfirsov reviewed Feb 8, 2025

View reviewed changes

Clean up and optimize

20e5596

JimBobSquarePants merged commit 40643c1 into release/3.1.x Feb 10, 2025
8 checks passed

JimBobSquarePants deleted the js/fix-2859 branch February 10, 2025 02:04

JimBobSquarePants restored the js/fix-2859 branch February 10, 2025 02:05

JimBobSquarePants mentioned this pull request Feb 10, 2025

V4: Gracefully handle LZW overflows #2887

Merged

4 tasks

bennetrr mentioned this pull request Mar 7, 2025

chore(deps): bump SixLabors.ImageSharp from 3.1.6 to 3.1.7 in /src/backend/Bennetr.BrickInv.RebrickableClient in the nuget group across 1 directory bennetrr/brickinv#79

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

V3: Gracefully handle LZW overflows #2880

V3: Gracefully handle LZW overflows #2880

JimBobSquarePants commented Feb 4, 2025

Copilot AI left a comment

antonfirsov Feb 4, 2025 •

edited

Loading

JimBobSquarePants Feb 4, 2025

antonfirsov Feb 4, 2025 •

edited

Loading

JimBobSquarePants Feb 5, 2025

antonfirsov left a comment •

edited

Loading

antonfirsov Feb 8, 2025

antonfirsov Feb 8, 2025

JimBobSquarePants Feb 10, 2025

antonfirsov Feb 8, 2025

JimBobSquarePants Feb 10, 2025 •

edited

Loading

antonfirsov Feb 8, 2025

JimBobSquarePants Feb 10, 2025

antonfirsov Feb 8, 2025

antonfirsov Feb 8, 2025 •

edited

Loading

JimBobSquarePants Feb 10, 2025

JimBobSquarePants commented Feb 10, 2025 •

edited

Loading

Method	TestImage	Mean	Error	StdDev	Ratio	RatioSD	Gen 0	Gen 1	Gen 2	Allocated
'System.Drawing Gif'	Gif/rings.gif	294.7 us	100.98 us	5.53 us	1.00	0.00	-	-	-	168 B
'ImageSharp Gif'	Gif/rings.gif	410.6 us	87.95 us	4.82 us	1.39	0.03	0.9766	-	-	7,557 B

Method	TestImage	Mean	Error	StdDev	Ratio	RatioSD	Gen 0	Gen 1	Gen 2	Allocated
'System.Drawing Gif'	Gif/rings.gif	299.0 us	189.8 us	10.41 us	1.00	0.00	-	-	-	168 B
'ImageSharp Gif'	Gif/rings.gif	486.9 us	153.3 us	8.40 us	1.63	0.06	0.9766	-	-	7,715 B

		prefix[availableCode] = oldCode;
		suffix[availableCode] = first;

V3: Gracefully handle LZW overflows #2880

V3: Gracefully handle LZW overflows #2880

Conversation

JimBobSquarePants commented Feb 4, 2025

Prerequisites

Description

Copilot AI left a comment

Choose a reason for hiding this comment

antonfirsov Feb 4, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

antonfirsov Feb 4, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

antonfirsov left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

JimBobSquarePants Feb 10, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

antonfirsov Feb 8, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

JimBobSquarePants commented Feb 10, 2025 • edited Loading

antonfirsov Feb 4, 2025 •

edited

Loading

antonfirsov Feb 4, 2025 •

edited

Loading

antonfirsov left a comment •

edited

Loading

JimBobSquarePants Feb 10, 2025 •

edited

Loading

antonfirsov Feb 8, 2025 •

edited

Loading

JimBobSquarePants commented Feb 10, 2025 •

edited

Loading