Jpeg compressed tiff with photometric interpretation Separated is not supported

[shapeh]

Prerequisites

• I have written a descriptive issue title
• I have verified that I am running the latest version of ImageSharp
• I have verified if the problem exist in both DEBUG and RELEASE mode
• I have searched open and closed issues to ensure it has not already been reported

ImageSharp version

3.0.1

Other ImageSharp packages and versions

None

Environment (Operating system, version and so on)

Windows 11 and Windows Server 2022

.NET Framework version

.NET 7.0.5

Description

Hi,

I am getting this error when trying to resize a tiff image:

Jpeg compressed tiff with photometric interpretation Separated is not supported

I looked at this: #12 and maybe it is still relevant, despite edited 2021?
"Photometric Interpretation Formats: Separated (TIFF Extension color spaces)" - not ticked.
Update: https://github.com/SixLabors/ImageSharp/blob/main/src/ImageSharp/Formats/Tiff/README.md - Separated seems to be supported?

Is there a plan for support at a later stage?

Thanks for a great library.

System.Exception: System.NotSupportedException: Jpeg compressed tiff with photometric interpretation Separated is not supported
at SixLabors.ImageSharp.Formats.Tiff.TiffThrowHelper.ThrowNotSupported(String message)
at SixLabors.ImageSharp.Formats.Tiff.Compression.Decompressors.JpegTiffCompression.DecodeJpegData(BufferedReadStream stream, Span1 buffer, CancellationToken cancellationToken) at SixLabors.ImageSharp.Formats.Tiff.Compression.Decompressors.JpegTiffCompression.Decompress(BufferedReadStream stream, Int32 byteCount, Int32 stripHeight, Span1 buffer, CancellationToken cancellationToken)
at SixLabors.ImageSharp.Formats.Tiff.Compression.TiffBaseDecompressor.Decompress(BufferedReadStream stream, UInt64 stripOffset, UInt64 stripByteCount, Int32 stripHeight, Span1 buffer, CancellationToken cancellationToken) at SixLabors.ImageSharp.Formats.Tiff.TiffDecoderCore.DecodeStripsChunky[TPixel](ImageFrame1 frame, Int32 rowsPerStrip, Span1 stripOffsets, Span1 stripByteCounts, CancellationToken cancellationToken)
at SixLabors.ImageSharp.Formats.Tiff.TiffDecoderCore.DecodeImageWithStrips[TPixel](ExifProfile tags, ImageFrame1 frame, CancellationToken cancellationToken) at SixLabors.ImageSharp.Formats.Tiff.TiffDecoderCore.DecodeFrame[TPixel](ExifProfile tags, CancellationToken cancellationToken) at SixLabors.ImageSharp.Formats.Tiff.TiffDecoderCore.Decode[TPixel](BufferedReadStream stream, CancellationToken cancellationToken) at SixLabors.ImageSharp.Formats.ImageDecoderUtilities.Decode[TPixel](IImageDecoderInternals decoder, Configuration configuration, Stream stream, Func3 largeImageExceptionFactory, CancellationToken cancellationToken)
at SixLabors.ImageSharp.Formats.ImageDecoderUtilities.Decode[TPixel](IImageDecoderInternals decoder, Configuration configuration, Stream stream, CancellationToken cancellationToken)
at SixLabors.ImageSharp.Formats.Tiff.TiffDecoder.Decode[TPixel](DecoderOptions options, Stream stream, CancellationToken cancellationToken)
at SixLabors.ImageSharp.Formats.Tiff.TiffDecoder.Decode(DecoderOptions options, Stream stream, CancellationToken cancellationToken)
at SixLabors.ImageSharp.Formats.ImageDecoder.<>c__DisplayClass12_01.<WithSeekableMemoryStreamAsync>g__PeformActionAndResetPosition|0(Stream s, Int64 position, CancellationToken ct) --- End of stack trace from previous location --- at SixLabors.ImageSharp.Formats.ImageDecoder.CopyToMemoryStreamAndActionAsync[T](DecoderOptions options, Stream stream, Func4 action, CancellationToken cancellationToken)
at SixLabors.ImageSharp.Formats.ImageDecoder.DecodeAsync(DecoderOptions options, Stream stream, CancellationToken cancellationToken)
at SixLabors.ImageSharp.Image.WithSeekableStreamAsync[T](DecoderOptions options, Stream stream, Func`3 action, CancellationToken cancellationToken)

Steps to Reproduce

The code is simple like this (image is coming from Azure Blob Storage but that is not the issue):

   ...
   ushort width = 600;
   var azureBlobData = await blob.OpenReadAsync(null, cancellationToken: token); // blob holds reference to the affected tiff image.
   using var image = await Image.LoadAsync(azureBlobData, token);
            var typeOfImage = image.Metadata.DecodedImageFormat; // for use later on.

            // resize original image to new width:
            image.Mutate(x =>
                x.AutoOrient()
                .Resize(width, 0)
            ); 
  ....

Images

LC-Snow_Snow.zip
Aeros.zip

[shapeh]

Added another image where ImageSharp complains about Separated.

[brianpopow]

We actually support photometric interpretation Separated, see CmykTests.

The issue here comes from the combination of JPEG compression and separated. I am not sure how to decode those image, that's why it's disabled and throws an exception with that message.

If I map the colorspace for separated to CMYK in TiffJpegSpectralConverter.cs the image does not look correct:

You can help figuring out what needs to be done to decode this image. Even better would be to provide a PR with a fix, but figuring out what needs to be done to decode this would already be very helpful.

[shapeh]

I am no expert in image formats and their underlying algorithms. I did some digging (not sure if it is helpful or you are already aware of this):

From the file:

<dc:format>image/tiff</dc:format>
<photoshop:ColorMode>4</photoshop:ColorMode>
<photoshop:ICCProfile>Generic CMYK</photoshop:ICCProfile>
<tiff:BitsPerSample>
<rdf:Seq>
   <rdf:li>8</rdf:li>
   <rdf:li>8</rdf:li>
   <rdf:li>8</rdf:li>
   <rdf:li>8</rdf:li>
</rdf:Seq>
</tiff:BitsPerSample>
<tiff:Compression>7</tiff:Compression>
<tiff:PhotometricInterpretation>5</tiff:PhotometricInterpretation>
<tiff:Orientation>1</tiff:Orientation>
<tiff:SamplesPerPixel>4</tiff:SamplesPerPixel>
<tiff:PlanarConfiguration>1</tiff:PlanarConfiguration>

So 7, JPEG Compression (new format), PhotometricInterpretation 5 equals CMYK.

From https://www.kronometric.org/phot/processing/DNG/dng_spec_1.4.0.0.pdf p. 19:

Two Compression tag values are supported in DNG versions before 1.4.0.0:

• Value = 1: Uncompressed data.
• Value = 7: JPEG compressed data, either baseline DCT JPEG, or lossless JPEG
  compression.

If PhotometricInterpretation = 6 (YCbCr) and BitsPerSample = 8/8/8, or if
PhotometricInterpretation = 1 (BlackIsZero) and BitsPerSample = 8, then the JPEG variant
must be baseline DCT JPEG.

Otherwise, the JPEG variant must be lossless Huffman JPEG. For lossless JPEG, the internal
width/length/components in the JPEG stream are not required to match the strip or tile's
width/length/components. Only the total sample counts need to match. It is common for CFA
images to be encoded with a different width, length or component count to allow the JPEG
compression predictors to work across like colors.

So, I guess we are looking at 'a lossless Huffman JPEG'?
Perhaps some code like this:
https://github.com/yigolden/JpegLibrary => https://github.com/yigolden/JpegLibrary/blob/main/src/JpegLibrary/ScanDecoder/JpegHuffmanLosslessScanDecoder.cs

Is this helpful (or am I out of my depth)?

[shapeh]

Here are the images converted to jpgs in Photoshop:

[brianpopow]

Is this helpful (or am I out of my depth)?

@shapeh: Thanks for looking into this, any help is always welcome!

Just to give some context how tiff decoding works (a bit simplified):

1. Decompress: Revert any compression of the data. The data can be compressed with various compression method, lzw, deflate or even being compressed as a jpeg stream, like we have here.
2. Convert the color to RGB from the colorspace the image data is in indicated by the Photometric interpretation. This image is Separated, which should mean its CMYK color space and the data needs to be converted from CMYK to RGB.

What makes decoding image data compressed as jpeg a bit complicated, is that usually the Jpeg decoding process also involves converting the data from internal representation (YCbCr, CMYK) to RGB. That's why my initial thought was to map the colorspace for separated to CMYK in TiffJpegSpectralConverter.cs. I believe now that was not the right call. I think we should map it to RGB, so no conversion is done when decoding the jpeg stream and leave the color conversion to CmykTiffColor{TPixel}.cs, otherwise the conversion will be done twice which produces the above wrong result.

I believe the first step (decompress jpeg data) is working correctly. I think the second step is what we need to look into and is not working as intended.

[shapeh]

@brianpopow - I am glad it is helpful.

I looks like the decompression works (because you can identify some of artifacts (window frame, funky chair etc) in your conversion that resembles the correct image).

Re no. 2 - Okay with the conversion to RGB from CMYK.
Is this necessary because JPEG does not support CMYK or does it have something to do with the internals of ImageSharp?

Please let me know if there is anything I can help with/investigate further.

[JimBobSquarePants]

@brianpopow TiffLibrary seems to have this covered. Is this useful to you?

https://github.com/yigolden/TiffLibrary/blob/cec3a858de270a74965abe5b57a19b12c7a0e8d0/src/TiffLibrary/ImageDecoder/TiffD
efaultImageDecoderFactory.cs#L735-L742

[brianpopow]

@JimBobSquarePants I was looking a bit further into this. I still think we should map the colorspace to RGB in TiffJpegSpectralConverter{TPixel}.cs and then handle the conversion to CMYK in CmykTiffColor{TPixel}.cs.

I have compared the decompressed data from TiffLibrary with ours. The decompressed data is not the same. It seems that the jpeg has 4 channels. Our Jpeg decompression only expecteds 3 channels with colorspace RGB.

I am not sure what the right steps are now to be able to decompress this. To me it seems:

• Add new entry in JpegColorSpace.cs RGBA
• Add new ColorConverter JpegColorConverter.RgbaScalar.cs
• Add JpegColorConverter.RgbaScalar to JpegColorConverterBase.cs
• (Have I missed something? anything else?)

Any opinions?

Here is simpler version to test with with just 4 colors:
Cmyk-jpeg.zip

[JimBobSquarePants]

Thanks for looking into this @brianpopow

Is it really Rgba?

It does seem that there are rgba jpegs out there.

https://stackoverflow.com/questions/6212897/can-i-use-libjpeg-to-read-jpegs-with-an-alpha-channel

Are you absolutely sure that the test CMYK image is the same as the sample? I was able to open the CMYK image with Windows Photo Viewer and Paint.NET but not LC-Snow_snow

[brianpopow]

It does seem that there are rgba jpegs out there.

https://stackoverflow.com/questions/6212897/can-i-use-libjpeg-to-read-jpegs-with-an-alpha-channel

I was not able to get those jpeg files from the stackoverflow post. The links seem to be broken.

Are you absolutely sure that the test CMYK image is the same as the sample? I was able to open the CMYK image with Windows Photo Viewer and Paint.NET but not LC-Snow_snow

Both images have Compression Scheme: JPEG and Photometric Interpretation: separated, so I thought they should be similar enough. The CMYK image has no color profile and only 4 colors, so I thought it would make debugging easier.
It was created with imagemagick: magick convert Cmyk.tiff -compress jpeg Cmyk.test.jpg
The input image is Cmyk.tiff from our test file directory. Not sure why some viewers have trouble with LC-Snow image. I noticed some warnings from tiffinfo:

TIFFReadDirectory: Warning, Unknown field with tag 347 (0x15b) encountered.
TIFFFetchNormalTag: Warning, Incorrect value for "RichTIFFIPTC"; tag ignored.

Other then that I am not sure what the problem is.

The only viewer I found which can display the LC-Snow image correctly is XnView (besides photoshop, which the OP has mentioned can open the files correctly):

[shapeh]

The only viewer I found which can display the LC-Snow image correctly is XnView (besides photoshop, which the OP has mentioned can open the files correctly):

Just a minor comment: I think this image from XnView is showing some weird greens/blues - it looks different from the image from Photoshop, i.e. less warm color tones.

[shapeh]

Here is the same image opened in Windows 11 Paint. Colors off but still opens.

[JimBobSquarePants]

@brianpopow @shapeh LibTiff.NET produces the same output as Paint.

LibTiff.NET does not recognize a known jpeg colorspace and makes no attempt at color conversion. However if I hardcode the input and output colorspaces from YCCK to CMYK...

...then I get the following conversion.

I don't have an understanding yet of how jpeg compression in tiff works so I cannot tell you whether your plan is sound. TiffJpegSpectralConverter<TPixel>.GetJpegColorSpaceFromPhotometricInterpretation confuses me. Why does it return Rgb for YcbCr?

[brianpopow]

I don't have an understanding yet of how jpeg compression in tiff works so I cannot tell you whether your plan is sound. TiffJpegSpectralConverter.GetJpegColorSpaceFromPhotometricInterpretation confuses me. Why does it return Rgb for YcbCr?

Usually the Photometric interpretation determines which Tiff ColorConverter we need to use. With tiff images its more complicated. Remember we usually do 1. Decompress, 2. Color Conversion. Take for example ycbcr_jpegcompressed.tiff from our tiff test images this is Jpeg compressed and has Photometric interpretation YCbCr.

If we would call JpegColorConverterBase.GetConverter(colorSpace, frame.Precision) with color space YCbCr as the photometric interpretation suggests, the jpeg decoder would convert to RGB from YCbCr and the again the Tiff color converter would again convert to YCbCr since the Phtotometric interpretation indicates so. That would be incorrect results.

So if we set GetJpegColorSpaceFromPhotometricInterpretation always to RGB there, the Jpeg decoder handles decompression while the YCbCrConverter then afterwards handles color conversion.

Some more context from the PR, which introduced this: #2177

LibTiff.NET does not recognize a known jpeg colorspace and makes no attempt at color conversion. However if I hardcode the input and output colorspaces from YCCK to CMYK...

So the question would be, how can we achieve the same here?