[Canary] Transcoding support using NetVips #3978

maxpiva · 2025-08-03T21:21:43Z

Added

Transcoding support via NetVips
Adds support for transcoding/converting and serving image formats such as JXL, JPEG 2000, HEIF, AVIF, and TIFF.
Also switches JPEG encoding/decoding to use JPEGLI instead of the native JPEG implementation, providing faster performance and improved quality-to-size ratio.
Removed all SixLabors dependencies.
Decoupled image processing into two interfaces:
- IImageFactory
- IImage
  Kavita is now decoupled from any specific graphics library.
Added two implementations:
- NetVips (default)
- ImageMagick (can be enabled via conditional compilation)
Extended ImageService:
- Exposes access to the image factory.
- Introduces ReplaceImageFileFormat, which auto-converts unsupported formats (e.g., AVIF, JXL) to JPEG based on browser compatibility.
- Adds tests to validate format handling across supported backends.
- Allows switching between NetVips and ImageMagick via project defines.
  Note: Only one backend is supported at a time.
Refactored ImageExtensions to remove SixLabors usage and delegate to the active backend.
Added local packages/ directory containing custom netvips.native NuGet packages with support for JXL, JPEG 2000, and JPEGLI.
These may be published to NuGet.org pending review from Kleisauke.

Changed

Converted some static methods in ImageService to instance methods to support dependency injection of the graphics backend.
Moved WillScaleWell and IsLikelyWideImage into ImageExtensions.

Notes

Tested successfully on Windows, Linux x64, and Docker over a 2-day period.
Further testing is recommended for macOS, ARM, and other platforms.
Building libvips with JPEGLI, JXL, and JPEG 2000 support required patching and custom builds.
Special thanks and hat tip to Kleisauke for his foundational work, without it, this integration would have been extremely difficult.
Custom forks of netvips, libvips-packaging, and build-win64-mxe are available in my GitHub account and can be reused if needed, just be aware I just pushed until it works, some changes might be unrequired.

maxpiva · 2025-08-03T21:24:27Z

FYI: @majora2007, @DieselTech Additional Notes: Docker this time was not changed since new format support are embedded into the new netvips packages.

Co-authored-by: Copilot <[email protected]>

maxpiva · 2025-08-04T01:20:37Z

Canary docker with latest changes in this PR can be found in maxpiva/kavita:nightly

majora2007

This is a preliminary review from just the code, I will pull this down as well and do some shakeout testing.

majora2007 · 2025-08-03T22:17:09Z

API.Benchmark/ArchiveServiceBenchmark.cs

-    private readonly PngEncoder _pngEncoder = new PngEncoder();
-    private readonly WebpEncoder _webPEncoder = new WebpEncoder();
+    private readonly IImageFactory _imageFactory;
    private const string SourceImage = "C:/Users/josep/Pictures/obey_by_grrsa-d6llkaa_colored_by_me.png";


Whatever test this is, we probably need to recreate or remove it. I dont even have this file anymore.

Just ported it over, should remove the test?

majora2007 · 2025-08-03T22:17:55Z

API.Benchmark/ArchiveServiceBenchmark.cs

+    public void ImageFactory_ExtractImage_PNG()
    {
-        var outputDirectory = "C:/Users/josep/Pictures/imagesharp/";
+        var outputDirectory = "C:/Users/josep/Pictures/netvips/";


We need to refactor this to use the TestData folder within the project and not paths on my dev machine. (../../../Services/Test Data/ArchiveService/)

API.Benchmark/ArchiveServiceBenchmark.cs

majora2007 · 2025-08-04T11:39:48Z

API.Tests/Services/ArchiveServiceTests.cs

+        var thumbnail = imageService.ImageFactory.Create(Path.Join(testDirectory, expectedOutputFile));
+        int width = 320;
+        int height =(int)(thumbnail.Height * (width / (double)thumbnail.Width));
+        thumbnail = thumbnail.Thumbnail(width, height);


Can we not wrap this in the Image Interface so it's not so manual across the tests? Looks like a duplication of logic to me.

We should also refactor this to use var dims = CoverImageSize.Default.GetDimensions();. It never got updated when I introduced the concept.

On second thought, I don't see the use case for this, archiveService.GetCoverImage would internally make the same calls anyway.

majora2007 · 2025-08-04T11:42:07Z

API.Tests/Services/CoverDbServiceTests.cs

        // Load and compare similarity

-        var similarity = expectedFaviconPath.CalculateSimilarity(actualFaviconPath); // Assuming you have this extension
+        var similarity = _imageService.ImageFactory.CalculateSimilarity(expectedFaviconPath, actualFaviconPath); // Assuming you have this extension


Curious why the imageService.CalculateSimularity doesn't internally call the Image factory leading to a much cleaner API?

Can go one way or the other, depending of SRP, will change

majora2007 · 2025-08-05T16:08:20Z

API/Services/ImageServices/NetVips/NetVipsImage.cs

+            }
+
+            // Ensure band count is 4 (RGBA)
+            if (image.Bands == 1)


Why do we need to maintain all this logic now even when with NetVips still?

I think so, The colourspace and similarity algorithms, uses rawpixel data.

API/Services/ImageServices/NetVips/NetVipsImageFactory.cs

majora2007 · 2025-08-05T16:09:51Z

API/Services/ImageServices/NetVips/NetVipsImageFactory.cs

+        using var res = im.Resize(percent / 100f);
+        float[] pixels = NetVipsImage.GetRGBAFloatImageDataFromImage(res);
+        if (pixels == null)
+            return new List<Vector3>();


Bring to same line. Can we use Array.Empty instead of a new instance?

majora2007 · 2025-08-05T16:10:19Z

API/Services/Tasks/Metadata/CoverDbService.cs

                    {
                        var existingPath = Path.Combine(_directoryService.CoverImageDirectory, person.CoverImage);
-                        var betterImage = existingPath.GetBetterImage(tempFullPath)!;
+                        var betterImage = _imageService.ImageFactory.GetBetterImage(existingPath,tempFullPath)!;


Space after ,

Also moved to IImageService.

majora2007 · 2025-08-05T16:46:37Z

API/Services/Tasks/Scanner/Parser/Parser.cs

    public const string MacOsMetadataFileStartsWith = @"._";

-    public const string SupportedExtensions =
+    public static string SupportedExtensions =


Why is this static and not const?

SupportedExtensions is what we support from the Disk, not from the client.

It's a bit tricky to explain, but here we go, there are two distinct concerns here.

I had to switch to a static field because the base value is no longer a constant. It's now a constructed string that depends on the supported formats. Otherwise, we’d have to duplicate the list of extensions here, and I’d rather keep a single source of truth to avoid human error during edits.

You're absolutely right: what the browser or client supports is not the same as what we support from disk.

So, we split the disk-supported formats into two collections, one for formats universally supported by clients, and another for those that aren't.

maxpiva · 2025-08-05T18:57:37Z

Just pushed the additional changes. Didn't change the Benchmark methods, or the intent of the test method. until have more clarity. also, the IOC part I didn't understand the context.

majora2007 · 2025-08-05T19:26:04Z

API/Extensions/HttpExtensions.cs

+        List<string> supportedExtensions = Parser.UniversalFileImageExtensionArray.ToList();
+        //Early eject if the browser or api do not provide an Accept header.
+        if (!request.Headers.ContainsKey("Accept"))
+            return supportedExtensions;


This should be on the same line as the if

majora2007 · 2025-08-20T15:26:22Z

I just got back from a holiday, I'm planning to do one more once over and get this into a canary release for dedicated user testing.

Copilot

Pull Request Overview

This PR introduces transcoding support for modern image formats (JXL, JPEG 2000, HEIF, AVIF, TIFF) using NetVips library and abstracts Kavita's image processing through a clean interface pattern. The changes decouple the application from specific graphics libraries while enabling automatic format conversion based on browser compatibility.

Replaces SixLabors.ImageSharp dependency with NetVips (default) and optional ImageMagick backend
Introduces IImageFactory and IImage interfaces to abstract image processing operations
Adds automatic image format transcoding with browser compatibility detection via Accept headers

Reviewed Changes

Copilot reviewed 32 out of 54 changed files in this pull request and generated 9 comments.

Show a summary per file

File	Description
API/Services/Tasks/Scanner/Parser/Parser.cs	Adds support for non-universal image formats and browser compatibility mappings
API/Services/ImageServices/*.cs	New abstraction layer with NetVips and ImageMagick implementations
API/Services/ImageService.cs	Refactored to use new image abstraction with format conversion capabilities
API/Extensions/ImageExtensions.cs	Updated image utility methods to work with new abstraction layer
API/Controllers/ReaderController.cs	Adds automatic image format replacement based on browser support

API/Services/Tasks/Scanner/Parser/Parser.cs

Copilot · 2025-08-20T15:31:10Z

API/Services/Tasks/Metadata/CoverDbService.cs

            var filename = ImageService.GetWebLinkFormat(baseUrl, encodeFormat);
-
-            image.WriteToFile(Path.Combine(_directoryService.FaviconDirectory, filename));
+            await image.SaveAsync(Path.Combine(_directoryService.FaviconDirectory, filename),encodeFormat);


Missing space after comma. Should be: await image.SaveAsync(Path.Combine(_directoryService.FaviconDirectory, filename), encodeFormat);

Copilot · 2025-08-20T15:31:10Z

API/Services/Tasks/Metadata/CoverDbService.cs

                    {
                        var existingPath = Path.Combine(_directoryService.CoverImageDirectory, series.CoverImage);
-                        var betterImage = existingPath.GetBetterImage(tempFullPath)!;
+                        var betterImage =_imageService.GetBetterImage(existingPath, tempFullPath)!;


Missing space after the '=' operator. Should be: var betterImage = _imageService.GetBetterImage(existingPath, tempFullPath)!;

Copilot · 2025-08-20T15:31:11Z

API/Services/Tasks/Metadata/CoverDbService.cs

                    {
                        var existingPath = Path.Combine(_directoryService.CoverImageDirectory, chapter.CoverImage);
-                        var betterImage = existingPath.GetBetterImage(tempFullPath)!;
+                        var betterImage = _imageService.GetBetterImage(existingPath,tempFullPath)!;


Missing space after comma. Should be: var betterImage = _imageService.GetBetterImage(existingPath, tempFullPath)!;

Copilot · 2025-08-20T15:31:11Z

API/Services/ImageService.cs

+    {
+        if (supportedImageFormats == null) return false;
+
+        string ext = Path.GetExtension(filename).ToLowerInvariant().Substring(1);


Potential bug: Calling Substring(1) on an empty string will throw an exception if the filename has no extension. Add a null/empty check before calling Substring.

Suggested change

string ext = Path.GetExtension(filename).ToLowerInvariant().Substring(1);

string extension = Path.GetExtension(filename);

if (string.IsNullOrEmpty(extension) || extension.Length < 2) return false;

string ext = extension.ToLowerInvariant().Substring(1);

Copilot · 2025-08-20T15:31:11Z

API/Services/ImageService.cs

            }
-
-            image = image.Insert(tile, x, y);
+            image.Composite(tile, (int)x, (int)y);


The cast from double to int may cause precision loss. Consider using Math.Round or ensure x and y are already integers.

Suggested change

image.Composite(tile, (int)x, (int)y);

image.Composite(tile, (int)Math.Round(x), (int)Math.Round(y));

API/Services/ImageServices/NetVips/NetVipsImage.cs

Copilot · 2025-08-20T15:31:12Z

API/Extensions/HttpExtensions.cs

+        if (!extensions.Contains(extension))
+        {
+            extensions.Add(extension);
+        }


This method performs a linear search with Contains() for each extension. Consider using a HashSet for better performance when dealing with many extensions.

Suggested change

}

private static void AddExtension(HashSet<string> extensions, string extension)

{

if (string.IsNullOrEmpty(extension))

return;

extensions.Add(extension);

majora2007 · 2025-08-27T16:06:28Z

I'm back from holiday and going to get this canary released for user testing.

majora2007 · 2025-09-18T00:07:38Z

I haven't forgotten about this, just been heads down on the new annotation system. Once I get that stabilized, I'll pull this in and get it to some users.

majora2007 · 2025-09-18T20:04:20Z

I took care of the merge for you and pushed up the updated code into feature/transcoding-support. I already performed a serious of code cleanups to align with my coding style.

I see that 2 unit tests are broken (JXL/JP2), I would appreciate a quick look @maxpiva

Let's leave this PR open, but please target any changes towards my new branch.

maxpiva · 2025-09-19T13:06:32Z

letme check :)

maxpiva added 2 commits August 3, 2025 17:51

Transcoding support using NetVips. Image Operations Refactor.

a1ce3e2

Added Netvips w/JPEGLI, JPEG2000 and JXL.

dbba9aa

majora2007 added this to v0.8.8 - Kavita+ / FR Aug 3, 2025

github-project-automation bot moved this to Todo in v0.8.8 - Kavita+ / FR Aug 3, 2025

majora2007 requested a review from Copilot August 3, 2025 22:00

This comment was marked as outdated.

Sign in to view

maxpiva and others added 6 commits August 3, 2025 19:09

Update API/Services/ImageServices/NetVips/NetVipsImage.cs

b21baaf

Co-authored-by: Copilot <[email protected]>

Update API/Services/Tasks/Metadata/CoverDbService.cs

33b62a1

Co-authored-by: Copilot <[email protected]>

copilot PR fixes.

31fcbc9

Merge branch 'develop' of https://github.com/maxpiva/kavita into develop

995c47d

image return fix

7bfde6a

Added CancellationRequest early eject as suggested by copilot.

bb67141

majora2007 requested changes Aug 5, 2025

View reviewed changes

maxpiva added 3 commits August 5, 2025 14:05

Update ArchiveServiceBenchmark.cs

39a6fe5

Update ReaderController.cs

ec60add

Additional edits.

c738137

majora2007 reviewed Aug 5, 2025

View reviewed changes

Simplify by reducing one level of nesting

900c20d

majora2007 changed the base branch from develop to canary August 20, 2025 15:27

majora2007 changed the title ~~Transcoding support using NetVips. Image Operations Refactor. Abstract Image Library from Kavita~~ [Canary] Transcoding support using NetVips. Image Operations Refactor. Abstract Image Library from Kavita Aug 20, 2025

majora2007 added the enhancement New feature or request label Aug 20, 2025

majora2007 requested review from Fesaa and Copilot August 20, 2025 15:29

Copilot AI reviewed Aug 20, 2025

View reviewed changes

majora2007 moved this from Todo to In Progress in v0.8.8 - Kavita+ / FR Aug 20, 2025

majora2007 changed the title ~~[Canary] Transcoding support using NetVips. Image Operations Refactor. Abstract Image Library from Kavita~~ [Canary] Transcoding support using NetVips Aug 27, 2025

majora2007 added this to Kavita Sep 18, 2025

github-project-automation bot moved this to In progress in Kavita Sep 18, 2025

majora2007 added this to v0.8.9 - Kavita+ / Journaling Progress Oct 10, 2025

majora2007 removed this from v0.8.8 - Kavita+ / FR Oct 10, 2025

github-project-automation bot moved this to Todo in v0.8.9 - Kavita+ / Journaling Progress Oct 10, 2025

-        string ext = Path.GetExtension(filename).ToLowerInvariant().Substring(1);
+        string extension = Path.GetExtension(filename);
+        if (string.IsNullOrEmpty(extension) || extension.Length < 2) return false;
+        string ext = extension.ToLowerInvariant().Substring(1);

	image.Composite(tile, (int)x, (int)y);
	image.Composite(tile, (int)Math.Round(x), (int)Math.Round(y));

-        }
+    private static void AddExtension(HashSet<string> extensions, string extension)
+    {
+        if (string.IsNullOrEmpty(extension))
+            return;
+        extensions.Add(extension);

Uh oh!

[Canary] Transcoding support using NetVips #3978

Are you sure you want to change the base?

[Canary] Transcoding support using NetVips #3978

Uh oh!

Conversation

maxpiva commented Aug 3, 2025 • edited by majora2007 Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Added

Changed

Notes

Uh oh!

maxpiva commented Aug 3, 2025

Uh oh!

This comment was marked as outdated.

Uh oh!

maxpiva commented Aug 4, 2025

Uh oh!

majora2007 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

maxpiva Aug 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

maxpiva commented Aug 5, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

majora2007 commented Aug 20, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Uh oh!

Copilot AI Aug 20, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Aug 20, 2025

Choose a reason for hiding this comment

Uh oh!

maxpiva commented Aug 3, 2025 •

edited by majora2007

Loading

maxpiva Aug 5, 2025 •

edited

Loading