[BUG / QUESTION] DrawBitmap with SKFilterQuality.High causes bad performance

[nor0x]

Description

I'm drawing several SKBitmaps on a SKGLView on macOS, I'm using SKFilterQuality.High to get the best quality of the images. If the canvas is zoomed in and only one image is visible the performance is okay, but once I zoom out and multiple SKBitmaps are visible performance and rendering speed drops. Some of the images are quite big (i.e. 4000*300px) - so I wanted to check if the performance drop is caused by my code or if Skia(Sharp)s performance is limited in this case (even on the GPU)

Code
This is my PaintSurface method - repro link contains my full source

void CanvasView_PaintSurface(object sender, SkiaSharp.Views.Mac.SKPaintGLSurfaceEventArgs e)
{
    e.Surface.Canvas.Clear(SKColors.Black);
    e.Surface.Canvas.SetMatrix(_m);

    for(int i = 0; i < imgs.Count; i++)
    {
        e.Surface.Canvas.DrawBitmap(imgs.ElementAt(i), rects.ElementAt(i), HighQualityImagePaint);
    }
}

Expected Behavior
Since drawing is happening on the GPU (SKGLView) i expect smooth rendering and good performance.

Actual Behavior
Framerate drops and panning / zooming feels really slow

Basic Information

• Version with issue: 1.60.3
• IDE: <!-- Visual Studio for Mac 7.7.2 (Build 21)
• Platform Target Frameworks: Xamarin.Mac
  • macOS: 10.14.2 (18C54)
• Target Devices: Macbook Pro 13" (2018 Touchbar)

Screenshots

Reproduction Link

https://github.com/nor0x/SkiaSharp_PerfomanceExperiments

VS bug #770230

[mattleibow]

Thanks for reporting this, I had a look and saw some improvements that could be made, but also asked on the skia forums just in case they have some suggestions: https://groups.google.com/forum/#!topic/skia-discuss/ZFnS-NaUcvY

I hope that they will be able to provide some real suggestions and maybe some features that can be used to improve performance.

The main improvement is to not draw bitmaps as they have to be copied each frame to the GPU. I create a intermediate image that is backed by a texture:

for (int i = 0; i < originals.Count; i++)
{
	// instead of creating all the intermediates up front, do it as they come in and are rendered
	if (intermediates[i] == null)
	{
		var original = originals[i];
		using (var surf = SKSurface.Create(CanvasView.GRContext, true, new SKImageInfo(original.Width, original.Height)))
		{
			surf.Canvas.DrawBitmap(original, 0, 0);
			surf.Canvas.Flush();

			intermediates[i] = surf.Snapshot();
		}
	}

	canvas.DrawImage(intermediates[i], rectangles[i], HighQualityImagePaint);
}

[nor0x]

Thank you for the answer, and for reaching out to the Skia devs. I will keep an eye on the thread.

[mattleibow]

Some info for filter quality: https://groups.google.com/forum/#!topic/skia-discuss/Hpjma8785HA

• None - No filtering, just nearest-neighbor sampling.
• Low - Bilinear filtering.
• Medium - Adds mipmaps (so improves speed/quality on downscales, but still does bilinear for upscaling).
  High - Adds bicubic filtering when upscaling (continues to use mipmaps for downscale).

So, if you're using High, there is an abrupt algorithm change when the scale goes > 1, and the bicubic filter means many more samples for filtering.

[nor0x]

Hi @mattleibow, i finally had some time to experiment with the suggestions here. Adjusting the filter quality is okay for me but not the best solution. As the community in the skia forums suggested I also tried setting SetResourceCacheLimits which unfortunately led to a NullReferenceException in my Xamarin.Mac project.

I get the following stacktrace:
" at (wrapper managed-to-native) SkiaSharp.SkiaApi.gr_context_set_resource_cache_limits(intptr,int,intptr)\n at SkiaSharp.GRContext.SetResourceCacheLimits (System.Int32 maxResources, System.Int64 maxResourceBytes) [0x0000d] in <273adfac44b84a14b4b6d7b5f15069bf>:0 \n at SkiaPlayground.ViewController+<ViewDidLoad>d__15.MoveNext () [0x002d7] in /Users/johnny/SkiaPlayground/SkiaPlayground/ViewController.cs:63 \n--- End of stack trace from previous location where exception was thrown ---\n at System.Runtime.CompilerServices.AsyncMethodBuilderCore+<>c.<ThrowAsync>b__6_0 (System.Object state) [0x00000] in /Library/Frameworks/Xamarin.Mac.framework/Versions/5.2.1.15/src/Xamarin.Mac/mcs/class/referencesource/mscorlib/system/runtime/compilerservices/AsyncMethodBuilder.cs:1023 \n at Foundation.NSAsyncSynchronizationContextDispatcher.Apply () [0x00002] in /Library/Frameworks/Xamarin.Mac.framework/Versions/5.2.1.15/src/Xamarin.Mac/Foundation/NSAction.cs:178 \n at (wrapper managed-to-native) AppKit.NSApplication.NSApplicationMain(int,string[])\n at AppKit.NSApplication.Main (System.String[] args) [0x00040] in /Library/Frameworks/Xamarin.Mac.framework/Versions/5.2.1.15/src/Xamarin.Mac/AppKit/NSApplication.cs:100 \n at SkiaPlayground.MainClass.Main (System.String[] args) [0x00007] in /Users/johnny/SkiaPlayground/SkiaPlayground/Main.cs:10 "

[nor0x]

Hi @mattleibow,
i'm back at my image performance experiments. SkiaSharp 1.68.1 fixed my issues with SetResourceCacheLimits. I was able to render images on the GPU and add pan and pinch to my test canvas. As I'm still trying to draw as many large images as possible with smooth performance i followed the hints in the Google Skia Thread (https://groups.google.com/forum/#!topic/skia-discuss/ZFnS-NaUcvY) Yeah, I would try setting the GrContext budget to something absurdly large to see if that is a factor.

I have also switched from SKBitmap to SKImage for better performance.

During my testings i noticed that once all images are visible on the view, performance takes a big hit and FPS go down. Once i drag the canvas so that some images are off-screen everything is smooth at ~60FPS.

Here is another GIF showing my test project:

I have also updated my playground repo with the latest source:
https://github.com/nor0x/SkiaSharp_PerfomanceExperiments

not sure if this is a new issue, but it's directly related to my previous performance issues with image drawing - so i drop this here.