Implemented support for backdrop effects

[kekekeks]

This PR adds support for backdrop effects (well, effect, drop-shadow is not supported for backdrop and we only have blur as our other effect for now).

While rendering backdrop was always an easy task (snapshotting SKSurface has worked like a charm for years, 3.119 has dedicated backdrop effect support for layers), the invalidation was a problem.

When backdrop has a non-zero sampling radius (e. g. any kind of blur), and you change one pixel (behind or above backdrop, doesn't matter)
you are marking that one pixel as dirty. However if you need to re-render (X:Y), you need to sample
from (X+1,Y) too, but when new frame is rendered, our preserved frame contains the drawn backdrop,
not what was behind the backdrop at the point of rendering. So you need to re-draw (X+1,Y) too.
This propagates to (X+n,Y) within backdrop area for the same reason.

So if one pixel intersecting any backdrop rects has to invalidate the entire backdrop area and
that chains through intersecting backdrops.

One approach for reducing drawn area is to cache what was behind the backdrop on the previous
frame at the time as rendering into a retained texture (in a way similar to our bitmap cache).

The PR implements both approaches by having two separate cache modes:

• Retained — sampled backdrop is retained in a texture; only affected by dirty rects from visuals
  behind the backdrop-casting visual
• Volatile — sampled backdrop is not retained; if the effect has a non-zero sampling radius,
  the visual is affected by damage from ALL visuals regardless of being in front of or behind it.

This is controllable explicitly from the UI thread, compositor is allowed can do some overrides
on its side too, however.

Support for those 2 modes lives in separate parts of the pipeline: for volatile ones we
just explicitly get the list of volatile backdrops in current render target (which can be BitmapCache layer)
space and compare them against accumulated dirty rects. Matching backdrop visuals are added
into dirty list wholesale (O(n²) in the worst case, sadly).

For retained visuals we track their pixel-aligned AABB rect in render target space and force a full
invalidation if we detect it being changed (pre-update pass recomputes their combined transform by
doing an upwards walk, size change is detected separately during the normal update pass).

For partial invalidation we force-include those visuals into the update walk if we've accumulated
any dirty rects up to the current point (this is done by tracking the number of backdrop visuals
in a subtree). When we get to retained backdrop visual, we check accumulated dirty rects agaist
its AABB rect and apply any expansions needed for sampling radius on intersections.

Due to some quirks of our existing system, we can't have dirty rects from some child list changes (remove/reorder)
to be available for backdrop visual when we encounter it (remove/reorder is applied for children in PostSubgraph),
which sort of broken scenarios like "visual above backdrop moved below it and now needs
to be included into the capture". So if children list changes for a visual that has retained backdrops
in its subtree, we now force-invalidate the entire subtree bounds when processing such visual.
While being somewhat non-optimal it's required for correctness.
We may later revisit this, but it will require more invasive changes in more places.

When we later do a volatile pass mentioned above, we collect visuals in painting order, so dirty rects
enforced by volatile visuals will only affect retained visuals above them.

This more or less summarizes the algorithm.

For rendering, on the backend side we have 2 separate rendering methods:

1. native skia one with SaveLayerRec, used for volatile backdrops
2. specialized one with opaque cache handle and dirty rects to update. I've opted out from using IBitmapImpl or ILayerImpl so we could have some meaningful optimization there on the backend side later (e. g. manually create a texture and do glBlitFramebuffer that's not exposed properly via Skia(Sharp)'s API).

[avaloniaui-bot]

You can test this PR using the following package version. 12.2.999-cibuild0067613-alpha. (feed url: https://nuget-feed-all.avaloniaui.net/v3/index.json) [PRBUILDID]

[zrt2399]

Is this behavior expected?

When a BoxShadow is applied to the same Border, the backdrop effect region expands to include the shadow bounds. Without the shadow, the backdrop effect is limited to the Border's width and height.

Should the backdrop effect be limited to the visual's own bounds, while allowing the shadow to render outside them?

<!-- Frosted overlay: BackdropEffect blurs everything drawn behind this Border. -->
<Border Width="360" Height="150" BoxShadow="0 0 20 0 #60000000"
        HorizontalAlignment="Center" VerticalAlignment="Center"
        CornerRadius="16" Background="#30FFFFFF"
        BorderBrush="#50FFFFFF" BorderThickness="1"
        BackdropEffectCache="Retained">
    <Border.BackdropEffect>
        <BlurEffect Radius="18" />
    </Border.BackdropEffect>
    <TextBlock Text="Backdrop blur" FontSize="20" Foreground="White"
               HorizontalAlignment="Center" VerticalAlignment="Center" />
</Border>

[kekekeks]

The rule of the thumb for backdrop is to work like WPF's Effect/OpacityMask bounds - use full subtree bounds as its size area. Box shadows currently affect subtree bounds elsewhere, so it gets applied to backdrop too.

[kekekeks]

We might want to have a separate property to constraint this to layout bounds like we do with ClipToBounds though. Since I do see the default one being inconvenient in cases like box-shadow.

[avaloniaui-bot]

You can test this PR using the following package version. 12.2.999-cibuild0067677-alpha. (feed url: https://nuget-feed-all.avaloniaui.net/v3/index.json) [PRBUILDID]

[Gillibald]

Can you clarify why the backdrop region is derived from the child subtree’s AABB? There is no other ui framework that does that. All are using the visual’s own layout box.

[kekekeks]

All are using the visual’s own layout box.

I'd like to remind that we are generally following WPF's model when layout box doesn't exist as a concept on compositor level.

As of AABB vs untransformed box - backdrop is snapshotting and sampling the AABB box.

[Gillibald]

Here are some scenarios I have tested against my solution and this PR. I can go into more detail after my vacation. In general, this shows why using sub-tree bounds (effect padding) isn't ideal.

[avaloniaui-bot]

You can test this PR using the following package version. 12.2.999-cibuild0068416-alpha. (feed url: https://nuget-feed-all.avaloniaui.net/v3/index.json) [PRBUILDID]

[Gillibald]

Can we adapt some parts of my effects branch so that backdrop rendering is built around Push/PopLayer and the caching strategy is chosen by the backend without exposing control to the user? Cache control is not needed as part of the user-facing API.

https://github.com/Gillibald/Avalonia/blob/pr2/effects/src/Skia/Avalonia.Skia/DrawingContextImpl.Layers.cs#L10

[kekekeks]

the caching strategy is chosen by the backend without exposing control to the user

User knows how often they expect the content behind backdrop to change (e. g. overlay on top of the video playback is expected to be changed every frame, blurred background for modal dialog is expected to not change for the lifetime of the dialog).

1. backend does not have this information and needs to speculate
2. retained cache is not free memory-wise, especially with memory-constrained devices. You can argue that backend can track memory budget, but that budget is shared between multiple apps, we are not a browser that can eat half of the system resources for rendering single active tab
3. skipping cache is not free perf-wise

Those are the knobs that have meaningful effect for the user code.

[Gillibald]

If we move to the PushLayer(LayerOptions shape), we could have both. Then we only need to introduce an enum { Auto, Retained, Volatile }. Volatile and Retained as described above; Auto means policy decides.

I also realised we cache different things. You are caching the input; I am caching the output. Each solution has its advantage/disatvange. By using the PushLayer shape, we do not have to introduce PushBackdropLayer and DrawRetainedBackdropEffect.

[maxkatz6]

User knows how often they expect the content behind backdrop to change

I am not sure if this is true.
Especially if this user is a developer of a control, theme or a framework, who don't know in what context's their controls will be used.

Some of the Fluent theme generic controls in UWP had acrylic backgrounds (via AcrylicBrush set on Background, a backdrop brush integrated into a compositor). Or let's say somebody implements Apple's Liquid Glass for Avalonia, with generic liquid buttons. All of these controls can be rendered on top of MediaPlayer.

It would make sense if controls like MediaPlayer could define a backdrop caching scope for their subtrees, rather than expecting every subtree element to be aware of their context.
BackdropEffectCache can be an inherited property in a way to achieve that.

[MrJul]

API diff

Avalonia.Base (net10.0, net8.0)

  namespace Avalonia
  {
      public class Visual : Avalonia.StyledElement
      {
+         public static readonly Avalonia.StyledProperty<Avalonia.Media.BackdropEffectBounds> BackdropEffectBoundsProperty;
+         public static readonly Avalonia.StyledProperty<Avalonia.Media.BackdropEffectCacheMode?> BackdropEffectCacheProperty;
+         public static readonly Avalonia.StyledProperty<Avalonia.Media.IEffect?> BackdropEffectProperty;
+         public Avalonia.Media.IEffect? BackdropEffect { get; set; }
+         public Avalonia.Media.BackdropEffectBounds? BackdropEffectBounds { get; set; }
+         public Avalonia.Media.BackdropEffectCacheMode? BackdropEffectCache { get; set; }
      }
  }
  namespace Avalonia.Media
  {
+     public enum BackdropEffectBounds
+     {
+         Layout = 0,
+         Subtree = 1
+     }
+     public abstract class BackdropEffectCacheMode
+     {
+         public static Avalonia.Media.BackdropEffectCacheMode? Parse(string s);
+     }
+     public sealed class RetainedBackdropEffectCacheMode : Avalonia.Media.BackdropEffectCacheMode
+     {
+         public RetainedBackdropEffectCacheMode();
+     }
+     public sealed class VolatileBackdropEffectCacheMode : Avalonia.Media.BackdropEffectCacheMode
+     {
+         public VolatileBackdropEffectCacheMode();
+     }
  }

[MrJul]

Notes from the API review meeting:

• Rename BackdropEffectBounds to BackdropEffectBoundsSource, for both the enum and the property.
• Rename RetainedBackdropEffectCacheMode to RetainedBackdropEffectCache (matching CacheMode/BitmapCache).
• Rename VolatileBackdropEffectCacheMode to VolatileBackdropEffectCache (matching CacheMode/BitmapCache).

[avaloniaui-bot]

You can test this PR using the following package version. 12.2.999-cibuild0069373-alpha. (feed url: https://nuget-feed-all.avaloniaui.net/v3/index.json) [PRBUILDID]