Skip to content

[Feature] DrawingRecording - #22061

Open
Gillibald wants to merge 6 commits into
AvaloniaUI:mainfrom
Gillibald:pr1/drawing-recording
Open

[Feature] DrawingRecording#22061
Gillibald wants to merge 6 commits into
AvaloniaUI:mainfrom
Gillibald:pr1/drawing-recording

Conversation

@Gillibald

Copy link
Copy Markdown
Contributor

What does the pull request do?

Adds DrawingRecording: a retained draw list that is recorded once and replayed cheaply, optionally bound to a compositor so it animates without being re-recorded, and optionally hosted as a composition visual so it animates entirely on the render thread.

Avalonia has no cheap retained-drawing primitive today. DrawingGroup re-walks a managed object tree on every draw, and a RenderTargetBitmap is rasterized at a fixed resolution and stops being vector content. Anything that draws the same vector content repeatedly - an icon set, a symbol reused many times in one document, a vector image drawn at several sizes, an animated vector document - pays full re-emission on every frame, and there is no way to hand a recorded drawing to the compositor at all.

Alongside the recording primitive:

  • DrawingRecordingBrush, a TileBrush that paints with an existing recording instead of re-recording a Drawing per compositor, the way DrawingBrush does.
  • Composition brushes (CompositionSolidColorBrush, the composition gradient brushes) work inside a compositor-bound recording, so a recorded fill can be animated from the render thread.
  • CompositionRecordingVisual, so a recording can be attached to a control as a child visual and animated server-side.

What drives it is SVG support: a document compiles once into a recording, replays shared sub-recordings at every use site, and animates paint through mutable brushes and structure through composition visuals. COLRv1 color-font rendering wants the same shape, a glyph's paint graph recorded once and replayed at every instance.

None of the API is specific to either. The recording primitive serves icon systems, chart and diagram renderers, and any control that draws the same vector content repeatedly.

What is the updated/expected behavior with this PR?

Record once and replay, with no re-emission per draw:

// Immutable: brushes and pens are snapshotted at record time.
var recording = DrawingRecording.Create(ctx =>
    ctx.DrawRectangle(Brushes.Crimson, null, new Rect(20, 20, 60, 60)));

context.DrawRecording(recording);                     // replay
context.DrawRecording(recording, Matrix.CreateScale(2, 2));   // replay transformed

Bind it to a compositor and mutable brushes animate without re-recording:

var brush = new SolidColorBrush(Colors.Red);
var recording = DrawingRecording.Create(compositor, ctx =>
    ctx.DrawEllipse(brush, null, bounds));

brush.Color = Colors.Blue;   // tracked; the recording is not rebuilt

Paint an area with a recording, tiled by the usual TileBrush properties:

var brush = new DrawingRecordingBrush(recording)
{
    TileMode = TileMode.FlipXY,
    DestinationRect = new RelativeRect(0, 0, 0.25, 0.25, RelativeUnit.Relative)
};

Host it as a composition visual and it animates on the render thread, with no per-frame UI-thread work:

var visual = compositor.CreateRecordingVisual();
visual.Recording = recording;                          // must share the compositor
ElementComposition.SetElementChildVisual(control, visual);

var spin = compositor.CreateScalarKeyFrameAnimation();
spin.InsertKeyFrame(0f, 0f);
spin.InsertKeyFrame(1f, (float)(Math.PI * 2));
spin.IterationBehavior = AnimationIterationBehavior.Forever;
visual.StartAnimation("RotationAngle", spin);

How was the solution implemented (if it's not obvious)?

drawingrecording-objectgraph

Recording. DrawingRecording.Create(...) runs the caller's callback against RenderDataDrawingContext, a DrawingContext that records draw operations as nodes instead of rasterizing them. The immutable overload snapshots brushes, pens and effects at record time and rejects content it cannot snapshot; the compositor-bound overload registers mutable resources as composition resources instead, so later mutation is change-tracked. Nested recordings are referenced and ref-counted rather than copied, and DrawingRecordingOwnership lets a parent recording take over disposal of a child.

Replay and the client/server bounds split. Each node exposes both a client Bounds (UI thread, may read live mutable resources) and a ServerBounds (render thread, immutable data and server shadows only). That split is what lets an animated pen thickness keep the server-side bounds current without re-recording anything.

Composition brushes need no new invalidation mechanism. A recording's server render data already observes every server resource in its stream, and a recording visual already observes that render data, so a composition brush change repaints the visuals showing the recording exactly like a tracked mutable Media brush does. Immutable recordings keep rejecting composition brushes, since no immutable form of one exists; the exception now points at the compositor-bound Create overload.

drawingrecording-pipeline

Composition hosting. Compositor.CreateRecordingVisual() produces a CompositionRecordingVisual, which renders one compositor-bound recording behind its children. It serializes a reference to the recording's server render data rather than the draw items, and the server visual observes that render data instead of owning it. Combined with the animatable visual properties this moves retained content entirely onto the render thread. The internal Image-side hosting that lets an IImage render as a composition visual subtree ships as its own follow-up PR.

Checklist

Breaking changes

One, needing a core-team call given the API freeze:

  1. CompositionVisual.ClipToBounds now defaults to false (was true), matching Visual.ClipToBounds and UWP composition semantics. The renderer sets it explicitly on element visuals every sync, so element visuals are unaffected. The old default combined with the (0,0) Size default gave every directly created visual an empty own-clip rect, which nulled its subtree bounds and made the render pass cull it and its children - a visual attached with ElementComposition.SetElementChildVisual never rendered unless Size was set explicitly.

Two smaller behavioral changes, both turning a failure into the useful result:

  • BrushExtensions.ToImmutable handles scene brushes. Brush implements neither IMutableBrush nor IImmutableBrush, so the old body's (IImmutableBrush)brush cast threw InvalidCastException for VisualBrush and DrawingBrush. It now returns an immutable snapshot of the scene brush's current content, with tile-brush properties captured at call time, or Brushes.Transparent when the brush has no content.
  • ElementComposition.SetElementChildVisual un-parents the previous child eagerly. The renderer sync that would reconcile children may never run again for that element - clearing the child visual from OnDetachedFromVisualTree runs before DetachFromCompositor discards the element's composition visual - and the stale parenting made any later attach throw.

No backend has to change: nothing is added to IDrawingContextImpl, and replay goes through the existing operations. (Backend interfaces are [Unstable] and not stable API, so this is a note about the work being self-contained, not about avoiding a break.)

Obsoletions / Deprecations

None.

Fixed issues

Avalonia has no cheap retained-drawing primitive. DrawingGroup re-walks a
managed object tree on every draw, and a RenderTargetBitmap is rasterized
at a fixed resolution and stops being vector content. Anything that draws
the same vector content repeatedly - an icon set, a symbol reused many
times in one document, a vector image drawn at several sizes - pays full
re-emission every frame, and a recorded drawing cannot be handed to the
compositor at all.

- DrawingRecording records draw operations once and replays them through
  DrawingContext.DrawRecording. The immutable form snapshots its brushes,
  pens and effects at record time; the compositor-bound form registers
  mutable resources instead, so animating a brush needs no re-recording.
- Nested recordings are referenced and ref-counted rather than copied, and
  DrawingRecordingOwnership lets a parent take over a child's disposal.
- Render data nodes gained a client/server bounds split, so an animated pen
  keeps server-side bounds current without re-recording.
- DrawingRecordingBrush paints a pre-recorded drawing as a tile brush.
- CompositionRecordingVisual (via Compositor.CreateRecordingVisual) renders
  a compositor-bound recording behind its children, so retained content
  animates entirely on the render thread.
- ICompositionImage lets an IImage opt into that hosting; CompositionImageHost
  and Image wire it up and fall back to IImage.Draw for other sources.

CompositionVisual.ClipToBounds now defaults to false, matching
Visual.ClipToBounds and UWP composition semantics. The previous true default
combined with the (0,0) Size default gave every directly created visual an
empty own-clip rect, which nulled its subtree bounds and made the render pass
cull it and its children, so a visual attached with SetElementChildVisual
never rendered unless Size was set explicitly.
The invalidation chain needs no new mechanism: the recording server
render data observes every server resource in its stream, and the
recording visual already observes the render data, so a composition
brush change repaints the visuals showing the recording just like a
tracked mutable Media brush. The suite pins that chain end to end:
client-driven and render-thread-driven changes, an animated gradient
stop, a nested recording, a brush on a pen, and DrawingRecordingBrush
content, plus bounds stability and disposal ordering.

Immutable recordings keep rejecting composition brushes; the message
now points at the compositor-bound Create overload instead of
suggesting an immutable brush that cannot exist for one.
The centre disc is filled with a CompositionLinearGradientBrush whose
stop colors cross-fade via render-thread keyframe animations while the
ring spins: the recording is captured once and repaints through the
brush observer chain, with no UI-thread work and no re-recording.
Composition objects have no public dispose, so the brush and stops are
created once per compositor and reused across attach cycles, stopping
their animations while detached.
@Gillibald Gillibald added feature enhancement area-composition api-needs-review The PR adds new public APIs that should be reviewed. labels Aug 25, 2026
Conflict resolution: CompositionRenderData needs both sides' usings
(our Render(IDrawingContextImpl) plus upstream's HitTest(Geometry)
from the geometry hit-testing feature), and OnDrawRecording in the
hit-test visitor is adapted to the reworked visitor state, where the
unconditional Current point became the nullable CurrentPoint.
Upstream's geometry hit testing walks the same visitor as point hit
testing, but a recording op only answered for points, and the visual
override fell back to the base's blanket Intersects.

- OnDrawRecording recurses into the child recording for geometry
  queries, transforming the query by the op's inverse matrix; the
  child's Empty does not stop the walk, since later ops in the outer
  stream can still intersect
- DrawingRecording.HitTest(Geometry) mirrors the point overload
  (existing HitTest(default) call sites disambiguate to Point)
- CompositionRecordingVisual answers geometry queries from its
  recording instead of the base default
- CompositorBoundDrawingRecordingTests constructs its services in the
  ctor: a field initializer runs before ScopedTestBase replaces the
  locator, discarding the registrations geometries resolve against
Release turns warnings into errors, and CI compiles projects the local
Debug gates skipped:

- DrawingRecordingBenchmarks used PushLayer, which does not exist at
  this point in the stack; the layered replay benchmark now records an
  opacity layer via PushOpacity
- the server-bounds thread-affinity test blocked on a task (xUnit1031);
  it now evaluates bounds on a dedicated joined thread, which also
  keeps the test body on the dispatcher thread for the trailing
  disposals - an awaited Task.Run resumes on a pool thread and trips
  VerifyAccess there
@avaloniaui-bot

Copy link
Copy Markdown

You can test this PR using the following package version. 12.2.999-cibuild0068786-alpha. (feed url: https://nuget-feed-all.avaloniaui.net/v3/index.json) [PRBUILDID]

@robloo

robloo commented Aug 26, 2026

Copy link
Copy Markdown
Contributor

Not to get hung up on the naming right away but DrawingRecording sounds like something completely different. A DrawingRecording sounds like it's recording events from the pointer to recreate an image drawn by the user. Not encapsulating the commands necessary to re-draw geometry and bitmaps. I understand they are conceptually similar but the nuance is important in naming here.

Why not something like:

  1. DrawingTemplate or
  2. DrawingStream ... (StreamDrawing would match StreamGeometry)
  3. DrawingCommands even DrawingInstructions

@Gillibald

Copy link
Copy Markdown
Contributor Author

Not to get hung up on the naming right away but DrawingRecording sounds like something completely different. A DrawingRecording sounds like it's recording events from the pointer to recreate an image drawn by the user. Not encapsulating the commands necessary to re-draw geometry and bitmaps. I understand they are conceptually similar but the nuance is important in naming here.

Why not something like:

  1. DrawingTemplate or
  2. DrawingStream ... (StreamDrawing would match StreamGeometry)
  3. DrawingCommands even DrawingInstructions

We record a drawing, so I chose that name. Look at the family of classes that are related to that type: DrawingRecordingBrush, CompositionRecordingVisual, CreateRecordingVisual, DrawRecording, DrawingRecordingOwnership. Would your suggestion fit here? The term Drawing is already part of the framework.

@robloo

robloo commented Aug 27, 2026

Copy link
Copy Markdown
Contributor

I'll compare names when I have a chance. But it looks like this concept already exists in DirectX 11+ as well:

DrawingCommands would be the closest .NET naming equivalent I think. This is also assuming a "drawing" encompasses vector geometries as well as bitmaps together. Otherwise "drawing" is overly general.

Anyway, I'll try to look at this more soon.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

api-needs-review The PR adds new public APIs that should be reviewed. area-composition enhancement feature

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants