VirtualizingStackPanel - content virtualization and advanced scroll position calculation supporting varying item sizes

[gentledepp]

Note for anyone who read an earlier version of this description. It has been rewritten from
scratch. The first revision advertised a set of tuning heuristics — a layout-cycle breaker, extent
oscillation detection with a frozen extent, boundary clamping, estimate caching, dampened extent
compensation, a sub-pixel resize threshold, warmup head sampling. Every one of those is gone,
along with the constants it carried. They were symptom-level dampers for a single root cause; that
root cause is now fixed directly, and the dampers turned out to be either dead or replaceable by
constant-free arithmetic. Nothing below describes code that is not in the diff.

What does the pull request do?

Two separable things, plus one correctness fix.

1. Makes VirtualizingStackPanel's extent estimate independent of which items happen to be
realized. This is the root-cause fix, and it is the part of the PR I would most like reviewed.

2. Adds opt-in container-level virtualization. A data template can implement
IVirtualizingDataTemplate — in XAML by setting EnableVirtualization="True", in code by giving a
FuncDataTemplate a RecycleKeySelector — to have its container
and its child recycled as a single unit, instead of the child being destroyed and rebuilt on every
scroll. Nothing changes for anyone who does not ask for it. Over a heterogeneous list of complex rows
this is ~3.4× faster scrolling with 83% less allocated and ~20× lower frame-to-frame variance, at
identical container prepare counts — see Performance.

3. Fixes a wrong-index render on collection edits coalesced into a Reset. Independent of both
of the above; see "Tier A correctness fix" below.

Important

Opting a template in changes what view lifecycle events mean, and this is the one way the feature
can break a working list. Container-level virtualization exists precisely because it stops tearing
the child down: a recycled container is hidden (IsVisible = false) and kept in Panel.Children
with the template's control tree still attached to it. Nothing leaves the visual tree.

So for an opted-in template, Loaded / Unloaded and
AttachedToVisualTree / DetachedFromVisualTree fire once, for the first item those controls
ever displayed. They do not fire again as the container is reused for other items, and the unload
side does not fire when an item scrolls out of view. Any control that initialises per item on load
or releases state on unload will misbehave: it initialises once against the wrong item and never
cleans up. Media players, map and chart controls, and anything that subscribes to a service in
OnAttachedToVisualTree are the usual cases.

The way out is to move per-item work to DataContextChanged, or to leave that template opted out,
in which case recycling behaves exactly as it does today. What the framework should do about this
is an open question for you, not a settled decision by me (see Checklist), and it is the reason
this is called out here rather than only under Breaking changes.

The headline: every fork-added tuning constant is gone

Value	Location	Why it is not a magic number
25	_lastEstimatedElementSizeU init	Stock Avalonia, not added here. Only consulted before any item has been measured.
3	DefaultWarmupPoolSizePerKey	Pool depth used only when the template does not specify MinPoolSizePerKey. Affects first-scroll performance only — never layout or correctness — and only when warmup is explicitly enabled.

That is the complete inventory. No threshold, no smoothing factor, no reversal counter, no pixel
tolerance beyond Avalonia's own LayoutHelper.LayoutEpsilon. If a value in this diff looks like a
tuning constant, it is a bug — please flag it.

What is the current behavior?

Stock VirtualizingStackPanel assumes item containers measure deterministically: the same item
measures to the same size on every pass. Real templates violate that routinely:

• async image or content loading (a container is 84px as a placeholder, then 292px once loaded),
• text wrapping and deferred bindings that change desired size across passes,
• ObservableAsPropertyHelper-backed properties that propagate a pass late, so pass 1 sees the old
  value and pass 2 the new.

EstimateElementSizeU estimates un-realized items from an average over the currently-realized
set. That set's membership changes on every scroll pass, so the estimate swings, so the reported
extent swings, so the ScrollViewer re-measures the panel, so different items realize — and around
again. Symptoms: scroll drift, a scrollbar whose length depends on where you scrolled from, and
repeated layout passes.

Separately, on the content side: only containers (ListBoxItem, ContentPresenter) are recycled.
The content built by the data template — for a form or card template, a subtree of 10–50 controls —
is destroyed and rebuilt every time an item scrolls into view.

What is the updated/expected behavior with this PR?

1. A persistent per-item size record replaces the realized-window average

_measuredSizes (Dictionary<int, double>: last-measured size along the scrolling axis, per item
index) plus a running sum:

• EstimateElementSizeU upserts every currently-realized, measure-valid element's size, then returns
  the mean over all recorded sizes — a function of everything ever measured, not of the current
  window.
• CacheBasedExtentU(itemCount) returns knownSum + unknownCount * mean. Once every item has been
  measured, unknownCount == 0 and the extent is exactly the true total.

Why this is upstreamable where the dampers were not:

• Provable no-op for uniform items. Every recorded size is equal, so the mean equals that size —
  identical to stock's realized average.
• Distribution-agnostic. No constant assumes a height distribution. Verified against uniform,
  bimodal, extreme outliers (20px rows with occasional 2000px), a monotonic ramp, async-grow
  (84→292) and cross-region shapes.
• Orientation-agnostic. The record stores a size along the scrolling axis and never learns which
  axis that is. The tests run as theories over Orientation, and two deliberately-injected axis-only
  defects turned only horizontal cases red.
• Reproducible. Revisiting an offset reports the same extent. Cross-region estimate spread went
  26000px → 1567px when this landed.
• Realizing more items only sharpens the estimate; it can never oscillate.

Record lifecycle: inserts and removes remap in place (an append — the infinite-scroll case —
moves nothing and allocates nothing); Move and a non-preserving Reset clear it, because the
index→item mapping is no longer trustworthy.

What it costs, because this is the first thing you will want to profile. One entry per item ever
measured, never trimmed. Measured against the merge-base, with the item collection itself
excluded from the delta:

Items browsed	Stock retains	This PR retains	The record
1,000	17,952 B	71,520 B	53,568 B (53.6 B/item)
100,000	145,856 B	4,520,728 B	4,374,872 B (43.7 B/item)

So ~4.2 MiB for a 100k-item list, held for as long as the panel is. It is not negotiable downwards —
trimming to the realized window is exactly the window-dependence the record exists to remove, and
the tests say so.

What it does not cost is per-pass time: the running sum is maintained incrementally at the single
upsert site (with Neumaier compensation), never by sweeping the record, so a measure pass stays
O(realized window) as in stock however large the record has grown. That is measured too, not just
argued — the same 40-pass scroll burst at the head of the collection, run once with the record empty
and once with an entry for every one of 100k items, differs by 0.053 ms: inside the run's own
standard deviation, and smaller than the same gap at 10k entries. A per-pass sweep would have grown
by 10×.

Staleness is safe by construction, and please don't "fix" it. While an item is out of view its
entry can go stale if its view model changes — no container exists to re-measure it. The record is
consulted only for items that are not currently realized; a visible item is always sized by its
live measure. On re-realization the container is re-measured and the entry is overwritten, so the
extent self-heals. Worst case is a transient scrollbar-length error for off-screen mutations,
corrected on scroll-in — inherent to all virtualization, and strictly better than stock, which
remembers nothing.

2. Tier A correctness fix: Reset preservation only when every element still validates

This fork preserves realized containers across a Reset for scroll stability (the infinite-scroll
append case). The gate used to be a bare majority (preservedCount > realizedCount / 2). A mid-list
insert or remove coalesced into a single Reset — which DynamicData's Bind reset-threshold
does routinely — leaves the prefix matching but shifts everything after the edit point. A bare
majority therefore preserved the whole stale mapping: shifted items pinned to the wrong
containers, children rendered under the wrong headline. Visible corruption.

Now preservation requires preservedCount == realizedCount; any partial match falls through to the
full-reset path. Cited tests: Reset_With_MidList_Insert_Realizes_Shifted_Items_At_Correct_Index
and the 28-case Collection_Edit_Keeps_Every_Container_On_Its_Own_Item matrix (7 edit kinds × 4
positions — the edit has to land past the middle of the realized window to reproduce, which is why
the matrix and not a single test).

Related: VirtualizingPanel.Refresh() is now virtual. An ItemTemplate / ItemContainerTheme /
DisplayMemberBinding change reaches the panel as a synthetic Reset in which every element
still matches its item — so preservation always kicked in, PrepareItemContainer was never called,
and those three properties had no effect on already-realized containers of a virtualized
ItemsControl. VirtualizingStackPanel overrides Refresh() to recycle first.

3. Constant-free anchor compensation

The anchor sits at StartU + Σ sizes before it. If that sum grew by preDelta, StartU must shrink
by the same amount or the anchor visually jumps. That is all ValidateStartU does now:
StartU -= preDelta, with MathUtilities.AreClose(..., LayoutHelper.LayoutEpsilon) — Avalonia's own
layout-significance epsilon — absorbing float noise and nothing else. Sizes are recorded and
re-checked through one accessor (GetElementSizeU), so the two can never disagree and read as a
resize on every pass. New signature:

bool ValidateStartU(int anchorIndex, Func<Control, int, double> getSizeU, out double preDelta)

4. Container-level virtualization, opt-in per template

Core principle: container + child = one reusable unit. On the way out, content clearing is
skipped and the container is pooled under its recycle key with the child still attached. On the way
back in, PrepareContainerForItemOverride sets Content, and ContentPresenter.CreateChild passes
the attached oldChild to the template's Build, which returns it unchanged — so there is no visual
tree mutation at all, only a DataContext change and the binding updates that follow.

Recycle-key selection in ItemsControl.NeedsContainer<T>:

1. item is T                                → recycleKey = null   (item is its own container)
2. IVirtualizingDataTemplate.GetKey(item)   → that key            (the only opt-in)
3. otherwise                                → DefaultRecycleKey   (stock behaviour)

Step 1 must come first, or items that are their own containers get wrapped — that ordering was a
real bug fix. Step 2 applies only when the kill switch is on and the panel is a
VirtualizingStackPanel, and it resolves the template through GetEffectiveItemTemplate(), the same
path everything else keys off, so keying and pool-capping cannot disagree.

public interface IVirtualizingDataTemplate : IRecyclingDataTemplate
{
    object? GetKey(object? data);   // null = no pooling for this data
    int MaxPoolSizePerKey { get; }  // container-pool cap
    int MinPoolSizePerKey { get; }  // warmup target; only consulted when warmup is enabled
}

It extends IRecyclingDataTemplate, so Build(data, existing) comes from the base interface. XAML
DataTemplate implements it with EnableVirtualization (default false); GetKey returns null
unless that is set.

<DataTemplate DataType="local:Person" EnableVirtualization="True" MaxPoolSizePerKey="10">
    <Border>
        <StackPanel>
            <TextBlock Text="{Binding Name}" />
            <TextBlock Text="{Binding Email}" />
        </StackPanel>
    </Border>
</DataTemplate>

A <DataTemplate DataType="local:Person"> that does not opt in behaves exactly as in stock,
DefaultRecycleKey and all. MaxPoolSizePerKey is honoured only for keys an
IVirtualizingDataTemplate handed out; DefaultRecycleKey pooling stays uncapped, as in stock.

From code, FuncDataTemplate opts in through a key selector:

new FuncDataTemplate<Row>((_, _) => BuildRow())
{
    RecycleKeySelector = d => ((Row)d!).Kind,
};

Null by default, so nothing changes for the code-defined templates that already exist — including
FuncDataTemplate.Default and the framework's own. A selector rather than a bool because the key
has to identify the shape the build function produced, not the data's type: a template that
branches on a property builds different subtrees for one CLR type, which is what a heterogeneous
list is made of, and which DataType-based keying cannot express. Where the build function does not
branch, d => d?.GetType() is the natural choice. Returning null for a particular item opts that
item back out.

ContainerVirtualization.IsEnabled (default true) is a kill switch, not the opt-in:
setting it false forces every ItemsControl back to stock recycling, which is how you establish
whether a layout problem comes from virtualization at all. It used to be called
ContentVirtualizationDiagnostics; a behaviour switch does not belong in a class named for
diagnostics, so it is now named for the feature it switches. Where that switch should live at all is
still on the API-review list below.

Supporting machinery in ItemsControl / ContentPresenter:

• SetIfUnsetOrDifferent — unlike SetIfUnset, forces the update when a recycled container
  already holds a different value. Without it a reused container keeps the previous item.
• ContentPresenter.BeginBatchUpdate / EndBatchUpdate — Content and ContentTemplate must
  land together, or UpdateChild runs once with a mismatched pair and rebuilds the child for
  nothing. The depth counter makes batches nest (only the outermost End publishes) and an
  unbalanced End a no-op; End also runs the UpdatePseudoClasses / InvalidateMeasure that
  ContentChanged skipped for the whole batch.
• Template resolution is stock. PrepareContainerForItemOverride sets ContentTemplate from
  ItemTemplate / DisplayMemberBinding and from nothing else; a template living in a
  DataTemplates collection is resolved by the ContentPresenter itself, as in stock, so the
  ItemsControl and the presenter can never disagree about which template an item uses.

5. RetainMatchingContainers (disjunct-scroll reuse)

On a viewport jump everything is recycled. Before that, RetainMatchingContainers pulls out
realized containers whose DataContext matches an item in the estimated new viewport;
GetOrCreateElement gives those a lightweight ItemContainerIndexChanged instead of a full
PrepareItemContainer. RealizedStackElements.NullifyElement removes an element from the realized
list without recycling it.

Correctness-safe by construction: keyed on the item reference, so a container is only ever reused for
the same item it already held. Because it skips PrepareItemContainer it is the one path that can
defeat the staleness self-heal above, hence Retained_Container_Reuse_Remeasures_Changed_Item.

Now measured: ~10% fewer container prepares on backwards scrolling and paging (132 → 120 over a
40-step wheel-up; 83 → 75 over 10 page-ups), and over complex rows ~7% fewer subtree rebuilds with
it. Forward scrolling and scrollbar jumps are unchanged. Real, but small, and below the noise floor of
the timing runs — so if the scope split below is wanted, this is the first thing I would move to its
own PR. It also carries the most machinery per unit of benefit (NullifyElement, _retainedForReuse,
RecycleUnusedRetainedContainers, and the scroll-anchor bookkeeping).

6. Opt-in warmup, default off

EnableWarmup (default false) pre-creates and pre-measures containers on a background dispatcher
tick, so the first scroll does not pay for container construction.

<ItemsPanelTemplate>
    <VirtualizingStackPanel EnableWarmup="True" />
</ItemsPanelTemplate>

The pool grows off the template keys the panel has actually needed a container for, topping up
when a new key appears and forgetting keys the collection no longer contains. There is no head
sampling: sampling the first N items assumes the head represents the whole collection's key
distribution, which is false for any grouped or sorted list. Warmup is skipped — not cancelled — when
its dispatcher tick lands on a panel that has left the visual tree, so a navigated-back-to page warms
up on re-attach.

7. The zero-viewport guard

OnEffectiveViewportChanged returns immediately when the incoming effective viewport is empty,
without touching _viewport, the extent, or invalidating measure. This reads as a defensive nicety;
it is load-bearing. On-device trace: a user scrolls into a list, launches the camera, and returns to
find the scroll position jumped. The cause is not the photo resizing — it is the window viewport
collapsing to 0×0 while the camera activity is in front:

Seq	Event
#01478	Baseline: vpY=983.8 startU=435.9 realized=[2..9] anchor=item3@821.7
#01499	Camera launches → effVp=0,983.8,0,0 (0×0) → needsMeasure
#01504	Empty-viewport measure → treated as disjunct → recycles everything → realized=[0..0] startU=0 (anchor lost)
#01532	Return: viewport correctly restored to vpY=983.8
#01535	But panel state is item0/startU=0 → anchorIdx=-1 (unrecoverable)
#01544	ScrollViewer clamps the out-of-range offset → effVp jumps 983.8 → 123.7

Any tab switch, minimise or foreign activity produces the same 0×0 viewport; this is not
camera-specific. Covered by Collapsing_Viewport_To_Empty_And_Restoring_Preserves_Scroll_Position
(red/green verified: with the guard disabled, FirstRealizedIndex collapses 20 → 0 across the round
trip).

Performance

Benchmarks are in tests/Avalonia.Benchmarks/Controls/ — BenchmarkDotNet for timings,
dotnet Avalonia.Benchmarks.dll --virtualization-report for the deterministic container counts. The
baseline is a worktree at the merge-base, built and run the same way, rather than a toggle inside one
build.

Container-level virtualization (item 4) is where the win is

5,000 heterogeneous rows — four row kinds of 10–20 visuals each with bound text, i.e. a form or feed
list. Plain is a template that does not opt in, which puts it on the stock path
(DefaultRecycleKey, content cleared on recycle, subtree rebuilt); Virtualized opts in. MediumRun,
i7-10750H, two independent runs shown so the spread is visible:

Scenario	Opt-in off	Opt-in on
Wheel scroll, 80 steps	24.08 / 19.80 ms	6.48 / 6.35 ms	~3.4× faster
20 scrollbar jumps	24.63 / 24.75 ms	9.92 / 10.77 ms	~2.4× faster
Allocated, scroll	7.85 MB	1.36 MB	−83%
Allocated, jumps	11.68 MB	2.00 MB	−83%
Std. dev., scroll	±6.79 / ±3.75 ms	±0.24 / ±0.19 ms	~20× steadier

That last row matters as much as the mean for a scrolling UI: jank is variance, not average cost.

The deterministic counts say why. Container prepare counts are identical in both arms — nothing
is being skipped or realized lazily; the entire difference is the child subtree surviving recycling:

Scenario	Prepares	Child subtree builds	Visuals constructed
Wheel, 80 steps	80 → 80	160 → 5	1,562 → 69
Paging, 20 steps	51 → 51	102 → 0	986 → 0
20 jumps	117 → 117	236 → 1	2,362 → 9

Once the per-key pools have filled, paging rebuilds nothing at all.

Two caveats stated up front rather than left for review to find:

• This is an in-branch A/B, and has to be — stock has no equivalent feature to compare against.
  It is a fair one because a non-opted-in template is put back on stock's exact path, and that is
  checked rather than assumed: the Plain arm is also run at the merge-base, where it comes out
  indistinguishable from this branch's.
• Row cost is the independent variable, not harness furniture. Over a one-Canvas row template
  the same benchmark shows no benefit whatsoever — there is nothing to preserve when building the
  child costs a single allocation. These figures describe templates with real content, which is the
  case the feature exists for. Desktop x64 is a lower bound: the avoided work is subtree
  construction, binding setup and text layout, all of which cost proportionally more on mobile, where
  this pattern hurts most. No mobile numbers have been taken.

The panel changes (items 1–3, 5) on their own

Over the same complex rows, this branch vs. the merge-base with a non-opted-in template:
indistinguishable — 18.51 / 20.93 ms stock against 24.08 / 19.80 ms here, the ordering flipping
between runs, both arms GC-dominated at ~8 MB per operation. Over a trivial one-Canvas template the
branch is 16–34% faster and allocates 12–23% less across a 1k/100k × uniform/variable matrix, with
equal or lower container counts, which is the regime where panel bookkeeping is the whole cost.

Read that as the extent rework does not regress, and helps when templates are cheap. The
performance case for this PR is the section above it; the case for items 1–3 is correctness and the
constants inventory, not speed.

Accepted trades

Stating these explicitly rather than letting review find them:

• Templates that never settle now iterate to the LayoutManager's cap. Removing the layout-cycle
  breaker means a template reporting a different size on every measure drives repeated measure
  passes until Avalonia's own iteration cap, instead of being hard-capped at one pass per layout
  cycle by the panel. Deliberate: "one pass suffices" was an assumption, the cap also dropped
  legitimate work (a second resize inside one measure→arrange cycle never reached ValidateStartU),
  and the deferred re-measure lagged real size changes by a dispatcher tick — applying a size change
  at the next scroll position, producing the very jump it was meant to prevent. Templates that
  settle converge. A template that never settles is a defect in the template, and it behaves the same
  way in a plain StackPanel.
• View lifecycle events do not fire per item for opted-in templates — see Breaking changes.
• The size record is memory the panel keeps — figures above.
• Panel.Children retains invisible pooled containers — see Breaking changes.

How was the solution implemented (if it's not obvious)?

Most of the above is in the "updated behavior" section. What is worth adding is what was removed,
because several of these were in the first revision of this PR and a reviewer may go looking for
them:

Removed	Constants it carried	Why
EMA smoothing in EstimateElementSizeU	0.3 smoothing, >50% overlap gate, realized-range skip	The oscillation, made visible. Replaced by the size-record mean.
Extent-oscillation freezing (_frozenExtentU, reversal counters, boundary clamping, dampening)	100px, 2 reversals, 2px noise floor, 5px/2 passes, 0.5/10%/0.3 dampening	Froze the extent reported to ScrollViewer — i.e. made the scrollbar wrong on purpose. Probing showed the whole method was dead: disabling it changed no test outcome, because its anchor-drift compensation duplicated ValidateStartU's constant-free StartU -= preDelta.
Layout-cycle breaker (_consecutiveMeasureCount, _measurePostponed, deferred Dispatcher.Post(InvalidateMeasure))	>1	See Accepted trades.
ValidateStartU bespoke logic (lockSizes, _suppressValidateStartU)	1px "real resize" threshold, once-per-arrange suppression	Reduced to constant-free arithmetic + LayoutHelper.LayoutEpsilon.
Estimate caching (_lastEstimateFirstIndex/LastIndex)	—	Existed to stop the swinging estimate from drifting; the estimate no longer swings.
Warmup head sampling (WarmupSampleSize)	50, validated 1..1000	Assumed the head represents the collection.
Tail-realization heuristic in RealizeElements	3 remaining items	By its own comment it existed so "the extent is based on actual measured sizes rather than estimates" — exactly the root cause. Subsumed: scrolling to the end measures the tail, unknownCount reaches 0, and the extent is the exact total.
Automatic virtualization for ITypedDataTemplate with DataType set, and an item.GetType() fallback	—	These are why virtualization was on by default while the PR text called it opt-in. Both gone.
Separate content pooling in ItemsControl	pool cap 5	Two levels of pooling could not be reconciled — the child was pooled while still attached to its old container, so reattaching threw "The control Border already has a visual parent". Superseded by container+child-as-one-unit. Profiling had also shown ~no gain, because the child was still detached and reattached.
Per-item-type template memo in ItemsControl (_templateCache)	—	Inert on the only path it served: PrepareItemContainer runs before AddInternalChild, so an item type's first lookup happens on an unparented container, finds nothing, and memoizes null for good. Making it work would have been worse than deleting it — it would set ContentTemplate from a DataTemplates-collection template while NeedsContainer still returned DefaultRecycleKey, so an opted-in template in a collection would buy the skip-clear while its containers sat in the single shared pool.
Distance-based disjunct gap tolerance	viewport-relative pixel thresholds	Replaced by the index test anchorIndex < FirstIndex || anchorIndex > LastIndex.
[VSP-*] trace logging + IsTracingEnabled	—	Diagnostic scaffolding. The §7 trace above came from it.
Dead public API: GetPoolStats, ClearPools, ContentPoolStats, PoolEntry	—	Never used. ContainerVirtualization is now just the kill switch.

Checklist

• Add unit tests

• Added XML documentation to any related classes?

• Benchmarks — added in tests/Avalonia.Benchmarks/Controls/, results in Performance above.
  Container-level virtualization is ~3.4× on scroll and −83% allocation over complex rows; the
  size record costs 43.7 bytes per item and no per-pass time; RetainMatchingContainers saves
  ~10% of prepares on backwards scrolling only. The one thing still unmeasured is mobile —
  every figure here is desktop x64, which understates the case, but I have not taken Android
  numbers and have not extrapolated any.

• Scope split — I would like guidance here, and the benchmarks now make a concrete proposal
  possible. This used to be "each of these is defensible alone"; the numbers say something
  sharper than that. Item 4 (container-level virtualization, plus warmup and the
  IVirtualizingDataTemplate surface) carries all of the speed: ~3.4× and −83% allocation.
  Items 1–3 (size record, Reset-preservation fix, constant-free anchor compensation) measure as
  performance-neutral and stand on correctness and the constants inventory instead. Item 5
  (RetainMatchingContainers) is ~10% of prepares on backwards scrolling only, and carries the
  most machinery per unit of benefit.

  So the proposal is **two PRs plus a deferral**, not three equal siblings: panel correctness
  (items 1–3) as one, container virtualization (item 4) as the other, `RetainMatchingContainers`
  dropped from both and re-proposed on its own evidence. Happy to do it the other way round, or
  not at all, if you would rather review it whole.
  

• Is Reset-preservation wanted at all? Stock treats Reset as a full rebuild. Preserving
  across it is non-upstream by nature; the scroll-anchor system may be the right owner instead.

• API review / naming — EnableVirtualization, RecycleKeySelector, MaxPoolSizePerKey /
  MinPoolSizePerKey, EnableWarmup, and whether a process-global switch belongs in a
  ContainerVirtualization static class at all.

• Test seam — fixed. AdjustElementSize was protected internal virtual purely so tests
  could inject non-deterministic measurement, and the protected half made it public API by
  accident. It is now internal Func<int, double, double>? ElementSizeAdjustmentForTesting,
  applied in GetElementSizeU exactly where the method was called; the three test panels that
  overrode it assign the delegate instead. (TryGetMeasuredSizeForTesting and
  RecyclePoolForTesting were already internal.)

• View lifecycle events — synthesise Loaded/Unloaded for recycled containers, add
  virtualization-aware equivalents, or document the trade? All three are implementable; each sets
  a framework-wide precedent for what those events mean under virtualization, which is why it is
  your call rather than mine. See the callout at the top.

• User documentation — Document container virtualization for VirtualizingStackPanel avalonia-docs#1113, opened as a draft against main. It
  covers both opt-in forms, choosing a recycle key, warmup, the size record's memory cost, and
  the lifecycle-event caveat above. It should not be merged before this PR, and it needs
  rewriting if the API review renames anything.

Optional

• Add virtualization support to FuncDataTemplate — done. RecycleKeySelector is the code-side
  opt-in, so the feature is no longer XAML-only.
• Implement for VirtualizingPanel too — deliberately out of scope for this PR.

Breaking changes

Virtualization itself is opt-in: a template that does not implement IVirtualizingDataTemplate (or
set EnableVirtualization="True") gets stock behaviour, including DefaultRecycleKey and content
clearing on recycle. Two behavioural changes are worth calling out anyway:

1. View lifecycle events, for opted-in templates only. Because the child stays attached across
   recycling, Loaded/Unloaded and AttachedToVisualTree/DetachedFromVisualTree fire once for
   the first item a container displayed and never again. A template that forwards those to its view
   model, or a control that initialises on load and releases on unload, will not work inside an
   opted-in template without moving that work to DataContextChanged. Unresolved — see the callout
   at the top and the Checklist. Since virtualization became opt-in, this is only paid by templates
   that asked for it, and it is documented in the docs PR below.

2. Panel.Children retains invisible pooled containers, for everyone. Stock's
   RecycleElementOnItemRemoved unparented the container (RemoveInternalChild); this PR pools it
   instead, so it stays in Children with IsVisible = false. That is the point — unparenting is
   exactly the detach/reattach churn container-level virtualization exists to avoid. It is not a
   ghost (invisible and absent from the realized set means nothing renders it; focus navigation
   filters on IsEffectivelyVisible), but it is publicly observable: anything enumerating
   Panel.Children and assuming every child is a live item will now see extras. Stock already
   retained the parent on the ordinary scroll-recycle path; only the item-removed path changed.
   ListBoxVirtualizationIssueTests.GhostItemTest_FocusManagement asserted the old contract and was
   updated.

Obsoletions / Deprecations

None.

Fixed issues

Fixes #20259

Test coverage

Avalonia.Controls.UnitTests is green: 3,840 cases, 3,839 passed, 0 failures, the one skip being
a pre-existing CalendarDatePicker skip unrelated to this change.
Avalonia.Markup.Xaml.UnitTests is green: 591 cases, 590 passed, 0 failures, again with one
pre-existing skip.

File	Tests	Cases
VirtualizingStackPanelTests.cs	153 (73 stock, +80)	310
ContainerVirtualizationTests.cs (new)	21	21
Presenters/ContentPresenterTests_BatchUpdate.cs (new)	8	8

What they cover, by area:

• The size record — remap on insert/remove/replace, drop on Move/Reset, one entry per item
  ever measured, not trimmed when items leave the viewport, survives detach but not a re-attach.
• Adversarial size distributions — uniform, bimodal, extreme outliers, monotonic ramp, async-grow
  (84→292), cross-region. Extent stability and, for the cross-region case, extent correctness
  (converged on the true 34000px total — a stability-only assertion cannot tell a settled estimate
  from a settled wrong one).
• Collection edits — the 28-case Collection_Edit_Keeps_Every_Container_On_Its_Own_Item matrix,
  plus reorder/append coalesced into a Reset.
• Anchor compensation — sub-pixel pre-anchor resize compensated not absorbed, a second resize in
  the same arrange cycle, anchor stability across repeated passes with few realized items.
• Container virtualization — the kill switch, pool capping for plain and opted-in templates,
  GetKey / Build(data, existing) / EnableVirtualization, the typed-template skip-clear,
  NeedsContainer<T> ordering, template resolution, nested/recursive virtualization, batch update.
• Warmup — key discovery from encountered keys, forgetting vanished keys, MinPoolSizePerKey,
  and the dispatcher-tick-after-detach case.
• Staleness self-heal — off-view height change re-measured on scroll back, retained-container
  reuse re-measures a changed item, visible items always use the live measure.

Everything whose contract is axis-independent runs as a theory over Orientation. That half is not
decorative: two deliberately-injected axis-only defects (reading DesiredSize.Height in
GetElementSizeU; reading viewport.Y/.Bottom in CalculateMeasureViewport) turned 27 and 50
cases red respectively, no vertical case either time.

Files changed

Production

• src/Avalonia.Controls/VirtualizingStackPanel.cs (+1309/-127) — the size record and its lifecycle,
  CacheBasedExtentU, constant-free ValidateStartU use, RetainMatchingContainers, Reset
  preservation gate, Refresh() override, warmup, the zero-viewport guard.
• src/Avalonia.Controls/Utils/RealizedStackElements.cs (+150/-11) — NullifyElement, and
  ValidateStartU's new signature (int anchorIndex, Func<Control,int,double> getSizeU, out double preDelta).
• src/Avalonia.Controls/ItemsControl.cs (+150/-10) — recycle keys, conditional content clearing,
  SetIfUnsetOrDifferent, GetEffectiveItemTemplate, GetMaxPoolSizePerKey / GetMinPoolSizePerKey,
  ContainerVirtualization.
• src/Avalonia.Controls/Presenters/ContentPresenter.cs (+50/-3) — BeginBatchUpdate /
  EndBatchUpdate (nesting depth counter) and the pseudo-class/measure refresh they owe the batch.
• src/Avalonia.Controls/Templates/IVirtualizingDataTemplate.cs (+34) — new.
• src/Avalonia.Controls/Templates/FuncDataTemplate.cs (+48/-2) — implements IVirtualizingDataTemplate;
  RecycleKeySelector (null by default) is the opt-in for code-defined templates.
• src/Markup/Avalonia.Markup.Xaml/Templates/DataTemplate.cs (+38/-1) — EnableVirtualization,
  MaxPoolSizePerKey, MinPoolSizePerKey.
• src/Avalonia.Controls/VirtualizingPanel.cs (+14/-1) — Refresh() made virtual.

Sample — samples/ControlCatalog/Pages/ListBoxComplexLayoutPage.{xaml,xaml.cs},
Pages/FieldTemplateSelector.cs, Converter/MarkdownToInlinesConverter.cs,
ViewModels/ListBoxComplexLayoutPageViewModel.cs, plus the page-list entry and small ListBoxPage
edits.

Tests — VirtualizingStackPanelTests.cs, new ContainerVirtualizationTests.cs, new
Presenters/ContentPresenterTests_BatchUpdate.cs, and updates to ItemsControlTests.cs /
ListBoxVirtualizationIssueTests.cs.

[gentledepp]

@MrJul @kekekeks I am sorry for bothering you.
I have put a decent amount of work into this already (rebasing on master, I saw that some of my changes already found their way into the VirtualizingStackPanel) and burned through hundreds of dollars in AI.
So before I put in even more effort, I'd like to know if this is of any interest of you.

From my point of view, this is a necessity, not an option.
As an avalonia user, I expected those features to already be in there, only to find out that our app scrolls slower than with Xamarin.Forms - so I got a platform developer.

Now to be fair: This PR is not complete yet.
There is a virtualizingstackpanel_test_todo.md file that outlines some of the fixes I introduced with the help of AI that cannot be easily tested.
And I'd have to remove some markdown files and put more effort into the sample app to showcase how my contribution actually improves the whole scrolling experience by a mile or two.

If you do not find this interesting, I will just extract out a "MyVirtualizingStackpanel" and let it live in our codebase.

[MrJul]

Hi @gentledepp,

First of all, thank you for your contribution!

General performance improvements are always welcome. VirtualizingStackPanel is no different. However, it is a critical and complex component, and major changes like this need a thorough review and usually a team discussion (especially since new APIs are involved).

Right now, as you might have noticed, we're in the process of shipping Avalonia v12 and won't be able to review this PR before the final release.

Personally, I really like the concept of reusing content across templates. It makes a lot of sense. Assuming the implementation is good enough and well tested (and it looks like it is!), this is something I'm interested in merging, so please keep it open for now :)

One thing that could really help us get this PR merged faster is to split it. For example, can all the improvements in scrolling calculation be extracted to a separate PR? That way, we could review the IVirtualizingDataTemplate separately, and have improvements be delivered incrementally.

[AnnDevNet]

I'm eager to see this project come to fruition so after for a VirtualizingCanvas for ItemsControls for 2D virtualization management, allowing to display 2D graphs with numerous controls on an unlimited canvas :-)
Do you know if this already exists?

[gentledepp]

Two questions for the Avalonia team, plus a fallback offer. Both are decisions about framework contracts rather than work we still owe, and we'd rather have your answer before pushing further in either direction.

1. Is Reset-preservation wanted at all?

Stock treats NotifyCollectionChangedAction.Reset as "throw everything away and rebuild". This branch preserves realized containers across a Reset — but only when every realized element still validates against the new collection — so an infinite-scroll list that coalesces its edits into a Reset keeps its scroll position.

That is not a bug fix. It is us overriding a framework contract because it suits one usage pattern. You may reasonably say the scroll-anchor system is the right owner of scroll stability, or that Reset should stay dumb.

One consequence worth knowing before deciding: the Tier A correctness fix in this PR only exists because we preserve. Reset_With_MidList_Insert_Realizes_Shifted_Items_At_Correct_Index covers a wrong-index render after a mid-list edit that got coalesced into a Reset — a failure mode a rebuild-everything Reset cannot have. So if preservation goes, that fix goes with it, and the two should come out together rather than leaving the guard behind. This question can delete a feature and a fix in one answer.

2. View lifecycle events under container virtualization

For templates that opted in, a container's Child stays attached across recycling, so Loaded/Unloaded do not fire per item: a control realized once and reused for 500 items sees one Loaded. Templates that did not opt in are unaffected.

Three options, all implementable, none of them ours to pick:

• Synthesise Loaded/Unloaded on recycle — existing per-item code keeps working, at the cost of those events no longer meaning "attached to / detached from the visual tree".
• Add virtualization-aware equivalents (something like Realized/Recycled) and leave Loaded/Unloaded meaning exactly what they mean today.
• Document it as a trade of the opt-in and change nothing. This is what the branch does now.

Whichever you choose sets a framework-wide precedent for what those events mean under virtualization, which is why we'd like the call to be yours rather than ours.

Fallback: a separate panel is fine by us

If the answer to either question is "not in VirtualizingStackPanel", that is a perfectly good outcome from our side. The container-virtualization half can ship as a separate panel — ContentVirtualizingStackPanel, say — that opts in by type rather than by flag, leaving the stock panel untouched. If you prefer that shape, we will restructure it rather than argue for it in place.

[cla-avalonia]

• All contributors have signed the CLA.

[gentledepp]

@MrJul — the description has been rewritten against the current diff, and the two things it was waiting on are now done, so this is at the point where a scope decision from your side would help most.

Is this split acceptable to you? The benchmarks turned what used to be a judgement call into something concrete:

Part	What it is	What it measures
Items 1–3: size record, Reset-preservation fix, constant-free anchor compensation	Correctness, plus the removal of every fork-added tuning constant	Performance-neutral on real templates; faster only where panel bookkeeping is the whole cost
Item 4: container-level virtualization (IVirtualizingDataTemplate, EnableVirtualization, RecycleKeySelector, warmup)	The opt-in that keeps a container's child attached across recycling	All of the speed: ~3.4× scroll, ~2.4× jumps, −83% allocation, ~20× lower variance, at identical prepare counts
Item 5: RetainMatchingContainers	Reuse of containers whose item survives a viewport jump	~10% of prepares on backwards scroll and paging only, below the timing noise floor, and the most machinery per unit of benefit

So the proposal is two PRs plus a deferral: panel correctness as one, container virtualization as the other, and RetainMatchingContainers pulled out of both and re-proposed separately on its own evidence. If you would rather review it whole, or split it differently, say so and I will restructure the branches. I would rather do that work once, in the shape you want to review, than guess.

Two other things worth your attention:

• View lifecycle events. For an opted-in template, the container and its child stay in the visual tree and are only hidden, so Loaded/Unloaded and AttachedToVisualTree/DetachedFromVisualTree fire once and never again per item. That is the feature working as designed, and it is also the one way it can break a working list. Synthesising the events, adding virtualization-aware equivalents, or documenting the trade are all implementable; each sets a framework-wide precedent, which is why it is a call for you rather than for me. It is called out at the top of the description now.
• Docs. Document container virtualization for VirtualizingStackPanel avalonia-docs#1113 is open as a draft covering both opt-in forms, recycle-key choice, warmup, the size record's memory cost, and the lifecycle caveat. It should not merge before this PR, and it will need rewriting if the API review renames anything.

The earlier questions from this comment are still open too, in particular whether Reset-preservation is wanted at all. If it is not, the Tier A correctness fix goes with it, since the wrong-index render only exists on the preserving path.

[MrJul]

@gentledepp

Is this split acceptable to you?

It looks reasonable: it will be way easier to review and merge the correctness fixes independently of the new content virtualization feature.

View lifecycle events. [...] That is the feature working as designed, and it is also the one way it can break a working list.

Since the feature is opt-in, I think it's completely fine for it to come with new, documented restrictions. They look reasonable to me. We can then wait for feedback and remove them in the future if there's demand.

Is Reset-preservation wanted at all?

My opinion is that it isn't wanted. Reset has always been "throw away everything" and we've rejected previous PRs that tried to reconcile old and new items. As usual, it's easier not to have it and add it back later if we need to than to remove the behavior if it happens to cause performance issues.

[grokys]

I will try to review this, but for reference @gentledepp: that PR description is useless, I'm not reading that wall of incomprehensible text. Can you get your LLM to rewrite it in non-Claude speak, fix line breaks and condense it to the useful information for a reviewer?

[grokys]

From the little of the description that I did read, it looks like this PR merges two things:

1. Makes VirtualizingStackPanel's extent estimate independent of which items happen to be
   realized.
2. Adds opt-in container-level virtualization

That makes the PR even more difficult to review. Tempted to close this as it's IMO unreviewable in this state.

I know you say that you've put a lot of your time into this, but honestly reading that LLM-generated wall of text just put me in a really bad mood, and the code isn't much better. I don't mind AI-generated stuff (I use them myself), but this is just shovelling slop on the poor reviewer.

[grokys]

Yeah apologies, I'm going to close this. I can't even tell what it's supposed to do. Not a good use of my time.

Please, open an issue describing the feature you'd like, and I'll tell you if it's a good idea. I think I know what this is wanting to achieve but I really can't be sure.