NullReferenceException in ConditionalWeakTable`2.Remove

[mattleibow]

Description

I am working on a PR here: https://github.com/dotnet/maui/pull/14846/files and I see a test failure that is an exception inside ConditionalWeakTable. I am semi-sure that my changes don't have anything to do with it and the test that is failing does not really use anything that I am working on.

However, since this is an exception inside CWT, I think that there may be an issue there at least:

System.NullReferenceException : Object reference not set to an instance of an object.
Stack Trace:
at System.Runtime.CompilerServices.ConditionalWeakTable`2.Remove(TKey key)

https://github.com/dotnet/runtime/blob/main/src/libraries/System.Private.CoreLib/src/System/Runtime/CompilerServices/ConditionalWeakTable.cs#L159

As far as I can see, the only things in that Remove method that can be null is _container... Bu that should not be null, so this is scary-ish.

Reproduction Steps

This is the hard part and I am trying to find something.

Expected behavior

No exception.

Actual behavior

Exception.

Regression?

No response

Known Workarounds

No response

Configuration

.NET 7.0.201 on macOS. Seems to work fine on Windows.

Onlyfailing on CI, cannot reproduce this on local Windows/macOS.

Other information

No response

[AustinWise]

FYI, _container can become null if the ConditionalWeakTable is resurrected.

The full stack trace from the test failure:

  Failed FooBz54334(False) [15 ms]
  Error Message:
   System.NullReferenceException : Object reference not set to an instance of an object.
  Stack Trace:
     at System.Runtime.CompilerServices.ConditionalWeakTable`2.Remove(TKey key)
   at Microsoft.Maui.Controls.Setter.UnApply(BindableObject target, Boolean fromStyle) in /Users/builder/azdo/_work/1/s/src/Controls/src/Core/Setter.cs:line 112
   at Microsoft.Maui.Controls.Style.UnApplyCore(BindableObject bindable, Style basedOn) in /Users/builder/azdo/_work/1/s/src/Controls/src/Core/Style.cs:line 167
   at Microsoft.Maui.Controls.Style.Microsoft.Maui.Controls.IStyle.UnApply(BindableObject bindable) in /Users/builder/azdo/_work/1/s/src/Controls/src/Core/Style.cs:line 109
   at Microsoft.Maui.Controls.MergedStyle.SetStyle(IStyle implicitStyle, IList`1 classStyles, IStyle style) in /Users/builder/azdo/_work/1/s/src/Controls/src/Core/MergedStyle.cs:line 187
   at Microsoft.Maui.Controls.MergedStyle.set_ImplicitStyle(IStyle value) in /Users/builder/azdo/_work/1/s/src/Controls/src/Core/MergedStyle.cs:line 93
   at Microsoft.Maui.Controls.MergedStyle.OnImplicitStyleChanged() in /Users/builder/azdo/_work/1/s/src/Controls/src/Core/MergedStyle.cs:line 132
   at Microsoft.Maui.Controls.MergedStyle.<RegisterImplicitStyles>b__30_0(BindableObject bindable, Object oldvalue, Object newvalue) in /Users/builder/azdo/_work/1/s/src/Controls/src/Core/MergedStyle.cs:line 148
   at Microsoft.Maui.Controls.BindableObject.SetValueActual(BindableProperty property, BindablePropertyContext context, Object value, Boolean currentlyApplying, SetValueFlags attributes, Boolean silent) in /Users/builder/azdo/_work/1/s/src/Controls/src/Core/BindableObject.cs:line 537
   at Microsoft.Maui.Controls.BindableObject.SetValueCore(BindableProperty property, Object value, SetValueFlags attributes, SetValuePrivateFlags privateAttributes) in /Users/builder/azdo/_work/1/s/src/Controls/src/Core/BindableObject.cs:line 469
   at Microsoft.Maui.Controls.BindableObject.SetValueCore(BindableProperty property, Object value, SetValueFlags attributes) in /Users/builder/azdo/_work/1/s/src/Controls/src/Core/BindableObject.cs:line 401
   at Microsoft.Maui.Controls.Element.OnResourceChanged(BindableProperty property, Object value) in /Users/builder/azdo/_work/1/s/src/Controls/src/Core/Element.cs:line 745
   at Microsoft.Maui.Controls.Element.OnResourcesChanged(IEnumerable`1 values) in /Users/builder/azdo/_work/1/s/src/Controls/src/Core/Element.cs:line 601
   at Microsoft.Maui.Controls.Element.OnParentResourcesChanged(IEnumerable`1 values) in /Users/builder/azdo/_work/1/s/src/Controls/src/Core/Element.cs:line 553
   at Microsoft.Maui.Controls.VisualElement.OnParentResourcesChanged(IEnumerable`1 values) in /Users/builder/azdo/_work/1/s/src/Controls/src/Core/VisualElement.cs:line 1178
   at Microsoft.Maui.Controls.Element.OnParentResourcesChanged(Object sender, ResourcesChangedEventArgs e) in /Users/builder/azdo/_work/1/s/src/Controls/src/Core/Element.cs:line 548
   at Microsoft.Maui.Controls.Element.OnResourcesChanged(IEnumerable`1 values) in /Users/builder/azdo/_work/1/s/src/Controls/src/Core/Element.cs:line 579
   at Microsoft.Maui.Controls.Element.OnParentResourcesChanged(IEnumerable`1 values) in /Users/builder/azdo/_work/1/s/src/Controls/src/Core/Element.cs:line 553
   at Microsoft.Maui.Controls.VisualElement.OnParentResourcesChanged(IEnumerable`1 values) in /Users/builder/azdo/_work/1/s/src/Controls/src/Core/VisualElement.cs:line 1178
   at Microsoft.Maui.Controls.Element.OnParentResourcesChanged(Object sender, ResourcesChangedEventArgs e) in /Users/builder/azdo/_work/1/s/src/Controls/src/Core/Element.cs:line 548
   at Microsoft.Maui.Controls.Element.OnResourcesChanged(IEnumerable`1 values) in /Users/builder/azdo/_work/1/s/src/Controls/src/Core/Element.cs:line 579
   at Microsoft.Maui.Controls.Element.OnParentResourcesChanged(IEnumerable`1 values) in /Users/builder/azdo/_work/1/s/src/Controls/src/Core/Element.cs:line 553
   at Microsoft.Maui.Controls.VisualElement.OnParentResourcesChanged(IEnumerable`1 values) in /Users/builder/azdo/_work/1/s/src/Controls/src/Core/VisualElement.cs:line 1178
   at Microsoft.Maui.Controls.Element.OnParentResourcesChanged(Object sender, ResourcesChangedEventArgs e) in /Users/builder/azdo/_work/1/s/src/Controls/src/Core/Element.cs:line 548
   at Microsoft.Maui.Controls.Element.OnResourcesChanged(IEnumerable`1 values) in /Users/builder/azdo/_work/1/s/src/Controls/src/Core/Element.cs:line 579
   at Microsoft.Maui.Controls.Element.OnParentResourcesChanged(IEnumerable`1 values) in /Users/builder/azdo/_work/1/s/src/Controls/src/Core/Element.cs:line 553
   at Microsoft.Maui.Controls.Element.OnParentResourcesChanged(Object sender, ResourcesChangedEventArgs e) in /Users/builder/azdo/_work/1/s/src/Controls/src/Core/Element.cs:line 548
   at Microsoft.Maui.Controls.Element.OnResourcesChanged(IEnumerable`1 values) in /Users/builder/azdo/_work/1/s/src/Controls/src/Core/Element.cs:line 579
   at Microsoft.Maui.Controls.Application.set_Resources(ResourceDictionary value) in /Users/builder/azdo/_work/1/s/src/Controls/src/Core/Application.cs:line 149
   at Microsoft.Maui.Controls.Xaml.UnitTests.Bz54334App.ToggleTheme() in /Users/builder/azdo/_work/1/s/src/Controls/tests/Xaml.UnitTests/Issues/Bz54334.xaml.cs:line 32
   at Microsoft.Maui.Controls.Xaml.UnitTests.Bz54334App.<.ctor>b__1_0(Object s, ContentPage m) in /Users/builder/azdo/_work/1/s/src/Controls/tests/Xaml.UnitTests/Issues/Bz54334.xaml.cs:line 26
   at CommunityToolkit.Mvvm.Messaging.Internals.MessageHandlerDispatcher.For`2.Invoke(Object recipient, Object message)
   at CommunityToolkit.Mvvm.Messaging.WeakReferenceMessenger.SendAll[TMessage](ReadOnlySpan`1 pairs, Int32 i, TMessage message)
   at CommunityToolkit.Mvvm.Messaging.WeakReferenceMessenger.Send[TMessage,TToken](TMessage message, TToken token)
   at Microsoft.Maui.Controls.Xaml.UnitTests.Bz54334.Tests.FooBz54334(Boolean useCompiledXaml) in /Users/builder/azdo/_work/1/s/src/Controls/tests/Xaml.UnitTests/Issues/Bz54334.xaml.cs:line 81
   at InvokeStub_Tests.FooBz54334(Object, Object, IntPtr*)
   at System.Reflection.MethodInvoker.Invoke(Object obj, IntPtr* args, BindingFlags invokeAttr)

Both the Setter class and the WeakReferenceMessenger class use ConditionalWeakTable. Probably the Setter object was finalized along with the ConditionalWeakTable it owns.

Here is a simplified example of dependent handles exposing finalized objects:

using System.Runtime;
using System.Runtime.CompilerServices;

class Program
{
    static void Main()
    {
        while (true)
        {
            var weak = Allocate();
            GC.Collect();
            Observe(weak);
            weak.Dispose();
        }
    }

    [MethodImpl(MethodImplOptions.NoInlining)]
    static DependentHandle Allocate()
    {
        var obj = new MyClass();
        return new DependentHandle(obj, obj);
    }

    [MethodImpl(MethodImplOptions.NoInlining)]
    static void Observe(DependentHandle refcwt)
    {
        var obj = (MyClass?)refcwt.Target;
        if (obj is null)
            return;
        if (obj._isFinialized)
        {
            throw new Exception("Found finialized object.");
        }
    }
}

class MyClass
{
    public bool _isFinialized;
    ~MyClass()
    {
        _isFinialized = true;
    }
}

I think this is expected behavior. If the dependent handle did not keep tracking the object after it become finializable, the conditional weak table would not be readable from finalizers.

Tagging subscribers to this area: @dotnet/area-system-runtime-compilerservices
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Description

I am working on a PR here: https://github.com/dotnet/maui/pull/14846/files and I see a test failure that is an exception inside ConditionalWeakTable. I am semi-sure that my changes don't have anything to do with it and the test that is failing does not really use anything that I am working on.

However, since this is an exception inside CWT, I think that there may be an issue there at least:

System.NullReferenceException : Object reference not set to an instance of an object.
Stack Trace:
at System.Runtime.CompilerServices.ConditionalWeakTable`2.Remove(TKey key)

https://github.com/dotnet/runtime/blob/main/src/libraries/System.Private.CoreLib/src/System/Runtime/CompilerServices/ConditionalWeakTable.cs#L159

As far as I can see, the only things in that Remove method that can be null is _container... Bu that should not be null, so this is scary-ish.

Reproduction Steps

This is the hard part and I am trying to find something.

Expected behavior

No exception.

Actual behavior

Exception.

Regression?

No response

Known Workarounds

No response

Configuration

.NET 7.0.201 on macOS. Seems to work fine on Windows.

Onlyfailing on CI, cannot reproduce this on local Windows/macOS.

Other information

No response

Author:	mattleibow
Assignees:	-
Labels:	area-System.Runtime.CompilerServices, untriaged, needs-area-label
Milestone:	-

[mattleibow]

Thanks for the comments - looking closer, I see a whole tree is being collected/finalized. Not sure why yet as these are direct field references.

We probably have something somewhere that is using weak references and our GC.Collect() happening in parallel for our GC/memory tests are probably causing this to be collected. This feels like it is back on me to find out why :)

[mattleibow]

But, maybe this should transition into a different error message? Not sure if an exception saying "The ConditionalWeakTable was finalized" is a better message because a null ref was unexpected.

[ericstj]

@mattleibow it sounds like you suspect a problem in maui here? Did you want to transfer this to the maui repo, or did you want us to use this issue to track adding an exception message to ConditionalWeakTable to throw when it's accessed after it's been

runtime/src/libraries/System.Private.CoreLib/src/System/Runtime/CompilerServices/ConditionalWeakTable.cs

Line 457 in 1e42167

private bool _finalized; // set to true when initially finalized

This issue has been marked needs-author-action and may be missing some important information.

[mattleibow]

Both? I already have things in Maui to track this issue, but the runtime repo can also have it as a better exception will probably save days for developers.