AppDomain.MonitoringSurvivedMemorySize incorrect in .NET 5.0

[timcassell]

While trying to add a new feature to BDN to measure survived memory (dotnet/BenchmarkDotNet#1596), I found that .NET Core 3.1 and .NET 5.0 were reporting unexpected values. I made a separate application to test whether the measurements I'm making are even accurate separate from all other BDN code. I found that it seems to be accurate in .NET Core 3.1, but it's off in .NET 5.0.

Console prints:

Survived bytes: 0
Survived bytes: 24
Survived bytes: 24

With this code:

using System;
using System.Runtime.CompilerServices;

namespace MemoryTest
{
    class Program
    {
        static int n;

        static void Main(string[] args)
        {
            n = 1_000_000;
            Measure(); // Run once for GC monitor to make its allocations.
            Console.WriteLine($"Survived bytes: {Measure()}");
            Console.WriteLine($"Survived bytes: {Measure()}");
            Console.WriteLine($"Survived bytes: {Measure()}");
        }

        static long Measure()
        {
            long beforeBytes = GetTotalBytes();
            NonAllocatingMethod();
            long afterBytes = GetTotalBytes();
            return afterBytes - beforeBytes;
        }


        [MethodImpl(MethodImplOptions.NoInlining)]
        static void NonAllocatingMethod()
        {
            for (int i = 0; i < n; i++) { }
        }

        static long GetTotalBytes()
        {
            AppDomain.MonitoringIsEnabled = true;

            // Enforce GC.Collect here to make sure we get accurate results.
            GC.Collect();
            GC.WaitForPendingFinalizers();
            GC.Collect();
            return AppDomain.CurrentDomain.MonitoringSurvivedMemorySize;
        }
    }
}

[Edit] This was ran with Visual Studio 16.8.2 on Windows 7 SP1 on AMD Phenom II x6.

Also, the results above were from running in DEBUG mode directly in Visual Studio. After building in RELEASE configuration and running it separately, I got these results (different, but still wrong):

Survived bytes: 0
Survived bytes: 24
Survived bytes: 0

[Dotnet-GitSync-Bot]

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

Tagging subscribers to this area: @dotnet/gc
See info in area-owners.md if you want to be subscribed.

Issue Details

---

While trying to add a new feature to BDN to measure survived memory (dotnet/BenchmarkDotNet#1596), I found that .NET Core 3.1 and .NET 5.0 were reporting unexpected values. I made a separate application to test whether the measurements I'm making are even accurate separate from all other BDN code. I found that it seems to be accurate in .NET Core 3.1, but it's off in .NET 5.0.

Console prints:

Survived bytes: 0
Survived bytes: 24
Survived bytes: 24

With this code:

using System;
using System.Runtime.CompilerServices;

namespace MemoryTest
{
    class Program
    {
        static int n;

        static void Main(string[] args)
        {
            n = 1_000_000;
            Measure(); // Run once for GC monitor to make its allocations.
            Console.WriteLine($"Survived bytes: {Measure()}");
            Console.WriteLine($"Survived bytes: {Measure()}");
            Console.WriteLine($"Survived bytes: {Measure()}");
        }

        static long Measure()
        {
            long beforeBytes = GetTotalBytes();
            NonAllocatingMethod();
            long afterBytes = GetTotalBytes();
            return afterBytes - beforeBytes;
        }


        [MethodImpl(MethodImplOptions.NoInlining)]
        static void NonAllocatingMethod()
        {
            for (int i = 0; i < n; i++) { }
        }

        static long GetTotalBytes()
        {
            AppDomain.MonitoringIsEnabled = true;

            // Enforce GC.Collect here to make sure we get accurate results.
            GC.Collect();
            GC.WaitForPendingFinalizers();
            GC.Collect();
            return AppDomain.CurrentDomain.MonitoringSurvivedMemorySize;
        }
    }
}

[Edit] This was ran with Visual Studio 16.8.2 on Windows 7 SP1 on AMD Phenom II x6.

Also, the results above were from running in DEBUG mode directly in Visual Studio. After building in RELEASE configuration and running it separately, I got these results (different, but still wrong):

Survived bytes: 0
Survived bytes: 24
Survived bytes: 0

Author:	timcassell
Assignees:	-
Labels:	area-GC-coreclr, untriaged
Milestone:	-

[timcassell]

I actually tried running that test again, and kept getting different results. So I rewrote the code to do many iterations to try to find any anomalies, and I ran it in different runtimes.
.NET 5.0 finds lots of anomalies sometimes (300+), sometimes only a few (4 or 5).
.NET Core 3.1 finds a few anomalies (3 or 4).
.NET Framework 4.7.2 finds no anomalies.

using System;
using System.Collections.Generic;
using System.Runtime.CompilerServices;

namespace MemoryTest
{
    class Program
    {
        static void Main(string[] args)
        {
            Measure(); // Run once for GC monitor to make its allocations.

            int abnormalCount = 0;

            const int iterations = 10_000;
            Console.WriteLine($"Searching for abnormal survived memory...");
            for (int i = 0; i < iterations; ++i)
            {
                long bytes = Measure();
                if (bytes != 0)
                {
                    ++abnormalCount;
                    Console.WriteLine($"Iteration {i}, survived bytes: {bytes}");
                }
            }

            Console.WriteLine($"Abnormal survived memory found in {abnormalCount} out of {iterations} iterations.");
        }

        static long Measure()
        {
            long beforeBytes = GetTotalBytes();
            NonAllocatingMethod();
            long afterBytes = GetTotalBytes();
            return afterBytes - beforeBytes;
        }


        [MethodImpl(MethodImplOptions.NoInlining)]
        static void NonAllocatingMethod() { }

        static long GetTotalBytes()
        {
            AppDomain.MonitoringIsEnabled = true;

            // Enforce GC.Collect here to make sure we get accurate results.
            GC.Collect();
            GC.WaitForPendingFinalizers();
            GC.Collect();
            return AppDomain.CurrentDomain.MonitoringSurvivedMemorySize;
        }
    }
}

[timcassell]

Interestingly, I found similar issues with regards to measuring allocations in both Core 3.1 and 5.0. See dotnet/BenchmarkDotNet#1543
I am unsure if the issue with allocations and total memory are related. @adamsitnik mentioned in that other issue that he tracked that issue down to the jitter running in a separate thread. It seems to be completely random when it occurs (for both measurements).

Searching for abnormal allocated memory...
Iteration 730, allocated bytes: 40
Iteration 1503, allocated bytes: 40
Abnormal allocated memory found in 2 out of 10000 iterations.

using System;
using System.Runtime.CompilerServices;

namespace MemoryTest
{
    class Program
    {
        static void Main(string[] args)
        {
            Measure(); // Run once for GC monitor to make its allocations.

            int abnormalCount = 0;

            const int iterations = 10_000;
            Console.WriteLine($"Searching for abnormal allocated memory...");
            for (int i = 0; i < iterations; ++i)
            {
                long bytes = Measure();
                if (bytes != 0)
                {
                    ++abnormalCount;
                    Console.WriteLine($"Iteration {i}, survived bytes: {bytes}");
                }
            }

            Console.WriteLine($"Abnormal allocated memory found in {abnormalCount} out of {iterations} iterations.");
            Console.ReadKey();
        }

        static long Measure()
        {
            long beforeBytes = GetAllocatedBytes();
            NonAllocatingMethod();
            long afterBytes = GetAllocatedBytes();
            return afterBytes - beforeBytes;
        }


        [MethodImpl(MethodImplOptions.NoInlining)]
        static void NonAllocatingMethod() { }

        static long GetAllocatedBytes()
        {
            AppDomain.MonitoringIsEnabled = true;

            // Enforce GC.Collect here to make sure we get accurate results.
            GC.Collect();
            GC.WaitForPendingFinalizers();
            GC.Collect();
            return AppDomain.CurrentDomain.MonitoringTotalAllocatedMemorySize;
        }
    }
}

[ivdiazsa]

Closing since by .NET 6.0 Preview 6, this issue doesn't reproduce anymore.