GC ReliabilityFramework with suppressConsoleOutputFromTests set to false fails

[a74nh]

Description

Running the GC ReliabilityFramework test with suppressConsoleOutputFromTests set to false fails with missing .dll

Reproduction Steps

Linux Arm64:

./build.sh -rc checked -c release -s clr+libs
export CORE_ROOT=$(pwd)/artifacts/tests/coreclr/linux.arm64.Checked/Tests/Core_Root
cd tests
./build.sh checked
cd ../artifacts/tests/coreclr/linux.arm64.Checked/GC/Stress/Framework/ReliabilityFramework

edit testmix_gc_ci.config to set suppressConsoleOutputFromTests to false

export RunningLongGCTests=1
./ReliabilityFramework.sh

Expected behavior

Test runs for an hour

Actual behavior

The test exits with cannot find Microsoft.Win32.Primitives.dll

❯ ./ReliabilityFramework.sh
BEGIN EXECUTION
/mnt/sda/home/alahay01/dotnet/runtime_sve/artifacts/tests/coreclr/linux.arm64.Checked/Tests/Core_Root/corerun -p System.Reflection.Metadata.MetadataUpdater.IsSupported=false -p System.Runtime.Serialization.EnableUnsafeBinaryFormatterSerialization=true ReliabilityFramework.dll '-unittest'
Using config file: /mnt/sda/home/alahay01/dotnet/runtime_sve/artifacts/tests/coreclr/linux.arm64.Checked/GC/Stress/Framework/ReliabilityFramework/testmix_gc_ci.config
NUMBER_OF_PROCESSORS environment variable not supplied, falling back to Environment
test is 573277.dll
test is allocationwithpins.dll
test is b115557.dll
test is bestfit-finalize.dll
test is concurrentspin2.dll
test is doubLinkStay.dll
test is ExpandHeap.dll
test is GCQueue.dll
test is GCSimulator.dll
test is GCVariant.dll
test is LeakGenThrd.dll
test is MulDimJagAry.dll
test is pinstress.dll
test is plug.dll
test is PlugGaps.dll
test is SingLinkStay.dll
test is StressAllocator.dll
test is StressAllocator.dll
test is ThdTreeGrowingObj.dll
Loading all tests: ...................
Process terminated. Test failed
   at System.Environment.FailFast(System.Runtime.CompilerServices.StackCrawlMarkHandle, System.String, System.Runtime.CompilerServices.ObjectHandleOnStack, System.String)
   at System.Environment.FailFast(System.Threading.StackCrawlMark ByRef, System.String, System.Exception, System.String)
   at System.Environment.FailFast(System.String, System.Exception)
   at ReliabilityFramework+<>c__DisplayClass36_0.<StartTestWorker>b__0()
   at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(System.Threading.Thread, System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object)
   at System.Threading.Tasks.Task.ExecuteWithThreadLocal(System.Threading.Tasks.Task ByRef, System.Threading.Thread)
   at System.Threading.ThreadPoolWorkQueue.Dispatch()
   at System.Threading.PortableThreadPool+WorkerThread.WorkerThreadStart()
   at System.Threading.Thread+StartHelper.RunWorker()
System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation.
 ---> System.IO.FileNotFoundException: Could not load file or assembly 'Microsoft.Win32.Primitives, Version=10.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'. The system cannot find the file specified.

File name: 'Microsoft.Win32.Primitives, Version=10.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'
 ---> System.IO.FileNotFoundException: Could not load file or assembly '/mnt/sda/home/alahay01/dotnet/runtime_sve/artifacts/tests/coreclr/linux.arm64.Checked/GC/Stress/Framework/ReliabilityFramework/Tests/Microsoft.Win32.Primitives.dll'. The system cannot find the file specified.

File name: '/mnt/sda/home/alahay01/dotnet/runtime_sve/artifacts/tests/coreclr/linux.arm64.Checked/GC/Stress/Framework/ReliabilityFramework/Tests/Microsoft.Win32.Primitives.dll'
   at System.Runtime.Loader.AssemblyLoadContext.LoadFromAssemblyPath(String assemblyPath)
   at CustomAssemblyResolver.Load(AssemblyName assemblyName) in /mnt/sda/home/alahay01/dotnet/runtime_sve/src/tests/GC/Stress/Framework/ReliabilityFramework.cs:line 62
   at System.Runtime.Loader.AssemblyLoadContext.ResolveUsingLoad(AssemblyName assemblyName)
   at System.Runtime.Loader.AssemblyLoadContext.Resolve(IntPtr gchManagedAssemblyLoadContext, AssemblyName assemblyName)
   at System.ConsolePal.EnsureInitializedCore()
   at System.ConsolePal.EnsureConsoleInitialized() in /mnt/sda/home/alahay01/dotnet/runtime_sve/src/libraries/System.Console/src/System/ConsolePal.Unix.cs:line 872
   at System.ConsolePal.WriteFromConsoleStream(SafeFileHandle fd, ReadOnlySpan`1 buffer) in /mnt/sda/home/alahay01/dotnet/runtime_sve/src/libraries/System.Console/src/System/ConsolePal.Unix.cs:line 958
   at System.ConsolePal.UnixConsoleStream.Write(ReadOnlySpan`1 buffer) in /mnt/sda/home/alahay01/dotnet/runtime_sve/src/libraries/System.Console/src/System/ConsolePal.Unix.ConsoleStream.cs:line 50
   at System.IO.StreamWriter.Flush(Boolean flushStream, Boolean flushEncoder)
   at System.IO.StreamWriter.WriteFormatHelper(String format, ReadOnlySpan`1 args, Boolean appendNewLine)
   at System.IO.StreamWriter.WriteLine(String format, Object arg0)
   at System.IO.TextWriter.SyncTextWriter.WriteLine(String format, Object arg0)
   at System.Console.WriteLine(String format, Object arg0) in /mnt/sda/home/alahay01/dotnet/runtime_sve/src/libraries/System.Console/src/System/Console.cs:line 825
   at B115557.Main() in /mnt/sda/home/alahay01/dotnet/runtime_sve/src/tests/GC/Stress/Tests/b115557.cs:line 54
   at System.Reflection.MethodBaseInvoker.InterpretedInvoke_Method(Object obj, IntPtr* args)
   at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)
   --- End of inner exception stack trace ---
   at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)
   at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
   at ReliabilityFramework.<>c__DisplayClass36_0.<StartTestWorker>b__0() in /mnt/sda/home/alahay01/dotnet/runtime_sve/src/tests/GC/Stress/Framework/ReliabilityFramework.cs:line 1010./ReliabilityFramework.sh: line 454: 733685 Aborted                 (core dumped) $LAUNCHER $ExePath "${CLRTestExecutionArguments[@]}"
Expected: 100
Actual: 134
END EXECUTION - FAILED

❯ ls Tests
573277.dll            GCSimulator.dll        LargeObjectAlloc3.dll       PlugGaps.dll           TestLibrary.pdb                 concurrentspin2.dll
573277.pdb            GCSimulator.pdb        LargeObjectAlloc3.pdb       PlugGaps.pdb           TestLibrary.runtimeconfig.json  concurrentspin2.pdb
DictionaryGrowth.dll  GCVariant.dll          LargeObjectAlloc4.dll       RedBlackTree.dll       ThdTreeGrowingObj.dll           doubLinkStay.dll
DictionaryGrowth.pdb  GCVariant.pdb          LargeObjectAlloc4.pdb       RedBlackTree.pdb       ThdTreeGrowingObj.pdb           doubLinkStay.pdb
DirectedGraph.dll     LargeObjectAlloc.dll   LargeObjectAllocPinned.dll  SingLinkStay.dll       allocationwithpins.dll          pinstress.dll
DirectedGraph.pdb     LargeObjectAlloc.pdb   LargeObjectAllocPinned.pdb  SingLinkStay.pdb       allocationwithpins.pdb          pinstress.pdb
ExpandHeap.dll        LargeObjectAlloc1.dll  LeakGenThrd.dll             StressAllocator.dll    b115557.dll                     plug.dll
ExpandHeap.pdb        LargeObjectAlloc1.pdb  LeakGenThrd.pdb             StressAllocator.pdb    b115557.pdb                     plug.pdb
GCQueue.dll           LargeObjectAlloc2.dll  MulDimJagAry.dll            TestLibrary.deps.json  bestfit-finalize.dll
GCQueue.pdb           LargeObjectAlloc2.pdb  MulDimJagAry.pdb            TestLibrary.dll        bestfit-finalize.pdb

Regression?

Unknown

Known Workarounds

No response

Configuration

Cobalt 100
Ubuntu 24.04.1 LTS

Other information

No response

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/gc
See info in area-owners.md if you want to be subscribed.

[mrsharm]

Hi @a74nh, I am working on repro'ing this issue - if you don't explicitly set suppressConsoleOutputFromTests = false, do you still observe the aforementioned error?

Additionally, it's worth stress-testing with the other configurations, as well i.e., loh.config, poh.config etc., here.

[a74nh]

Hi @a74nh, I am working on repro'ing this issue - if you don't explicitly set suppressConsoleOutputFromTests = false, do you still observe the aforementioned error?

If suppressConsoleOutputFromTests is true then I don't see any errors and the tests run ok.

[VincentBu]

Please try following steps:

1. Copy "Microsoft.Win32.Primitives.dll" from "Core_Root" to "Tests"(next to the "ReliabilityFramework.sh");
2. If there is a folder named "Tests" in "Core_Root", just delete it;
3. Start test by running ./ReliabilityFramework.sh -coreroot <your_core_root>

[a74nh]

Please try following steps:

1. Copy "Microsoft.Win32.Primitives.dll" from "Core_Root" to "Tests"(next to the "ReliabilityFramework.sh");
2. If there is a folder named "Tests" in "Core_Root", just delete it;
3. Start test by running ./ReliabilityFramework.sh -coreroot <your_core_root>

cp artifacts/tests/coreclr/linux.arm64.Checked/Tests/Core_Root/Microsoft.Win32.Primitives.dll artifacts/tests/coreclr/linux.arm64.Checked/GC/Stress/Framework/ReliabilityFramework/Tests/

That "fixes" the issue. Thanks! Not sure if this is just a workaround, or the real fix.