Benchmarks sporadically failing on Executable <GUID>Emitted.dll not found

[fanasere]

hi
I am running multiple benchmarks with InProcessEmitToolchain running in net7.0 framework
and I'm sporadically (different benchmark every time) getting error:

// Benchmark Process Environment Information:
// Runtime=.NET 7.0.1 (7.0.122.56804), X64 RyuJIT AVX2
// GC=Concurrent Workstation
// HardwareIntrinsics=AVX2,AES,BMI1,BMI2,FMA,LZCNT,PCLMUL,POPCNT,AvxVnni,SERIALIZE VectorSize=256
// Job: Job-RFHJAM(Platform=X64, Runtime=.NET 7.0, Toolchain=InProcessEmitToolchain, IterationCount=3, LaunchCount=1, WarmupCount=3)

OverheadJitting 1: 1 op, 373900.00 ns, 373.9000 us/op

System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation.
---> System.NullReferenceException: Object reference not set to an instance of an object.
at Microsoft.SqlServer.SqlThreatProtectionService.TdService.Benchmarks.DetectionStorageBenchmarks.ObfuscatedQueryStorageOuterStatementFirstBenchmark.GetObfuscatedQuery() in C:\dev\ads-tpscore\src\TdService\Benchmarks\DetectionStorageBenchmarks\ObfuscatedQueryStorageOuterStatementFirstBenchmark.cs:line 90
at BenchmarkDotNet.Autogenerated.Runnable_4.WorkloadActionNoUnroll(Int64 invokeCount)
at BenchmarkDotNet.Engines.Engine.RunIteration(IterationData data)
at BenchmarkDotNet.Engines.EngineFactory.Jit(Engine engine, Int32 jitIndex, Int32 invokeCount, Int32 unrollFactor)
at BenchmarkDotNet.Engines.EngineFactory.CreateReadyToRun(EngineParameters engineParameters)
at BenchmarkDotNet.Autogenerated.Runnable_4.Run(BenchmarkCase benchmarkCase, IHost host)
at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
at System.Reflection.MethodInvoker.Invoke(Object obj, IntPtr* args, BindingFlags invokeAttr)
--- End of inner exception stack trace ---
at System.Reflection.MethodInvoker.Invoke(Object obj, IntPtr* args, BindingFlags invokeAttr)
at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
at System.Reflection.MethodBase.Invoke(Object obj, Object[] parameters)
at BenchmarkDotNet.Toolchains.InProcess.Emit.Implementation.RunnableProgram.Run(BenchmarkId benchmarkId, Assembly partitionAssembly, BenchmarkCase benchmarkCase, IHost host)
Executable C:\dev\ads-tpscore\bin\release\TdService.Benchmarks\net7.0\win-x64\bbdca8b1-3e76-45d9-b910-fa15822c521bEmitted.dll not found

what can cause the "Emitted.dll" to not be present some of the times?
the same benchmark sometimes succeeds and sometimes fails on this error
any help would be appreciated

[fanasere]

I managed to fix my problem by changing my [GlobalSetup] function from an "async task function()" to a "void function()"
my benchmark was sporadically failing because it did not wait for my [GlobalSetup] function to complete, if my [GlobalSetup] finished before the benchmark started it would succeed if not it would fail.
I am using benchmark dotnet version 0.13.3.
Are you aware of this issue?

[timcassell]

#1738 (with fix pending review #2109)

[fanasere]

thank you