Skip to content

Potential deadlock on EF Core instrumentation #1381

New issue
New issue
Open
Open
Potential deadlock on EF Core instrumentation#1381
Labels
comp:instrumentation.entityframeworkcoreThings related to OpenTelemetry.Instrumentation.EntityFrameworkCore
@Jossec101

Description

@Jossec101
Jossec101
opened on Sep 29, 2023

Issue with OpenTelemetry.Instrumentation.EntityFrameworkCore

We are running a ASP.NET Core 6 app with OpenTelemetry.Instrumentation.EntityFrameworkCore, after a dotnet-dump session we found it has a lock which was making our app freeze.

List of all OpenTelemetry NuGet packages and version that you are using (e.g. OpenTelemetry 1.3.2):

  • OpenTelemetry.Instrumentation.EntityFrameworkCore - Version 1.0.0-beta.3
  • OpenTelemetry.Exporter.Console - Version 1.5.1
  • OpenTelemetry.Exporter.OpenTelemetryProtocol - Version 1.5.1
  • OpenTelemetry.Extensions.Hosting - Version 1.5.1
  • OpenTelemetry.Instrumentation.AspNetCore - Version 1.5.1-beta.1
  • OpenTelemetry.Instrumentation.Hangfire - Version 1.5.0-beta.1
  • OpenTelemetry.Instrumentation.Runtime - Version 1.5.1
  • Npgsql.EntityFrameworkCore.PostgreSQL - Version 6.0.7

Runtime version: net6.0

Is this a feature request or a bug?

  • Feature Request
  • Bug

What is the expected behavior?

No freezing

What is the actual behavior?

Over 65% of the following dump threads are blocked waiting for a lock to be released, this makes the app freeze and hangs for minutes (actually we dont know if it actually recovers).

Example thread stuck waiting:

OS Thread Id: 0x1bf
        Child SP               IP Call Site
00007F7E569EDBB0 00007f81a5be07b2 [HelperMethodFrame_1OBJ: 00007f7e569edbb0] System.Threading.Monitor.ReliableEnter(System.Object, Boolean ByRef)
00007F7E569EDD00 00007F81341485CB OpenTelemetry.SimpleExportProcessor`1[[System.__Canon, System.Private.CoreLib]].OnExport(System.__Canon)
00007F7E569EDD90 00007F81338CC976 System.Diagnostics.SynchronizedList`1[[System.__Canon, System.Private.CoreLib]].EnumWithAction(System.Action`2<System.__Canon,System.Object>, System.Object) [/_/src/libraries/System.Diagnostics.DiagnosticSource/src/System/Diagnostics/ActivitySource.cs @ 467]
00007F7E569EDDE0 00007F8134144377 System.Diagnostics.Activity.Stop() [/_/src/libraries/System.Diagnostics.DiagnosticSource/src/System/Diagnostics/Activity.cs @ 743]
00007F7E569EDE00 00007F8134101266 OpenTelemetry.Instrumentation.EntityFrameworkCore.Implementation.EntityFrameworkDiagnosticListener.OnCustom(System.String, System.Diagnostics.Activity, System.Object)
00007F7E569EE090 00007F81340FF88D OpenTelemetry.Instrumentation.DiagnosticSourceListener.OnNext(System.Collections.Generic.KeyValuePair`2<System.String,System.Object>)
00007F7E569EE0E0 00007F81338D41BB System.Diagnostics.DiagnosticListener.Write(System.String, System.Object) [/_/src/libraries/System.Diagnostics.DiagnosticSource/src/System/Diagnostics/DiagnosticListener.cs @ 262]
00007F7E569EE110 00007F81340FF690 Microsoft.EntityFrameworkCore.Diagnostics.IDiagnosticsLogger.DispatchEventData(Microsoft.EntityFrameworkCore.Diagnostics.EventDefinitionBase, Microsoft.EntityFrameworkCore.Diagnostics.EventData, Boolean, Boolean) [/_/src/EFCore/Diagnostics/IDiagnosticsLogger.cs @ 100]
00007F7E569EE150 00007F8134147F56 Microsoft.EntityFrameworkCore.Diagnostics.Internal.RelationalCommandDiagnosticsLogger.CommandReaderExecutedAsync(Microsoft.EntityFrameworkCore.Storage.IRelationalConnection, System.Data.Common.DbCommand, Microsoft.EntityFrameworkCore.DbContext, System.Guid, System.Guid, System.Data.Common.DbDataReader, System.DateTimeOffset, System.TimeSpan, Microsoft.EntityFrameworkCore.Diagnostics.CommandSource, System.Threading.CancellationToken) [/_/src/EFCore.Relational/Diagnostics/Internal/RelationalCommandDiagnosticsLogger.cs @ 895]
00007F7E569EE2A0 00007F813411E2F5 Microsoft.EntityFrameworkCore.Storage.RelationalCommand+<ExecuteReaderAsync>d__19.MoveNext() [/_/src/EFCore.Relational/Storage/RelationalCommand.cs @ 598]
00007F7E569EE620 00007F81340D3C09 System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object) [/_/src/libraries/System.Private.CoreLib/src/System/Threading/ExecutionContext.cs @ 183]
00007F7E569EE670 00007F813411DA12 System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1+AsyncStateMachineBox`1[[System.__Canon, System.Private.CoreLib],[Microsoft.EntityFrameworkCore.Storage.RelationalCommand+<ExecuteReaderAsync>d__19, Microsoft.EntityFrameworkCore.Relational]].MoveNext(System.Threading.Thread) [/_/src/libraries/System.Private.CoreLib/src/System/Runtime/CompilerServices/AsyncTaskMethodBuilderT.cs @ 324]
00007F7E569EE6C0 00007F81340D6A91 System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(System.Runtime.CompilerServices.IAsyncStateMachineBox, Boolean) [/_/src/libraries/System.Private.CoreLib/src/System/Threading/Tasks/TaskContinuation.cs @ 795]
00007F7E569EE700 00007F813407835B System.Threading.Tasks.Task.RunContinuations(System.Object) [/_/src/libraries/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs @ 3374]
00007F7E569EE780 00007F81340E5745 System.Threading.Tasks.Task`1[[System.__Canon, System.Private.CoreLib]].TrySetResult(System.__Canon) [/_/src/libraries/System.Private.CoreLib/src/System/Threading/Tasks/Future.cs @ 400]
00007F7E569EE7B0 00007F813411D850 Npgsql.NpgsqlCommand+<ExecuteDbDataReaderAsync>d__109.MoveNext()
00007F7E569EE7F0 00007F81340D3C09 System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object) [/_/src/libraries/System.Private.CoreLib/src/System/Threading/ExecutionContext.cs @ 183]
00007F7E569EE840 00007F81341479C2 System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1+AsyncStateMachineBox`1[[System.__Canon, System.Private.CoreLib],[Npgsql.NpgsqlCommand+<ExecuteDbDataReaderAsync>d__109, Npgsql]].MoveNext(System.Threading.Thread) [/_/src/libraries/System.Private.CoreLib/src/System/Runtime/CompilerServices/AsyncTaskMethodBuilderT.cs @ 324]
00007F7E569EE890 00007F81340D6A91 System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(System.Runtime.CompilerServices.IAsyncStateMachineBox, Boolean) [/_/src/libraries/System.Private.CoreLib/src/System/Threading/Tasks/TaskContinuation.cs @ 795]
00007F7E569EE8D0 00007F813407835B System.Threading.Tasks.Task.RunContinuations(System.Object) [/_/src/libraries/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs @ 3374]
00007F7E569EE950 00007F81340E5745 System.Threading.Tasks.Task`1[[System.__Canon, System.Private.CoreLib]].TrySetResult(System.__Canon) [/_/src/libraries/System.Private.CoreLib/src/System/Threading/Tasks/Future.cs @ 400]
00007F7E569EE980 00007F8132F31B6C Npgsql.NpgsqlCommand+<ExecuteReader>d__116.MoveNext()
00007F7E569EEA80 00007F81340D3C09 System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object) [/_/src/libraries/System.Private.CoreLib/src/System/Threading/ExecutionContext.cs @ 183]
00007F7E569EEAD0 00007F8134147772 System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1+AsyncStateMachineBox`1[[System.__Canon, System.Private.CoreLib],[Npgsql.NpgsqlCommand+<ExecuteReader>d__116, Npgsql]].MoveNext(System.Threading.Thread) [/_/src/libraries/System.Private.CoreLib/src/System/Runtime/CompilerServices/AsyncTaskMethodBuilderT.cs @ 324]
00007F7E569EEB20 00007F81340D6A91 System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(System.Runtime.CompilerServices.IAsyncStateMachineBox, Boolean) [/_/src/libraries/System.Private.CoreLib/src/System/Threading/Tasks/TaskContinuation.cs @ 795]
00007F7E569EEB60 00007F813407835B System.Threading.Tasks.Task.RunContinuations(System.Object) [/_/src/libraries/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs @ 3374]
00007F7E569EEBE0 00007F813410F974 System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1[[System.Boolean, System.Private.CoreLib]].SetExistingTaskResult(System.Threading.Tasks.Task`1<Boolean>, Boolean) [/_/src/libraries/System.Private.CoreLib/src/System/Runtime/CompilerServices/AsyncTaskMethodBuilderT.cs @ 446]
00007F7E569EEC20 00007F8132F3A476 Npgsql.NpgsqlDataReader+<NextResult>d__47.MoveNext()
00007F7E569EEE70 00007F81340D3C09 System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object) [/_/src/libraries/System.Private.CoreLib/src/System/Threading/ExecutionContext.cs @ 183]
00007F7E569EEEC0 00007F813414749A System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1+AsyncStateMachineBox`1[[System.Boolean, System.Private.CoreLib],[Npgsql.NpgsqlDataReader+<NextResult>d__47, Npgsql]].MoveNext(System.Threading.Thread) [/_/src/libraries/System.Private.CoreLib/src/System/Runtime/CompilerServices/AsyncTaskMethodBuilderT.cs @ 324]
00007F7E569EEF10 00007F81340D6A91 System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(System.Runtime.CompilerServices.IAsyncStateMachineBox, Boolean) [/_/src/libraries/System.Private.CoreLib/src/System/Threading/Tasks/TaskContinuation.cs @ 795]
00007F7E569EEF50 00007F813407835B System.Threading.Tasks.Task.RunContinuations(System.Object) [/_/src/libraries/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs @ 3374]
00007F7E569EEFD0 00007F81340E5745 System.Threading.Tasks.Task`1[[System.__Canon, System.Private.CoreLib]].TrySetResult(System.__Canon) [/_/src/libraries/System.Private.CoreLib/src/System/Threading/Tasks/Future.cs @ 400]
00007F7E569EF000 00007F81328D23A4 Npgsql.Internal.NpgsqlConnector+<<ReadMessage>g__ReadMessageLong|215_0>d.MoveNext()
00007F7E569EF140 00007F81340D3C09 System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object) [/_/src/libraries/System.Private.CoreLib/src/System/Threading/ExecutionContext.cs @ 183]
00007F7E569EF190 00007F81340FE2A2 System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1+AsyncStateMachineBox`1[[System.__Canon, System.Private.CoreLib],[Npgsql.Internal.NpgsqlConnector+<<ReadMessage>g__ReadMessageLong|215_0>d, Npgsql]].MoveNext(System.Threading.Thread) [/_/src/libraries/System.Private.CoreLib/src/System/Runtime/CompilerServices/AsyncTaskMethodBuilderT.cs @ 324]
00007F7E569EF1E0 00007F81340D6A91 System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(System.Runtime.CompilerServices.IAsyncStateMachineBox, Boolean) [/_/src/libraries/System.Private.CoreLib/src/System/Threading/Tasks/TaskContinuation.cs @ 795]
00007F7E569EF220 00007F813407835B System.Threading.Tasks.Task.RunContinuations(System.Object) [/_/src/libraries/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs @ 3374]
00007F7E569EF2A0 00007F81340D6913 System.Threading.Tasks.Task`1[[System.Threading.Tasks.VoidTaskResult, System.Private.CoreLib]].TrySetResult(System.Threading.Tasks.VoidTaskResult) [/_/src/libraries/System.Private.CoreLib/src/System/Threading/Tasks/Future.cs @ 400]
00007F7E569EF2E0 00007F81328BDECD Npgsql.Internal.NpgsqlReadBuffer+<<Ensure>g__EnsureLong|41_0>d.MoveNext()
00007F7E569EF3A0 00007F81340D3C09 System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object) [/_/src/libraries/System.Private.CoreLib/src/System/Threading/ExecutionContext.cs @ 183]
00007F7E569EF3F0 00007F81340D64CA System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1+AsyncStateMachineBox`1[[System.Threading.Tasks.VoidTaskResult, System.Private.CoreLib],[Npgsql.Internal.NpgsqlReadBuffer+<<Ensure>g__EnsureLong|41_0>d, Npgsql]].MoveNext(System.Threading.Thread) [/_/src/libraries/System.Private.CoreLib/src/System/Runtime/CompilerServices/AsyncTaskMethodBuilderT.cs @ 324]
00007F7E569EF440 00007F81340D6A91 System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(System.Runtime.CompilerServices.IAsyncStateMachineBox, Boolean) [/_/src/libraries/System.Private.CoreLib/src/System/Threading/Tasks/TaskContinuation.cs @ 795]
00007F7E569EF480 00007F813407835B System.Threading.Tasks.Task.RunContinuations(System.Object) [/_/src/libraries/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs @ 3374]
00007F7E569EF500 00007F81340E5F24 System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1[[System.Int32, System.Private.CoreLib]].SetExistingTaskResult(System.Threading.Tasks.Task`1<Int32>, Int32) [/_/src/libraries/System.Private.CoreLib/src/System/Runtime/CompilerServices/AsyncTaskMethodBuilderT.cs @ 446]
00007F7E569EF540 00007F81340EDFD7 System.Net.Security.SslStream+<ReadAsyncInternal>d__188`1[[System.Net.Security.AsyncReadWriteAdapter, System.Net.Security]].MoveNext() [/_/src/libraries/System.Net.Security/src/System/Net/Security/SslStream.Implementation.cs @ 1066]
00007F7E569EF6C0 00007F81340D3C09 System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object) [/_/src/libraries/System.Private.CoreLib/src/System/Threading/ExecutionContext.cs @ 183]
00007F7E569EF710 00007F81340FD71A System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1+AsyncStateMachineBox`1[[System.Int32, System.Private.CoreLib],[System.Net.Security.SslStream+<ReadAsyncInternal>d__188`1[[System.Net.Security.AsyncReadWriteAdapter, System.Net.Security]], System.Net.Security]].MoveNext(System.Threading.Thread) [/_/src/libraries/System.Private.CoreLib/src/System/Runtime/CompilerServices/AsyncTaskMethodBuilderT.cs @ 324]
00007F7E569EF760 00007F81340D6A91 System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(System.Runtime.CompilerServices.IAsyncStateMachineBox, Boolean) [/_/src/libraries/System.Private.CoreLib/src/System/Threading/Tasks/TaskContinuation.cs @ 795]
00007F7E569EF7A0 00007F813407835B System.Threading.Tasks.Task.RunContinuations(System.Object) [/_/src/libraries/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs @ 3374]
00007F7E569EF820 00007F81340E5F24 System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1[[System.Int32, System.Private.CoreLib]].SetExistingTaskResult(System.Threading.Tasks.Task`1<Int32>, Int32) [/_/src/libraries/System.Private.CoreLib/src/System/Runtime/CompilerServices/AsyncTaskMethodBuilderT.cs @ 446]
00007F7E569EF860 00007F81340EE545 System.Net.Security.SslStream+<EnsureFullTlsFrameAsync>d__186`1[[System.Net.Security.AsyncReadWriteAdapter, System.Net.Security]].MoveNext() [/_/src/libraries/System.Net.Security/src/System/Net/Security/SslStream.Implementation.cs @ 865]
00007F7E569EF900 00007F81340D3C09 System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object) [/_/src/libraries/System.Private.CoreLib/src/System/Threading/ExecutionContext.cs @ 183]
00007F7E569EF950 00007F81340FD3AA System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1+AsyncStateMachineBox`1[[System.Int32, System.Private.CoreLib],[System.Net.Security.SslStream+<EnsureFullTlsFrameAsync>d__186`1[[System.Net.Security.AsyncReadWriteAdapter, System.Net.Security]], System.Net.Security]].MoveNext(System.Threading.Thread) [/_/src/libraries/System.Private.CoreLib/src/System/Runtime/CompilerServices/AsyncTaskMethodBuilderT.cs @ 324]
00007F7E569EF9A0 00007F81340D620D System.Net.Sockets.Socket+AwaitableSocketAsyncEventArgs.InvokeContinuation(System.Action`1<System.Object>, System.Object, Boolean, Boolean) [/_/src/libraries/System.Net.Sockets/src/System/Net/Sockets/Socket.Tasks.cs @ 1242]
00007F7E569EF9E0 00007F8134119081 System.Net.Sockets.SocketAsyncEngine.System.Threading.IThreadPoolWorkItem.Execute()
00007F7E569EFA60 00007F81340777B1 System.Threading.ThreadPoolWorkQueue.Dispatch()
00007F7E569EFAE0 00007F813414C644 System.Threading.PortableThreadPool+WorkerThread.WorkerThreadStart() [/_/src/libraries/System.Private.CoreLib/src/System/Threading/PortableThreadPool.WorkerThread.cs @ 63]
00007F7E569EFCF0 00007f81a524c2a7 [DebuggerU2MCatchHandlerFrame: 00007f7e569efcf0]

Lock:

> syncblk
Index         SyncBlock MonitorHeld Recursion Owning Thread Info          SyncBlock Owner
   99 00007F7E741A5B28          143         1 00007F811C03D2A0 1b7  51   00007f7ed47b0270 System.Object
-----------------------------
Total           5423
Free            4880

On thread 51, we found the locking came from the EntityFrameworkCore lib:

clrstack 51
OS Thread Id: 0x1b7 (51)
        Child SP               IP Call Site
00007F7E5AFF4630 00007f81a5be07b2 [HelperMethodFrame_1OBJ: 00007f7e5aff4630] System.Threading.Monitor.ObjWait(Int32, System.Object)
00007F7E5AFF4760 00007F81345CA58D System.Threading.ManualResetEventSlim.Wait(Int32, System.Threading.CancellationToken) [/_/src/libraries/System.Private.CoreLib/src/System/Threading/ManualResetEventSlim.cs @ 570]
00007F7E5AFF47E0 00007F81345C333A System.Threading.Tasks.Task.SpinThenBlockingWait(Int32, System.Threading.CancellationToken) [/_/src/libraries/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs @ 2985]
00007F7E5AFF4840 00007F813459F2C1 System.Threading.Tasks.Task.InternalWaitCore(Int32, System.Threading.CancellationToken) [/_/src/libraries/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs @ 2920]
00007F7E5AFF4890 00007F81344D0935 System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(System.Threading.Tasks.Task) [/_/src/libraries/System.Private.CoreLib/src/System/Runtime/CompilerServices/TaskAwaiter.cs @ 141]
00007F7E5AFF48B0 00007F81345B67EC Grpc.Net.Client.Internal.HttpClientCallInvoker.BlockingUnaryCall[[System.__Canon, System.Private.CoreLib],[System.__Canon, System.Private.CoreLib]](Grpc.Core.Method`2<System.__Canon,System.__Canon>, System.String, Grpc.Core.CallOptions, System.__Canon)
00007F7E5AFF4930 00007F81345B66BE Grpc.Core.Interceptors.InterceptingCallInvoker.<BlockingUnaryCall>b__3_0[[System.__Canon, System.Private.CoreLib],[System.__Canon, System.Private.CoreLib]](System.__Canon, Grpc.Core.Interceptors.ClientInterceptorContext`2<System.__Canon,System.__Canon>)
00007F7E5AFF49F0 00007F81345B6245 Grpc.Core.ClientBase+ClientBaseConfiguration+ClientBaseConfigurationInterceptor.BlockingUnaryCall[[System.__Canon, System.Private.CoreLib],[System.__Canon, System.Private.CoreLib]](System.__Canon, Grpc.Core.Interceptors.ClientInterceptorContext`2<System.__Canon,System.__Canon>, BlockingUnaryCallContinuation`2<System.__Canon,System.__Canon>)
00007F7E5AFF4AC0 00007F81345B609A Grpc.Core.Interceptors.InterceptingCallInvoker.BlockingUnaryCall[[System.__Canon, System.Private.CoreLib],[System.__Canon, System.Private.CoreLib]](Grpc.Core.Method`2<System.__Canon,System.__Canon>, System.String, Grpc.Core.CallOptions, System.__Canon)
00007F7E5AFF4BB0 00007F81345B5D46 OpenTelemetry.Proto.Collector.Trace.V1.TraceService+TraceServiceClient.Export(OpenTelemetry.Proto.Collector.Trace.V1.ExportTraceServiceRequest, Grpc.Core.Metadata, System.Nullable`1<System.DateTime>, System.Threading.CancellationToken)
00007F7E5AFF4C60 00007F81345B5A81 OpenTelemetry.Exporter.OpenTelemetryProtocol.Implementation.ExportClient.OtlpGrpcTraceExportClient.SendExportRequest(OpenTelemetry.Proto.Collector.Trace.V1.ExportTraceServiceRequest, System.Threading.CancellationToken)
00007F7E5AFF4CA0 00007F81345B4B57 OpenTelemetry.Exporter.OtlpTraceExporter.Export(OpenTelemetry.Batch`1<System.Diagnostics.Activity> ByRef)
00007F7E5AFF4D00 00007F8134148627 OpenTelemetry.SimpleExportProcessor`1[[System.__Canon, System.Private.CoreLib]].OnExport(System.__Canon)
00007F7E5AFF4D90 00007F81338CC976 System.Diagnostics.SynchronizedList`1[[System.__Canon, System.Private.CoreLib]].EnumWithAction(System.Action`2<System.__Canon,System.Object>, System.Object) [/_/src/libraries/System.Diagnostics.DiagnosticSource/src/System/Diagnostics/ActivitySource.cs @ 467]
00007F7E5AFF4DE0 00007F8134144377 System.Diagnostics.Activity.Stop() [/_/src/libraries/System.Diagnostics.DiagnosticSource/src/System/Diagnostics/Activity.cs @ 743]
00007F7E5AFF4E00 00007F8134101266 OpenTelemetry.Instrumentation.EntityFrameworkCore.Implementation.EntityFrameworkDiagnosticListener.OnCustom(System.String, System.Diagnostics.Activity, System.Object)
00007F7E5AFF5090 00007F81340FF88D OpenTelemetry.Instrumentation.DiagnosticSourceListener.OnNext(System.Collections.Generic.KeyValuePair`2<System.String,System.Object>)
00007F7E5AFF50E0 00007F81338D41BB System.Diagnostics.DiagnosticListener.Write(System.String, System.Object) [/_/src/libraries/System.Diagnostics.DiagnosticSource/src/System/Diagnostics/DiagnosticListener.cs @ 262]
00007F7E5AFF5110 00007F81340FF690 Microsoft.EntityFrameworkCore.Diagnostics.IDiagnosticsLogger.DispatchEventData(Microsoft.EntityFrameworkCore.Diagnostics.EventDefinitionBase, Microsoft.EntityFrameworkCore.Diagnostics.EventData, Boolean, Boolean) [/_/src/EFCore/Diagnostics/IDiagnosticsLogger.cs @ 100]
00007F7E5AFF5150 00007F8134147F56 Microsoft.EntityFrameworkCore.Diagnostics.Internal.RelationalCommandDiagnosticsLogger.CommandReaderExecutedAsync(Microsoft.EntityFrameworkCore.Storage.IRelationalConnection, System.Data.Common.DbCommand, Microsoft.EntityFrameworkCore.DbContext, System.Guid, System.Guid, System.Data.Common.DbDataReader, System.DateTimeOffset, System.TimeSpan, Microsoft.EntityFrameworkCore.Diagnostics.CommandSource, System.Threading.CancellationToken) [/_/src/EFCore.Relational/Diagnostics/Internal/RelationalCommandDiagnosticsLogger.cs @ 895]
00007F7E5AFF52A0 00007F813411E2F5 Microsoft.EntityFrameworkCore.Storage.RelationalCommand+<ExecuteReaderAsync>d__19.MoveNext() [/_/src/EFCore.Relational/Storage/RelationalCommand.cs @ 598]
00007F7E5AFF5620 00007F81340D3C09 System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object) [/_/src/libraries/System.Private.CoreLib/src/System/Threading/ExecutionContext.cs @ 183]
00007F7E5AFF5670 00007F813411DA12 System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1+AsyncStateMachineBox`1[[System.__Canon, System.Private.CoreLib],[Microsoft.EntityFrameworkCore.Storage.RelationalCommand+<ExecuteReaderAsync>d__19, Microsoft.EntityFrameworkCore.Relational]].MoveNext(System.Threading.Thread) [/_/src/libraries/System.Private.CoreLib/src/System/Runtime/CompilerServices/AsyncTaskMethodBuilderT.cs @ 324]
00007F7E5AFF56C0 00007F81340D6A91 System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(System.Runtime.CompilerServices.IAsyncStateMachineBox, Boolean) [/_/src/libraries/System.Private.CoreLib/src/System/Threading/Tasks/TaskContinuation.cs @ 795]
00007F7E5AFF5700 00007F813407835B System.Threading.Tasks.Task.RunContinuations(System.Object) [/_/src/libraries/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs @ 3374]
00007F7E5AFF5780 00007F81340E5745 System.Threading.Tasks.Task`1[[System.__Canon, System.Private.CoreLib]].TrySetResult(System.__Canon) [/_/src/libraries/System.Private.CoreLib/src/System/Threading/Tasks/Future.cs @ 400]
00007F7E5AFF57B0 00007F813411D850 Npgsql.NpgsqlCommand+<ExecuteDbDataReaderAsync>d__109.MoveNext()
00007F7E5AFF57F0 00007F81340D3C09 System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object) [/_/src/libraries/System.Private.CoreLib/src/System/Threading/ExecutionContext.cs @ 183]
00007F7E5AFF5840 00007F81341479C2 System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1+AsyncStateMachineBox`1[[System.__Canon, System.Private.CoreLib],[Npgsql.NpgsqlCommand+<ExecuteDbDataReaderAsync>d__109, Npgsql]].MoveNext(System.Threading.Thread) [/_/src/libraries/System.Private.CoreLib/src/System/Runtime/CompilerServices/AsyncTaskMethodBuilderT.cs @ 324]
00007F7E5AFF5890 00007F81340D6A91 System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(System.Runtime.CompilerServices.IAsyncStateMachineBox, Boolean) [/_/src/libraries/System.Private.CoreLib/src/System/Threading/Tasks/TaskContinuation.cs @ 795]
00007F7E5AFF58D0 00007F813407835B System.Threading.Tasks.Task.RunContinuations(System.Object) [/_/src/libraries/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs @ 3374]
00007F7E5AFF5950 00007F81340E5745 System.Threading.Tasks.Task`1[[System.__Canon, System.Private.CoreLib]].TrySetResult(System.__Canon) [/_/src/libraries/System.Private.CoreLib/src/System/Threading/Tasks/Future.cs @ 400]
00007F7E5AFF5980 00007F8132F31B6C Npgsql.NpgsqlCommand+<ExecuteReader>d__116.MoveNext()
00007F7E5AFF5A80 00007F81340D3C09 System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object) [/_/src/libraries/System.Private.CoreLib/src/System/Threading/ExecutionContext.cs @ 183]
00007F7E5AFF5AD0 00007F8134147772 System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1+AsyncStateMachineBox`1[[System.__Canon, System.Private.CoreLib],[Npgsql.NpgsqlCommand+<ExecuteReader>d__116, Npgsql]].MoveNext(System.Threading.Thread) [/_/src/libraries/System.Private.CoreLib/src/System/Runtime/CompilerServices/AsyncTaskMethodBuilderT.cs @ 324]
00007F7E5AFF5B20 00007F81340D6A91 System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(System.Runtime.CompilerServices.IAsyncStateMachineBox, Boolean) [/_/src/libraries/System.Private.CoreLib/src/System/Threading/Tasks/TaskContinuation.cs @ 795]
00007F7E5AFF5B60 00007F813407835B System.Threading.Tasks.Task.RunContinuations(System.Object) [/_/src/libraries/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs @ 3374]
00007F7E5AFF5BE0 00007F813410F974 System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1[[System.Boolean, System.Private.CoreLib]].SetExistingTaskResult(System.Threading.Tasks.Task`1<Boolean>, Boolean) [/_/src/libraries/System.Private.CoreLib/src/System/Runtime/CompilerServices/AsyncTaskMethodBuilderT.cs @ 446]
00007F7E5AFF5C20 00007F8132F3A476 Npgsql.NpgsqlDataReader+<NextResult>d__47.MoveNext()
00007F7E5AFF5E70 00007F81340D3C09 System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object) [/_/src/libraries/System.Private.CoreLib/src/System/Threading/ExecutionContext.cs @ 183]
00007F7E5AFF5EC0 00007F813414749A System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1+AsyncStateMachineBox`1[[System.Boolean, System.Private.CoreLib],[Npgsql.NpgsqlDataReader+<NextResult>d__47, Npgsql]].MoveNext(System.Threading.Thread) [/_/src/libraries/System.Private.CoreLib/src/System/Runtime/CompilerServices/AsyncTaskMethodBuilderT.cs @ 324]
00007F7E5AFF5F10 00007F81340D6A91 System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(System.Runtime.CompilerServices.IAsyncStateMachineBox, Boolean) [/_/src/libraries/System.Private.CoreLib/src/System/Threading/Tasks/TaskContinuation.cs @ 795]
00007F7E5AFF5F50 00007F813407835B System.Threading.Tasks.Task.RunContinuations(System.Object) [/_/src/libraries/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs @ 3374]
00007F7E5AFF5FD0 00007F81340E5745 System.Threading.Tasks.Task`1[[System.__Canon, System.Private.CoreLib]].TrySetResult(System.__Canon) [/_/src/libraries/System.Private.CoreLib/src/System/Threading/Tasks/Future.cs @ 400]
00007F7E5AFF6000 00007F81328D23A4 Npgsql.Internal.NpgsqlConnector+<<ReadMessage>g__ReadMessageLong|215_0>d.MoveNext()
00007F7E5AFF6140 00007F81340D3C09 System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object) [/_/src/libraries/System.Private.CoreLib/src/System/Threading/ExecutionContext.cs @ 183]
00007F7E5AFF6190 00007F81340FE2A2 System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1+AsyncStateMachineBox`1[[System.__Canon, System.Private.CoreLib],[Npgsql.Internal.NpgsqlConnector+<<ReadMessage>g__ReadMessageLong|215_0>d, Npgsql]].MoveNext(System.Threading.Thread) [/_/src/libraries/System.Private.CoreLib/src/System/Runtime/CompilerServices/AsyncTaskMethodBuilderT.cs @ 324]
00007F7E5AFF61E0 00007F81340D6A91 System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(System.Runtime.CompilerServices.IAsyncStateMachineBox, Boolean) [/_/src/libraries/System.Private.CoreLib/src/System/Threading/Tasks/TaskContinuation.cs @ 795]
00007F7E5AFF6220 00007F813407835B System.Threading.Tasks.Task.RunContinuations(System.Object) [/_/src/libraries/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs @ 3374]
00007F7E5AFF62A0 00007F81340D6913 System.Threading.Tasks.Task`1[[System.Threading.Tasks.VoidTaskResult, System.Private.CoreLib]].TrySetResult(System.Threading.Tasks.VoidTaskResult) [/_/src/libraries/System.Private.CoreLib/src/System/Threading/Tasks/Future.cs @ 400]
00007F7E5AFF62E0 00007F81328BDECD Npgsql.Internal.NpgsqlReadBuffer+<<Ensure>g__EnsureLong|41_0>d.MoveNext()
00007F7E5AFF63A0 00007F81340D3C09 System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object) [/_/src/libraries/System.Private.CoreLib/src/System/Threading/ExecutionContext.cs @ 183]
00007F7E5AFF63F0 00007F81340D64CA System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1+AsyncStateMachineBox`1[[System.Threading.Tasks.VoidTaskResult, System.Private.CoreLib],[Npgsql.Internal.NpgsqlReadBuffer+<<Ensure>g__EnsureLong|41_0>d, Npgsql]].MoveNext(System.Threading.Thread) [/_/src/libraries/System.Private.CoreLib/src/System/Runtime/CompilerServices/AsyncTaskMethodBuilderT.cs @ 324]
00007F7E5AFF6440 00007F81340D6A91 System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(System.Runtime.CompilerServices.IAsyncStateMachineBox, Boolean) [/_/src/libraries/System.Private.CoreLib/src/System/Threading/Tasks/TaskContinuation.cs @ 795]
00007F7E5AFF6480 00007F813407835B System.Threading.Tasks.Task.RunContinuations(System.Object) [/_/src/libraries/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs @ 3374]
00007F7E5AFF6500 00007F81340E5F24 System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1[[System.Int32, System.Private.CoreLib]].SetExistingTaskResult(System.Threading.Tasks.Task`1<Int32>, Int32) [/_/src/libraries/System.Private.CoreLib/src/System/Runtime/CompilerServices/AsyncTaskMethodBuilderT.cs @ 446]
00007F7E5AFF6540 00007F81340EDFD7 System.Net.Security.SslStream+<ReadAsyncInternal>d__188`1[[System.Net.Security.AsyncReadWriteAdapter, System.Net.Security]].MoveNext() [/_/src/libraries/System.Net.Security/src/System/Net/Security/SslStream.Implementation.cs @ 1066]
00007F7E5AFF66C0 00007F81340D3C09 System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object) [/_/src/libraries/System.Private.CoreLib/src/System/Threading/ExecutionContext.cs @ 183]
00007F7E5AFF6710 00007F81340FD71A System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1+AsyncStateMachineBox`1[[System.Int32, System.Private.CoreLib],[System.Net.Security.SslStream+<ReadAsyncInternal>d__188`1[[System.Net.Security.AsyncReadWriteAdapter, System.Net.Security]], System.Net.Security]].MoveNext(System.Threading.Thread) [/_/src/libraries/System.Private.CoreLib/src/System/Runtime/CompilerServices/AsyncTaskMethodBuilderT.cs @ 324]
00007F7E5AFF6760 00007F81340D6A91 System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(System.Runtime.CompilerServices.IAsyncStateMachineBox, Boolean) [/_/src/libraries/System.Private.CoreLib/src/System/Threading/Tasks/TaskContinuation.cs @ 795]
00007F7E5AFF67A0 00007F813407835B System.Threading.Tasks.Task.RunContinuations(System.Object) [/_/src/libraries/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs @ 3374]
00007F7E5AFF6820 00007F81340E5F24 System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1[[System.Int32, System.Private.CoreLib]].SetExistingTaskResult(System.Threading.Tasks.Task`1<Int32>, Int32) [/_/src/libraries/System.Private.CoreLib/src/System/Runtime/CompilerServices/AsyncTaskMethodBuilderT.cs @ 446]
00007F7E5AFF6860 00007F81340EE545 System.Net.Security.SslStream+<EnsureFullTlsFrameAsync>d__186`1[[System.Net.Security.AsyncReadWriteAdapter, System.Net.Security]].MoveNext() [/_/src/libraries/System.Net.Security/src/System/Net/Security/SslStream.Implementation.cs @ 865]
00007F7E5AFF6900 00007F81340D3C09 System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object) [/_/src/libraries/System.Private.CoreLib/src/System/Threading/ExecutionContext.cs @ 183]
00007F7E5AFF6950 00007F81340FD3AA System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1+AsyncStateMachineBox`1[[System.Int32, System.Private.CoreLib],[System.Net.Security.SslStream+<EnsureFullTlsFrameAsync>d__186`1[[System.Net.Security.AsyncReadWriteAdapter, System.Net.Security]], System.Net.Security]].MoveNext(System.Threading.Thread) [/_/src/libraries/System.Private.CoreLib/src/System/Runtime/CompilerServices/AsyncTaskMethodBuilderT.cs @ 324]
00007F7E5AFF69A0 00007F81340D620D System.Net.Sockets.Socket+AwaitableSocketAsyncEventArgs.InvokeContinuation(System.Action`1<System.Object>, System.Object, Boolean, Boolean) [/_/src/libraries/System.Net.Sockets/src/System/Net/Sockets/Socket.Tasks.cs @ 1242]
00007F7E5AFF69E0 00007F8134119081 System.Net.Sockets.SocketAsyncEngine.System.Threading.IThreadPoolWorkItem.Execute()
00007F7E5AFF6A60 00007F81340777B1 System.Threading.ThreadPoolWorkQueue.Dispatch()
00007F7E5AFF6AE0 00007F813414C644 System.Threading.PortableThreadPool+WorkerThread.WorkerThreadStart() [/_/src/libraries/System.Private.CoreLib/src/System/Threading/PortableThreadPool.WorkerThread.cs @ 63]
00007F7E5AFF6CF0 00007f81a524c2a7 [DebuggerU2MCatchHandlerFrame: 00007f7e5aff6cf0]

Metadata

Metadata

Assignees

No one assigned

    Labels

    comp:instrumentation.entityframeworkcoreThings related to OpenTelemetry.Instrumentation.EntityFrameworkCore

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions