Skip to content

Unobserved task exception thrown by an active client streaming operation when the server is killed  #349

Open
@tobi1449

Description

@tobi1449

When a server is killed while an client-server streaming operation is open, a GC run will cause an unobserved task exception, even if the exception thrown by the operation call itself is caught.

To reproduce:

  1. Start a streaming operation from a gRPC client to a gRPC server within a try-catch block
  2. Kill the server
  3. This results in a IncompleteSendRpcException from the gRPC call, caught properly by the try-catch block
  4. Force a GC run
  5. This results in an RpcException, only caught via TaskScheduler.UnobservedTaskException
Unobserved Task exception: System.AggregateException: A Task's exception(s) were not observed either by Waiting on the Task or accessing its Exception property. As a result, the unobserved exception was rethrown by the finalizer thread. ---> Grpc.Core.RpcException: Status(StatusCode="Unknown", Detail="Strea
m removed", DebugException="Grpc.Core.Internal.CoreErrorDetailException: {"created":"@1731967532.940000000","description":"Error received from peer ipv6:[::1]:5001","file":"..\..\..\src\core\lib\surface\call.cc","file_line":953,"grpc_message":"Stream removed","grpc_status":2}") ---> Grpc.Core.Internal.CoreErrorDetailException: {"created":"@1731967532.940000000","description":"Error received from peer ipv6:[::1]:5001","file":"..\..\..\src\core\lib\surface\call.cc","file_line":953,"grpc_message":"Stream removed","grpc_status":2}
   --- End of inner exception stack trace ---
   --- End of inner exception stack trace ---
---> (Inner Exception #0) Grpc.Core.RpcException: Status(StatusCode="Unknown", Detail="Stream removed", DebugException="Grpc.Core.Internal.CoreErrorDetailException: {"created":"@1731967532.940000000","description":"Error received from peer ipv6:[::1]:5001","file":"..\..\..\src\core\lib\surface\call.cc","fil
e_line":953,"grpc_message":"Stream removed","grpc_status":2}") ---> Grpc.Core.Internal.CoreErrorDetailException: {"created":"@1731967532.940000000","description":"Error received from peer ipv6:[::1]:5001","file":"..\..\..\src\core\lib\surface\call.cc","file_line":953,"grpc_message":"Stream removed","grpc_status":2}

I've created a small sample project at tobi1449/protobuf-net.grpc-bug.
To reproduce simply start NetFrameworkServer and NetFrameworkClient and kill the server.

As a workaround, I've found that setting the IgnoreStreamTermination flag, as described in the IncompleteSendRpcException, also suppresses the unobserved task exception.
It works for me, but the client not getting an exception or notification that the server is offline is a bit of a bummer.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions