Second level retry handler is resolved from the same ServiceScope as the handler for the last first level retry

[SamPersson]

DependencyInjectionHandlerActivator.GetOrCreateScopeAndReturnServiceProvider() creates a new AsyncServiceScope and saves it in the IncomingStepContext, unless there is one already saved. When DefaultRetryStep dispatches a second level retry, it will still use the same IncomingStepContext, which means there will always be an AsyncServiceScope saved when the handlers for the IFailed<Message> are resolved.

I guess this is usually not a big problem - second level retry handlers are usually simple and maybe only send another message. It became a problem for us when we had a scoped entity framework DbContext, and used it in the second level retry handler. If the original message failed because of some unhandled conflict, that conflicting change will still be in the change tracker of the DbContext when the second level retry handler runs, so that too will fail.

The behavior was confusing, because in other ways handling an IFailed<Message> feels just like handling a regular message.

We implemented a workaround like this for now:

    private class NewScopeForFailedIncomingStep : IIncomingStep
    {
        public Task Process(IncomingStepContext context, Func<Task> next)
        {
            if (context.Load<bool>(DefaultRetryStep.DispatchAsFailedMessageKey))
            {
                // Clear saved scope to make sure a new scope is created for the failed message handler.
                context.Save<IServiceScope?>(null);
                context.Save<AsyncServiceScope?>(null);
            }

            return next();
        }
    }

...

.OnReceive(new NewScopeForFailedIncomingStep(), PipelineRelativePosition.Before, typeof(ActivateHandlersStep))

...