Skip to content

Race condition in Akka.Remote RemoteActorRefProvider initialization #7966

@gunjambi

Description

@gunjambi

Version Information
Akka.Remote 1.5.52

Describe the bug
On Akka.Remote, the RemoteActorRefProvider initialization has a race condition in logging adapter creation. The impact of this race are potentially inconsistent log source naming of the Terminator actor.

This also causes a spurious NRE which is caught and handled, but this causes noise if debugging with break-on-first-chance-exception mode enabled.

The Race

public virtual void Init(ActorSystemImpl system)
{
_system = system;
_actorRefResolveThreadLocalCache = ActorRefResolveThreadLocalCache.For(system);
_actorPathThreadLocalCache = ActorPathThreadLocalCache.For(system);
_local.Init(system);
_remotingTerminator =
_system.SystemActorOf(
RemoteSettings.ConfigureDispatcher(Props.Create(() => new RemotingTerminator(_local.SystemGuardian))),
"remoting-terminator");
_internals = CreateInternals();
_remotingTerminator.Tell(RemoteInternals);
Transport.Start();
_remoteWatcher = CreateRemoteWatcher(system);
_remoteDeploymentWatcher = CreateRemoteDeploymentWatcher(system);
}

RemotingTerminator is initialized during Provider Start. RemotingTerminator inherits from FSM and FSM constructor initializes the _log with context.GetLogger() which eventually leads into LogSource.FromActorRef(). This method attempts to access Provider.DefaultAddress, which is potentially still being initialized.

If Provider Init() loses the race, either its RemoteInternals is not set, causing NRE, or if Transport.Start() is not completed, the _defaultAddress
is not set and returns null.

Additional context

The RemotingTerminator itself handles the log creation carefully avoiding this issue, and refers to the issue #4530. What's missing here is that the base FSM class doesn't do this as carefully.

public RemotingTerminator(IActorRef systemGuardian)
{
_systemGuardian = systemGuardian;
// can't use normal Logger.GetLogger(this IActorContext) here due to https://github.com/akkadotnet/akka.net/issues/4530
_log = Logging.GetLogger(Context.System.EventStream, "remoting-terminator");
InitFSM();
}

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions