Ok, so after doing some internal investigation, I think this is caused by the following lines of code and how replication graph associates actor channels with actors. Normally without the replication graph, the net driver will immediately disassociate an actor channel from an actor upon closing the actor channel for whatever reason via SetClosingFlag():
void RemoveActorChannel(AActor* Actor)
{
ActorChannels.Remove(Actor); // <-- Actor & channel pairing removed / disassociated
if (ReplicationConnectionDriver)
{
ReplicationConnectionDriver->NotifyActorChannelRemoved(Actor);
}
}
However, with a replication graph, there is an additional association inside the NetReplicationGraphConnection, which keeps a map of actors and channels. There seems to be a comment about waiting for the client to ack the channel closing before removing the association:
void UNetReplicationGraphConnection::NotifyActorChannelRemoved(AActor* Actor)
{
// No need to do anything here. This is called when an actor channel is closed, but
// we're still waiting for the close bunch to be acked. Until then, we can't safely replicate
// the actor from this channel. See NotifyActorChannelCleanedUp.
}
The association would finally get cleaned up in `NotifyActorChannelCleanedUp`. However, during this period between server closing and receiving the client ack, all reliable RPCs will get dropped. This is because the channel still exists and is in the closing state:
for (UNetReplicationGraphConnection* Manager : Connections)
{
FConnectionReplicationActorInfo& ConnectionActorInfo = Manager->ActorInfoMap.FindOrAdd(Actor);
UNetConnection* NetConnection = Manager->NetConnection;
if (NetConnection->IsClosingOrClosed())
{
return true;
}
// This connection isn't ready yet
if (NetConnection->ViewTarget == nullptr)
{
continue;
}
// Streaming level actor that the client doesn't have loaded. Do not send.
if (ActorStreamingLevelName != NAME_None && NetConnection->ClientVisibleLevelNames.Contains(ActorStreamingLevelName) == false)
{
continue;
}
// While the channel is closing we cannot send new multicast RPCs
if (ConnectionActorInfo.Channel && ConnectionActorInfo.Channel->Closing)
{
continue;
}
Therefore, the RPC has no opportunity to open a new actor channel to send the RPC. This is totally valid for actors that are falling out of relevancy, as the client will destroy these actors anyway before receiving the RPC. However, for net dormancy, I assume we want the actor to still receive reliable RPCs, as having the rpc either get received or not essentially based on the client’s time to ack to the server will create very unreliable behavior.
Our current workaround has been to just create our own custom RepliationGraphConnection class and immediately disassociate on whether the actor is dormant or not:
void UModifiedNetReplicationGraphConnection::NotifyActorChannelRemoved(AActor* Actor)
{
// Default function does not remove the channel from the actor info map until the channel
// is ready to be cleaned up. Unfortunately, this causes an issue where if a reliable RPC is called
// before the server has received the ack from the client about the closed actor channel, the rpc will
// get dropped, which is not what we want for dormant actors. So instead, I remove this from the actor
// info map immediately so that the repliation graph believes it can open a new actor channel for the actor.
Super::NotifyActorChannelRemoved(Actor);
FConnectionReplicationActorInfo* InfoMapItem = ActorInfoMap.Find(Actor);
if (InfoMapItem && InfoMapItem->bDormantOnConnection)
{
UActorChannel* Channel = InfoMapItem->Channel;
PendingCloseFromDormancyActorChannels.AddUnique(Channel);
InfoMapItem->ResetFrameCounters();
ActorInfoMap.RemoveChannel(Channel);
}
}
This feels extremely hacky and possibly buggy, but has allowed our dormant actors to receive reliable rpcs reliably.
[Attachment Removed]