Unstreaming and Streaming a level causes client actor to get destroyed and not spawn again

We have encountered a rare situation when playing our game that we have finally managed to reproduce in a vanilla Unreal project.

The project is attached in the repro steps.

You also have an mp4 file showcasing the issue.

What seems to happen is the following:

  1. We are in a situation with high ping
  2. A sublevel with an actor that is Awake for replication and has a high net frequency is loaded
  3. The level gets unstreamed
  4. Quickly the level gets re-streamed
  5. That actor gets destroyed from the client instance but never gets created again
  6. An actor channel warning appears both for that actor and for the WorldSettings

Warning:

LogNet: Warning: UActorChannel::ProcessBunch: SerializeNewActor failed to find/spawn actor. Actor: BP_ProblematicBP_C /Game/ThirdPerson/Maps/UEDPIE_1_LVL_SubLevel.LVL_SubLevel:PersistentLevel.BP_ProblematicBP_C_1, Channel: 12 LogNet: Warning: UActorChannel::ProcessBunch: SerializeNewActor failed to find/spawn actor. Actor: WorldSettings /Game/ThirdPerson/Maps/UEDPIE_1_LVL_SubLevel.LVL_SubLevel:PersistentLevel.WorldSettings, Channel: 13

Steps to Reproduce
Open the attached project.

Hit play. This will open two players and after some time you will se a level being streamed in, then out, then in again.

Observe that one of the actors is no longer there after the level has been re-stremed.

This can also be reproduced via streaming volumes and quickly walking in an out.

Hi,

Thank you for the report and for providing a repro project. I’ve opened a new issue, UE-278590, which should be visible in the public tracker in a day or so.

Also just to double check, are you able to reproduce the same issue when using World Partition’s Level Instances, particularly when using the “Standalone” level behavior? https://dev.epicgames.com/documentation/en\-us/unreal\-engine/level\-instancing\-in\-unreal\-engine\#levelstreamingmode

Thanks,

Alex

Hi,

Thank you for the additional information!

The reason I asked about World Partition is because we did recently fix an issue that was almost identical to this, but it was discovered when moving in and out of World Partition cells quickly. The solution was to make sure that actor channels skip destroying net startup actors when they are closed due to the actor’s level being unloaded (CL 24651313 in UE5/Main).

However, it seems this fix isn’t affecting your situation here. I wasn’t able to find any changes between 5.1 and 5.2 that looked to affect this behavior, but after digging more into this, I believe I’ve tracked down what is causing the problem here.

When the streaming level is unloaded on the server, we intentionally will not close the net startup actors’ channels (UNetDriver::OnLevelRemovedFromWorld). However, when the unloaded streaming level is garbage collected (which in this case appears to be happening on the next frame or so), the NetDriver will close these channels and send the close bunches to the client (UNetDriver::NotifyStreamingLevelUnload). However, the close reason passed to the client here is EChannelCloseReason::Destroyed, so when the client receives the close bunch for the actor channel, it will just destroy that actor.

This normally isn’t an issue, as the client usually will then unload and clean up the streaming level anyway, destroying all the actors in it. Then when the streaming level is reloaded, the client will recreate these actors.

However, as you’ve found, with enough latency and with a small enough delay between the server unloading and loading the level, the client won’t actually unload the streaming level, but it will still receive the close bunches for the actors and destroy them.

Later when the server tries to replicate one of these net startup actors, it expects it to already exist on the client, as it should have been loaded with the level. When the actor isn’t present on the client, the warnings and replication failures occur.

The solution here is likely to make sure the server is sending the correct EChannelCloseReason when the streaming level is unloaded. In your repro project, I tested out this change to UNetDriver::NotifyActorDestroyed, and it did resolve the issue:

if( Channel ) { check(Channel->OpenedLocally); Channel->bClearRecentActorRefs = false; // If the actor is being destroyed due to the level being unloaded, make sure we send that as the reason. // We can check IsSeamlessTravel for this, as this looks to only be set to true when NotifyActorDestroyed is called from NotifyActorLevelUnloaded. Channel->Close( IsSeamlessTravel ? EChannelCloseReason::LevelUnloaded : EChannelCloseReason::Destroyed); }That being said, this change has not been thoroughly tested, so there may be some side effects I didn’t catch. Hopefully this at least provides a better idea of the problem though.

Thanks,

Alex

As our game is using Level Streaming and not World Partition I have not tried that approach.

I can try tomorrow and come back here to respond.

I might need some help with the world partition setup ( I’m not fully familiar with the system )

Currently I managed to setup a level instance with server streaming that gets streamed in/out, but client does not seem to get those changes.

On the client the only thing I see is when hitting play, the level instance is loaded and then it gets unloaded.

Video attached.

And sorry here is with standalone Level Streaming on the Level Instance, for whatever reason the client gets disconnected and an error pops up, so most probably I have something incorrectly setup.

The only change between these two versions is the level streaming mode.

Some extra info:

Seems like things broke on 5.2, I have tested in many engine versions with the following setup:

[Image Removed]Players Join then:

  1. Stream in
  2. Stream out
  3. Re-stream in

Here are the result of what these actors do in different engine versions:

[Image Removed]

I have attached videos of 5.1 and 5.2 so that you can see it in action.

That being said we may have something extra in our code, as in vanilla unreal only the [ Awake, NetLoadOnClient, High Frequency ] breaks.

But in our game the [ Awake, HighFrequency ] also breaks, without the need of NetLoadOnClient, that being said we suspect it could be related.

One interesting difference is the actors staying in 5.2 onwards, instead of disappearing and re-appearing like in 5.1, maybe that gives us a clue on the affected code in the engine.

Attaching 5.2 version in a reply, as UDN only allows one attachment per reply.

Thanks for taking a deep look at this issue.

What I will do is integrate this change and then have our QA team do a pass on the game, and follow up here with further issues.

Thanks again for your time.

I confirm that the issue seems to be gone, if we find any extra weird things with it should I reply here or open a new post?

Hi,

Great, I’m glad that’s working for you! If you do run into any issues, feel free to reply here. This thread will be automatically closed after a long enough period of inactivity, at which point you can create a new question if needed.

Thanks,

Alex

Just to confirm, will these fixes be added to the next unreal engine release ( like 5.7 or 5.8 )?

Hi,

I can’t say for sure if this change will be added to a later release or not. We’ll need to investigate the issue further on our end and make sure any potential fix does not introduce side effects. You can keep an eye on the public link for the issue to see when/if a fix a made: Unreal Engine Issues and Bug Tracker (UE\-278590)

Thanks,

Alex