Multiplayer matinee crash

anonymous_user_611d7a24 · September 10, 2014, 10:07pm

Our game uses matinees throughout to control objects and trigger events at specific times. Using a packaged build on two different machines connected by wireless ethernet, in multiplayer there is a random crash attempting to tick an AMatineeActor. All AMatineeActors are placed in level or sub-levels using editor (not manually spawned), but it appears that AMatineeActor’s World pointer is null, which causes a crash as it attempts to call UWorld::GetWorldSettings in AActor::Tick function. It is possible that crashing AMatineeActor is in a steamed-in sublevel. I assume that there are guards to prevent objects with null worlds from ticking. Is there a known cause for this kind of crash, or a way to work around it?

We are running 4.4.0 compiled from source. So far we have seen this in shipping and development builds but haven’t been able to reproduce it in debug. Here is our crash stack:

UWorld::GetWorldSettings() 0x405368cf + O bytes [File= c:\gamedev\Game\ue4\engine\source\runtime\engine\private\world.cpp:3208] [in C:\gamedev\Game\GameRelease\WindowsNoEditor\Game\Binaries\Win64\Game-Win64-Shipping.exe]
AActor::Tick() 0x402aeff7 + 19 bytes [File= c:\gamedev\Game\ue4\engine\source\runtime\engine\private\actor.cpp:754] [in C:\gamedev\Game\GameRelease\WindowsNoEditor\Game\Binaries\Win64\Game-Win64-Shipping.exe]
AMatineeActor::Tick() 0x403924a6 + O bytes [File= c:\gamedev\Game\ue4\engine\source\runtime\engine\private\interpolation.cpp:587] [in C:\gamedev\Game\GameRelease\WindowsNoEditor\Game\Binaries\Win64\Game-Win64-Shipping.exe]
AActor::TickActor() 0x402af1bd + O bytes [File=c:\gamedev\Game\ue4\engine\source\runtime\engine\private\actor.cpp:748] [in C:\gamedev\Game\GameRelease\WindowsNoEditor\Game\Binaries\Win64\Game-Win64-Shipping.exe]
TGraphTask< FTickTaskSequencer::FTickFunctionTask>::ExecuteTask() 0x40505dee + O bytes [File=c:\gamedev\Game\ue4\engine\source\runtime\core\public\async\taskgraphinterfaces.h:675] [in C:\gamedev\Game\GameRelease\WindowsNoEditor\Game\Binaries\Win64\Game-Win64-Shipping.exe]
FTaskThread::ProcessTasks() 0x3fd9a58b + O bytes [File= c:\gamedev\Game\ue4\engine\source\runtime\core\private\async\taskgraph.cpp:329] [in C:\gamedev\Game\GameRelease\WindowsNoEditor\Game\Binaries\Win64\Game-Win64-Shipping.exe]
FTaskThread::ProcessTasksUntilQuit() 0x3fd9a6cd + O bytes [File=c:\gamedev\Game\ue4\engine\source\runtime\core\private\async\taskgraph.cpp:173] [in C:\gamedev\Game\GameRelease\WindowsNoEditor\Game\Binaries\Win64\Game-Win64-Shipping.exe]
FTaskGraphlmplementation::WaitUntilTasksComplete() 0x3fda3929 + O bytes [File= c:\gamedev\Game\ue4\engine\source\runtime\core\private\async\taskgraph.cpp:865] [in C:\gamedev\Game\GameRelease\WindowsNoEditor\Game\Binaries\Win64\Game-Win64-Shipping.exe]
FTaskGraphInterface::WaitUntilTaskCompletes() 0x3fdf996d + O bytes [File=c:\gamedev\Game\ue4\engine\source\runtime\core\public\async\taskgraphinterfaces.h:183] [in C:\gamedev\Game\GameRelease\WindowsNoEditor\Game\Binaries\Win64\Game-Win64-Shipping.exe]
FTickTaskSequencer::ReleaseTickGroup() 0x40518ae6 + 206 bytes [File= c:\gamedev\Game\ue4\engine\source\runtime\engine\private\ticktaskmanager.cpp:181] [in C:\gamedev\Game\GameRelease\WindowsNoEditor\Game\Binaries\Win64\Game-Win64-Shipping.exe]
FTickTaskManager::RunTickGroup() 0x4051b32f + O bytes [File= c:\gamedev\Game\ue4\engine\source\runtime\engine\private\ticktaskmanager.cpp:714] [in C:\gamedev\Game\GameRelease\WindowsNoEditor\Game\Binaries\Win64\Game-Win64-Shipping.exe]
UWorld::Tick() 0x403dbaeb + 21 bytes [File= c:\gamedev\Game\ue4\engine\source\runtime\engine\private\leveltick.cpp:1108] [in C:\gamedev\Game\GameRelease\WindowsNoEditor\Game\Binaries\Win64\Game-Win64-Shipping.exe]
UGameEngine::Tick() 0x4035e23c + O bytes [File=c:\gamedev\Game\ue4\engine\source\runtime\engine\private\gameengine.cpp:873] [in C:\gamedev\Game\GameRelease\WindowsNoEditor\Game\Binaries\Win64\Game-Win64-Shipping.exe]
FEngineLoop::Tick() 0x3fd3b13d + O bytes [File=c:\gamedev\Game\ue4\engine\source\runtime\launch\private\launchengineloop.cpp:2098] [in C:\gamedev\Game\GameRelease\WindowsNoEditor\Game\Binaries\Win64\Game-Win64-Shipping.exe]
GuardedMain() 0x3fd3725c + O bytes [File=c:\gamedev\Game\ue4\engine\source\runtime\launch\private\launch.cpp:133] [in C:\gamedev\Game\GameRelease\WindowsNoEditor\Game\Binaries\Win64\Game-Win64-Shipping.exe]
GuardedMainWrapper() 0x3fd372aa + 5 bytes [File=c:\gamedev\Game\ue4\engine\source\runtime\launch\private\windows\launchwindows.cpp:125] [in C:\gamedev\Game\GameRelease\WindowsNoEditor\Game\Binaries\Win64\Game-Win64-Shipping.exe]
WinMain() 0x3fd3ba2O + 17 bytes [File=c:\gamedev\Game\ue4\engine\source\runtime\launch\private\windows\launchwindows.cpp:201] [in C:\gamedev\Game\GameRelease\WindowsNoEditor\Game\Binaries\Win64\Game-Win64-Shipping.exe]
_tmainCRTStartup() 0x40f5cd2d + 21 bytes [File=f:\dd\vctools\crt\crtw32\dllstuff\crtexe.c:618] [in C:\gamedev\Game\GameRelease\WindowsNoEditor\Game\Binaries\Win64\Game-Win64-Shipping.exe]
Address = 0x76c859ed (filename not found) [in C:\Windows\system32\kernel32.dll]
Address = 0x76ebc541 (filename not found) [in C:\Wind0ws\SYSTEM32\ntdll.dll]
Address = 0x76ebc541 (filename not found) [in C:\Wind0ws\SYSTEM32\ntdll.dll]

anonymous_user_611d7a24 · September 11, 2014, 7:09pm

Noticed that this happens right when transitioning to next level, so a null world might make sense.

anonymous_user_b9532c9a · September 11, 2014, 9:10pm

Hi ,

Is your game primarily C++ or blueprints? Additionally, can you reproduce this in a clean project with no additional content?

anonymous_user_33800a51 · September 11, 2014, 9:18pm

Hi ,

Thank you for your report. We have assigned this issue to a member of our support staff, and they will begin investigation as soon as possible. If we are unable to reproduce issue, or we need additional information, our staff member will comment here with additional questions. Otherwise, we will post an ‘Answer’ once we have logged issue in our bug database or we have a solution for it.

In meantime, please be sure to review our suggestions for how to report a bug, and feel free to edit your post if you have additional information to provide:

Please be sure to include crash logs and dump files for your project after it crashes.

Thanks!

anonymous_user_611d7a24 · September 11, 2014, 11:59pm

There is an even mix of C++ and blueprints. All matinee stuff is blueprints and almost all multiplayer connection logic is in blueprints.

I created a simplified test project with multiplayer and matinee but haven’t been able to reproduce crash yet.

anonymous_user_b9532c9a · September 12, 2014, 3:59pm

Hi ,

Can you post your crash logs here so we can take a look? Thank you!

anonymous_user_611d7a24 · September 12, 2014, 6:26pm

Here are crash dump and logs. This happened on slave machine. I haven’t had success with a test project, so maybe frame rate or level streaming are part of equation. last matinee before going to results map (using blueprint command Open Level) is in a streamed-in sublevel.
link text

anonymous_user_b9532c9a · September 12, 2014, 6:45pm

Hi ,

Are you calling your matinee in a sublevel or do you have it in your persistent level?

anonymous_user_611d7a24 · September 12, 2014, 7:01pm

All matinees are in sublevels and played in sublevel’s level blueprint. play happens in response to a Remote Event call from persistent level. When matinee finishes, it uses a Remote Event call to an event in persistent level to stream and start playing next sublevel. Or in case of last sublevel, load results map.

anonymous_user_b9532c9a · September 12, 2014, 7:05pm

Hi ,

Move script that plays blueprint to persistent level. See if that fixes crash.

anonymous_user_b9532c9a · September 19, 2014, 5:34pm

Hi ,

We have not heard from you in quite some time. I am marking this thread as answered for tracking purposes. If you are still experiencing this error, please respond here with information requested.

anonymous_user_611d7a24 · September 22, 2014, 7:51pm

Moving matinee to persistent level may work, but our levels are too complex and matinee is touching too many things to be able to move. Is there something else we can try?

anonymous_user_b9532c9a · September 22, 2014, 8:48pm

Hi ,

Have you tried a pause before matinee to make sure that everything loads properly? I checked and you should be able to run matinee in sublevel, but it may be trying to load everything at once and crashes due to this.

anonymous_user_611d7a24 · September 22, 2014, 9:07pm

Yes, pausing seems to help. We’ve seen issues where if server is far enough ahead of client that client isn’t loaded yet, matinee will not play at all. Determining when both server and client are ready and signaling server to unpause is a complicated problem that I spent most of last two weeks on, and haven’t come up with a solution that won’t introduce noticable pauses even when systems are already synched up. Is there a built-in feature that I overlooked that makes it easy to determine if client and server sublevels are ready or executing?

anonymous_user_b9532c9a · September 22, 2014, 9:13pm

Hi ,

There isn’t anything pre-built that I am aware of, however you can build a setup similar to this:

Note, you may need to add a pause here as well to prevent an infinite loop or an extended delay from occurring. Place this in your persistent level and let me know if it helps.

anonymous_user_611d7a24 · September 22, 2014, 9:33pm

Since playing matinee is a server-only command, is there an easy way to communicate to server that client is ready, that won’t get messed up if server isn’t ready yet?

anonymous_user_e3515d22 · September 22, 2014, 9:37pm

Hello, I work with . Unfortunately, pausing isn’t an option for us because our application is an interactive ride that needs to play from start to finish as one continuous user experience.

anonymous_user_b9532c9a · September 22, 2014, 9:49pm

I would actually ask that on forums. There are many users who have a great grasp on networking. You can post it in blueprint scripting section at http://forums.unrealengine.com.