Spawned Actor Crashing Game - Problem finding root cause within actor

Development Cinematics & Media
1

Hello there! I have been working on a multiplayer game the last 8 months and everything works fine for the first minute or two. You can possess vehicles and characters, partake in their actions/abilities, points tally, widgets and overlaps work. It seems great. A minute or two in all actions start to fail and I cannot figure out the root cause. I’ve used the log to eliminate any visible runtime errors with my actors, and I used the various stat commands to isolate the mentioned problem to a specific actor. If I remove this actor from the level everything works. If I double the number of actors being spawned the problem happens twice as fast. I created a new actor and remade the blueprint from scratch, but my level had the same problem. This makes me believe the issue is in my blueprints, one of the settings for a component, or the setup actor itself.

This actor is an area of fire, measured by a collision box component. It has a root ‘Scene’ component and a ParticleSystem component as the second and third parts of the actor. After a random amount of time between 1.5-3.5 seconds, the fire burns long enough to spread. The server and clients are notified of the spread, and the actor scans a 360-degree area on the server by doing a line trace to check for impeding objects. These line traces from above are done in a square like pattern from the center, and there is a total of 8 traces done on tick the 1 time the fire is spreading.

file:///C:/Users//AppData/Local/Temp/msohtmlclip1/01/clip_image001.png

If there are none present a new fire cell is generated in the green area. That new fire cell will do this same 8-point check 1.5-3.5 seconds later. These line traces check for previously burned areas and do not burn back on themselves. So, if I place 2 of these in the level at begin play, they spawn 8 more fire cells, and then burn out a few seconds later. An invisible actor (Collision box) is placed as a place holder to prevent any fire from burning back in the area for a while. Those 8 cells do a scan detecting each other and the still burning fire/ invisible marker. They then spread to any open area. I do not believe the invisible marker or spawning of that actor causes the problem, because I have bypassed/removed the blueprints for this spawning, and the problem still occurs.

When things start to break like vehicle possession, widget displays, the server taking input keys and such, the fire continues to spread on the clients and appears to perform like it should over the landscape. It spawns more actors visible to the server and client, crawling over the landscape in the proper position, up and down hills. If I fly around in my helicopter (having got in before the break occurs) watching the fire and press stat FPS, I get 30-35 FPS. This is with over 2000 fire cells on the map.
In an effort to isolate the problem within the actor further, I bypassed each function (Other than the build surrounding locations function,) and let the actor run on 90% functionality. Inputs and actions still break a minute into the level no matter what is bypassed. I used the startfile and stopfile to look at the profile, but I do not see any culprits. Using all the stat commands didn’t provide me a direction within the actor on what I should look at. Admittedly I don’t know by default what these times should be. This is the first game I’ve ever made and my first attempt at using unreal engine. But I was wondering if:

  1. Does anyone see a problem within my branch of building one of the surrounding locations? There are 8 in a sequence node which I have a larger picture of (8 branches are identical (Does running 8 identical lines of code trigger a loop problem?))
  2. Are there any issues with my server/client/multicast calls in the event graph?
  3. Other than the ‘-log’ command on the server and stat commands, is there anything else I can use to find a problem within an actor?

I feel if I can figure out what is causing this process to break, I’ll have a game ready for alpha testing. I’m pretty tech savvy and run a team of Sr. Sys Admins so I am knowledgeable around tech, but this is not my usual line of work. Since I was running out of ideas, I thought I would come ask you all! I”ve spent weeks trying to figure out the problem with this 1 actor and don’t seem to be able to do it. When using the stat commands after inputs, animations, and such break, it appears like the server continues on processing commands ok without spikes. The fire spreads like it should and spawning 200 actors, 30 seconds into the game, should not crash it. I just feel stuck! Thanks for the time spent reading!

2

I don’t think you need Components to replicate.

The CheckFlame function, I would rather suggest to have a MasterFlameBP, with all the flame types being childs of, makes it easier to just check for the MasterFlameBP, then you could set an integer for the flame BP type. To make all these casts on fail doesn’t seem right.

Also I would begin with looking at all the involved code, cleanup the blueprints, add PrintStrings to check how far the BPs run without crashing…

3

Thanks, I appreciate the advice and will take a look at these ideas this morning. I had the components replicating because originally my actor was only appearing on the server. I think I can undo this now for the components, because I believe I fixed the issue when I added my multicast function into the event graph.

I have not been using master/child blueprints, but will give that a test as well. I could probably reduce those 8 branches to 1, and pass the function a location. I used the line trace / casting to detect actors because it seemed to give me better performance versus overlap detection and other methods when I was watching my fps and stats. When it detects nothing there (cast fails) it is safe to proceed down the line of code, otherwise I need it to break out of the function. But I have to be able to detect one of these 6-7 impeding objects that the casts go to for each fire cell.

If I assign an input key to show me FPS and the number of Flame actors in game, it continues to work and spawn/spread the fire and shows me correct numbers via print string. If I get in my helicopter before things break, I can fly to the fire and put out the fire, repeat, and play the game like its working. Strings print, and the game appears to continue, but as soon as I exit my character possession breaks, widgets stop showing info, and the problems appear.

In the above situation where I am flying around, other participating players, and new players who join the server, experience the bug and can not possess vehicles while I am flying around. If other players join the server and get in/out of vehicles and such in the first minute, it works for them. And thank you so much for taking the time to read my long post!

4

I just wanted to provide an update. I removed the component replication and tested. Input actions, widgets, possessions now seem to work past the point of failure. I tested most functionality but did notice my FPS drop out of the 30-35 range and get down to around 22-23 with 300-400 fire cells on the map, and down to 18 fps at 500-600. Using ‘stat unit’ and stat uobjects it appears the flame actor is still the issue. So I shouldn’t need to adjust any other parts of the game. Unit23’s recommendation of cleaning up the blueprints seems to be the next logical move to get this fire spreading across the map. I will try to use the profiler with this new information, I just don’t know much of what I am looking for and last time was following Epic’s guide on improving the game thread.