Gradually increasing hitching caused by massive page faults?

Starting in 4.19 (including 4.19.0, 4.19.1, and 4.19.2), my project has been having a problem of increasing intensity hitching as the game stays on. Profiling and stats have revealed very little to me other than the following:

  • Stat Renderthreadcommands shows CreateRenderThreadResources is jumping way up in time.
  • Stat Engine shows frame sync time jumping way up. (If I have “One Frame Input Lag” enabled, this doesn’t jump up, but if it’s disabled it does jump up)
  • Using Process Hacker, I realized that every time a hitch occurs, the number of page faults caused by the game jump up. It gets to the point where literally hundreds of thousands of page faults occur alongside a hitch.

When I first start the game, it’s perfectly smooth at 300+ FPS. Gradually those three metrics I listed start jumping to 5ms for a frame… then 10ms… and 30ms… and if I wait enough it even gets to 100+ms. So FPS drops from 300 to 10 or worse. Depending on what I’m doing in game, sometimes the hitches happen once a second, other times once every 10. I can do things that exacerbate it and make those frame times rise faster - like enable floating combat text and continuing to do damage to things, but the objects are in a pool and are just the same 10 being re-used over and over. I tried to start from a fresh project and put my floating combat text in place, but it did not exhibit the same hitching behavior.

If I don’t use floating combat text, it still eventually starts to hitch, but it takes a lot longer. At first I was thinking garbage collection was causing the issues, but I’ve tried tweaking that in so many ways and seen no difference in the progression of hitching.

Notably, this was not an issue when I was on 4.18. I have a copy of my project from 4.18, and the instant I update it to 4.19, it immediately exhibits the hitching/page fault issue.

Any help in troubleshooting this would be deeply appreciated!

Hello,

We’ve recently made a switch to a new bug reporting method using a more structured form. Please visit the link below for more details and report the issue using the new Bug Submission Form. Feel free to continue to use this thread for community discussion around the issue.

https://epicsupport.force.com/unrealengine/s/

Thanks

I see that link has a required field of steps to reproduce in a fresh project, but I haven’t been able to pin it down quite yet (working 60 hrs/week at my day job, and I wish I could be working with Ue4). I assume you guys don’t want me to just write “N/A” there, right?

Until I can make the issue happen in a fresh project, is there anything else I could do to provide more information to assist in troubleshooting?

Jeff (or another UE4 staff member),

Could you provide clarity regarding the follow-up question I asked 3 days ago?

Thanks!

Hey KovaaK I saw your post and just want to add that I’m having the same issue with my project too. Thought it was something that I did with my project for this to happen, I was getting the issue with using stat commands but even when not using any stat commands the hitching gets progressively worse the longer the game runs.

All I did was upgrade from 4.18 to 4.19 and this hitching issue seemed to manifest.

I did a stat DUMPHITCHES command when it got really really bad and below is what was dumped out to the log:

[Hitch Dump][1]

Just to add I use object pooling for my project, so I can’t see garbage collection being an issue, I did check the stats for that too.

240843-bth-hitching.gif

I have to say this is relieving to hear it’s happening to someone else. As a shot in the dark, what plugins are you using? Of the non-stock ones, I’m using Rama’s Victory Plugin and UWorks.

My hitch dump looks very similar to yours - https://pastebin.com/1SRLzzg4. One thing I just noticed when getting that hitch dump is that the higher the framerate, the more often the hitches occur. When I cap it to 60 FPS, I’d estimate it happens 1/5th as frequently as when I cap it at 300 FPS (linear scaling).

The only non-stock plugins I’m using are FMOD and a Object Pooling Plugin by Bruno Xavier Leite.

It’s interesting that both your game thread and render thread are at 90ms. For me or at least what the dump is showing is the render thread having a 419ms spike and the game thread at 16ms…

The hitches do seem to happen at the intervals that you mention however, I didn’t notice the regular frequency of them in editor as I run it at an unlocked framerate, but standalone I have it locked to 60 FPS and it does seem to be around every 20th frame or so it would stall.

Just that it gets worse and worse, you wouldn’t notice it or it wouldn’t happen at the beginning of the game but it got to the point where each hitch would last up to a second, which was pretty ridiculous.

I thought it was an optimisation issue which as a plus has forced me to do an optimisation pass, but even then I noticed no change in behaviour.

For me it looks like an issue with Dynamic materials, which I will take a look at in more depth, but for you something funky is happening with text rendering :confused:

Has anyone found a fix for the hitching? We have a game coming out and 4.19 is making it unplayable in editor and a shipping build. Worked great in 4.18. We are too far along in 4.19 to switch back.

I didn’t find a fix but a workaround instead. I replaced all my billboards with a plane and material and worked fine for me after. If possible I guess it’s best to do the same if not similar?

Create a blueprint that gets you the same effect as a billboard, else if you have the engine code I guess you can try to fix it yourself. A ticket has been raised with Epic, with the possible fix in 4.21 according to Unreal Engine Issues and Bug Tracker (UE-58673).

So it could be a while until this is fixed…

Use this link instead.

“I replaced all my billboards with a plane and material and worked fine for me after.”

Weird - I’m not actually using billboards, but rather the “Text Render” component just slapped on a cube. Are your planes/materials displaying text? If so, is there any chance you could share how your material is set up?

Hey everyone,

So after an entire day of running the profiler and almost giving up completely, I stumbled upon a setting nested in the Project Settings > Navigation Mesh > Do Fully Async Nav Data Gathering that resolved our constant hitching.

In our case, we are using a Persistent Level in which we stream all other levels in from, and our Nav Mesh is placed within the persistent. For some reason, having the navigation mesh not asynchronously gathering navigation data caused our project to have a hitch as if it were a heartbeat. A very consistent hitch that followed a pattern.

In any case, I do not know if this is going to help everyone else as we still notice some random hitching in the engine that doesn’t seem to have much of an explanation thus far. Hope this helped anyone struggling with hitching!

Cheers,

Andrew H.

Hi everyone,

Did anyone find a fix for this? 4.19 is hitching and stuttering for us in editor and on shipping builds. Our game is supposed to come out in a few months. This is killing it. We can’t go back to 4.18 at this point since so much has been done. Does anyone have a solution for this?

Tested it with my project, but it did not help. Upvoted because hopefully it will help someone else though.

My issue is without a doubt related to drawing 2D things in the 3D world - when I get the game to the start where hitching occurs 3 times a second, I can go into my options and turn off floating combat text + health bars, and the problem immediately stops. Turning them back on brings it back.

I just want to cross-post this forum thread where we are discussing this issue and related issues and are trying to get Epic to respond. It’s 10 pages long already…

Here’s a link:

https://forums.unrealengine.com/unreal-engine/feedback-for-epic/1457519-4-19-4-20-editor-hangs-hiccups-constantly

Unfortunately it didn’t help me either, but I would like to say it’s great to see people trying to help out like you are @AHLabrodex.

Thank you for trying and posting your results for others who may be helped by your investigative work, much appreciated!

Thanks @DotCam

I definitely can confirm the mesh particle hitching issue that also exists in 4.19. Really stinks cause our game utilizes mesh particles fairly often. :frowning:

Heya, the material I’m using is actually very simple and is a base material for what I use it for. I only use billboards for projectiles in my game.

Im confused, in my project “Do Fully Async Nav Data Gathering” is False. Should it be on?

Also when i did some profiling, i noticed something with async loading hitches. And seems that the 4.19 release notes mention alot about async loading fixes. I really think that the hitches we all see are somehow related to something with async loading.

Also worth nothing, running stat commends of any kind, ruins performance completely for my game, while running without any stat commands, it runs ok most of the time, but then suddenly every now and then i get like 1-2 secs of non stop stuttering which is really devastating in a VR game. So it’s really unpleasent to play atm. Really really hoping something is done to this issue soon.

Yes, in our project it was off by default. As soon as I enabled that setting, our constant and very noticeable frame stuttering went away. I also noticed the performance drop when attempting to profile, which kind of defeats the purpose of profiling in the first place…

What you can try is disabling realtime in your viewport, and use the ‘Play in Standalone’ option instead of the regular ‘Play in Editor’ mode. Close all your external windows and applications to make sure nothing else is affecting your project’s performance in a negative way. Then begin profiling for your game.

I did see a large number of ‘Async’ terms used in the 4.19 release notes, as well as a complete overhaul of the temporal AA upsampling which changed around the order of things in the rendering thread…perhaps it was a combination of both of these that is creating the problem. That is just a guess though. Best of luck!