Instanced foliage is causing massive performance drop when changing visibility of ordinary static meshes

Hello,

I’ve discovered an issue which is causing the Instanced Foliage Actor (maybe also some other actors that are based on instancing but I haven’t verified it, instanced foliage is 100% sure to be causing this) to heavily affect the performance (mainly on GPU) when toggling visibility of ordinary static meshes (probably for those that are in close proximity to this foliage).

I’ve discovered this since, with my team, in our game we are using a lot of spline meshes and we have developed a custom solution that is replacing the spline meshes with static meshes when they are far away from the player to reduce the draw calls and to increase performance. On UE4 this was working very well and we had no issues, but when we switched to UE5, we had massive lag when our pawn was moving and the view was changing, so we had to temporarily disable the mentioned solution for the spline meshes. I’ve recently started to investigate it and I found out that the foliage is the reason of the huge performance drop.

It looks like it has probably something to do with the changes in the GPUScene class. When the static mesh visibility is being changed, its scene info is being either removed or added and the primitive is added as a primitive to update. Those primitives are being processed in the FScene::UpdateAllPrimitiveSceneInfos, and here, inside the FScene_SwapPrimitiveSceneInfos scoped events, the swap may cause more primitives to be marked to be updates as well. If primitives related to the part of the instanced foliage will be marked this way, then during the GPU Scene Update, for the FGPUScene::UploadGeneral call it will re-upload all such instances related to the updated primitive to the GPU, and since there will be foliage instances, there will be tens of thousands or even hundreds of thousands instances that are uploaded to the GPU this way (NumInstanceDataUploads becomes very large). I’ve compared this file with version from UE 4.26.2 and instances uploading was not present there (no NumInstanceDataUploads) so it must be something new or that has been refactored. This is at least what seemed to be happening when I was debugging our game with our spline mesh optimization enabled, where the game was lagging causing frames to be rendered for so long that single frame could take more than a second(!) to render on a more complex map.

I managed to reproduce this issue (although on a smaller performance impact scale) on an empty project. I wrote a very simple script that every frame selects one mesh from array of specified meshes and toggles its visibility. I’ve placed foliage on the basic floor mesh using Foliage Editing Mode (around 70k instances), and also put around 160 actors with mesh that were selected as actors that will have their visibility toggled by this test script. I was facing the view in the opposite direction than the foliage was placed to show that this is not related to the fact of the foliage being in the view frustum. Here are the performance result:

Script enabled, foliage present:

Script disabled, foliage present:

Script enabled, foliage removed:

Script disabled, foliage removed:

As you can se, the performance drop is only here if both the script is enabled and so it is toggling a single mesh visibility every frame (the foliage is not being changed or affected and the script itself is only changing the UE’s MatPreviewMesh which are just Static Mesh Actors) and the foliage instances are present on the map. If the script is disabled or the the foliage is removed, the performance is almost the same in such cases.

Is there any way we can currently prevent this issue to happen? Maybe there is a setting or a CVar that we are missing or can change that will help with the instancing affecting the performance in a such heavy way? Has anyone else encountered this problem or managed to already fix it? Should I also report this bug via the Issue tracker or mentioning it here would be sufficient for Epic Games to notice it? I really hope that this can be fixed since even toggling visibility of a single mesh with a foliage present on a map can cause a spike on the performance and this prevents us from performing any complex operations on mesh visibility due to performance reasons, including the mentioned system for reducing spline mesh overhead and there was no such performance impact on UE4 in that case.

Thanks in advance for any suggestions.

5 Likes

Having a similiar problem, but I’m peurposefully changing the foliage actors visibility. When I start the game, all is ok. When I change foliage the invisible, it’s still ok. But when I bring it back framerate drops to 5-10 fps…

In the source code Engine\Source\Runtime\Renderer\Private\GPUScene.cpp for 4.26/4.27/UE5.0, there are CVar functions
r.GPUScene.UploadEveryFrame=0 // that means EveryFrame is uploaded

r.GPUScene.ValidatePrimitiveBuffer = 0 // that means not validating GPU buffer, but if you do set this performance does not exist, maybe it’s for RTX debug?

r.GPUScene.MaxPooledUploadBufferSize= 256000 //Maximum size of GPU Scene upload buffer size to pool
The code for UE 5 is here same as 4.27

I am not anywhere near an expert in UE5 just an indie coder and VS2022 fan!
So I just know where to find the code.
The code changes considerably
the real problem is that the code is checked in GPU, presumably for RTX?.
Rather worrying is for some reason the logging has been commented out

If you uncommented this you could at least see if this was the code causing the problem .

I have no way of checking as I am not a graphics designer and don’t build my own graphics projects just test others

Can I also suggest you look at
ShadowDepths in GPU visualizer hurting performance

The YouTube video is really interesting by

sg.shadowquality 0 where value is between 1 and 4

I know I get a 25% increase and the example is for UE4, but this works with UE5 as well WITH Nanite and Lumen and it’s really sg.shadowquality that’s the problem. When you pull out from the temple you get 78FPS on my under minimum spec GeForce GTX 1050
This is Sun Temple running twice FPS with StatGroup_GPU


All that’s needed is to learn LumenSceneLighting

Maybe this affects your other question
Foliage light problem depending on the orientation of the camera - Unreal Engine 5 Early Access

If you want to build UE5 code please see the post on how to
Visual Studio 2022 builds and runs UE5.0

Hi,

thanks for the answer. I’ve quickly tested your suggestions.

The first two CVars are disabled (set to false - 0) by default. Enabling (setting to true - 1) the first one cause even further performance decrease (which is expected based on what it does). Enabling the second one causes either another performance drop or triggers the assertion inside the related code and makes the project crash (it depends on the map which is chosen for the tests). As for the third one, it seems to have no effect on performance no matter what value do I set (either a very low or a very high).

But still, the check is only performed if the GGPUSceneValidatePrimitiveBuffer is set to true which by default is set to false so the check is not actually performed, so this can’t be (and isn’t) the source of the problem.

I’ve checked this as well and yes, it seems to affect the performance but it’s not connected to the issue described here. Setting sg.shadowquality to 0 improves the overall performance but then I still receive massive performance drops when toggling the visibility of single meshes with foliage placed around the level (in worst case I have drops from 120FPS to 10FPS) and it could be easily verified as apart from the test project I described here, in my actual project I had a solution which toggles the meshes visibility based on the player movement, and standing still I easily can get 100+ fps but any slight movement causes this massive issue. We’ve fortunately come up recently with better solution that doesn’t actually toggles the visibility of the meshes but still I think this may become problematic again at some point.

If the shadows are the problem in case of this question then it is also another bug in UE5EA which I hope will be fixed in the upcoming release since the foliage usage shouldn’t force people to disable shadows in order to have it work properly (and in project with realistic graphics sacrificing shadows is a no-go).

As I mentioned in the other topic, I know about possibility of building custom engine source but we can’t risk using the potentially more unstable main UE5 branch.

Thanks for the feedback though and I think we just need to wait for the another stable UE5 release and hope Epic will fix all those issues and problems that the Early Access is struggling with.

Hi @Patryk_Walewski,
I have had to search for documentation on Foilage problems and it would seem Foilage is not supported in Early Access see a quote from the documentation

Does Lumen work with foliage?

Lumen Reflections should be extended to work with global illumination in lightmaps in the future, which will provide a way to further scale up render quality.

Foliage is not well supported in this Early Access build because of the
heavy reliance on downsampled rendering and temporal filters.

I found this quote by searching
Google first “unreal engine lumen foliage shadow depth”

then referring to the documentation page
Lumen Technical Details | Unreal Engine Documentation

I cannot say I understand this its too technical for me!

Ii have found FREE
interactive Open World Foliage in Props - UE Marketplace (unrealengine.com)

Project+Nature - UE Marketplace (unrealengine.com)
This is UE50EA Compatible and made by Epic MarketPlace

I have loaded the FREE EM
Dynamic Grass System Lite in Blueprints - UE Marketplace (unrealengine.com)

For the people who don’t know you load this into any compatible project like 4.26. You then Windows file explorer copy the folder from the 4.26 project folder Content\DynamicGrassLite to BLANK C:\Users\Owner\Documents\Unreal Projects\EMDynamicGrassSystem\Content\DynamicGrassLite

Now open your UE5 EA2 look in your Content folder Select “Show FPS” in my case 70 to 90 fps. I have set t.maxFPS 1000 (infinite)

Now move a Textures → Foilage into the main screen. The FPS slows whilst the shaders build. When finished FPS is back to speed and its UE5 compatible

We can only hope they fully support it in the final release later in 2022

I have now had time to follow the video
Unreal Engine 5 FOLIAGE Tutorial - YouTube

This uses Quixel Megascans Tropical Foliage in a much larger area to really push the processing. The content is compiled whilst loading and during this time FPS drops to 16FPS when all content is compiled then this goes back to 67FPS. I can move around without any problem. I have tried this on EA2 works just the same as ue5_main leading edge. Oh my graphics card is the far below Spec Nvidia GTX 1050, it works flat out at 92%

@Patryk_Walewski ,
In case you haven’t noticed BINARY 5.0 Preview 2 is out and whilst testing I have seen that some projects are working BETTER in BINARY P2 than in source built!

Yes, I noticed it and we have already moved out to the Preview version. I’ve checked and it seems that in the Preview version this issue is fixed, so I’m glad that Epic finally released an updated version. Thanks for your input in this matter :slight_smile: .