Performance issues with Nanite (Unreal Engine 5.7)

Hello!
I’m writting this because I have huge performance issues with my project when enabling nanite on at even 1 static mesh. I tried to change the nanite settings in the static mesh editor and I cleared up my UV0s to ensure better results. I also did delete DerivedDataCache and Intermediate.

I moved my project from UE5.3 to 5.7. and when I did, I edited the Engine.ini file and I restored the rendering settings too.
I have no idea why this keeps on happening… Do you have any ideas on how to fix this issue?

The crash only occurs when I open a level in Lit mode. When it’s Unlit everything works fine.

Here are the crash notes I get:

LoginId:ebcd6e0d4b637becb3d6409f5012ca6c EpicAccountId:84387fac1e0f4d82a17cf15874cf0053 Fatal error: [File:D:\build\++UE5\Sync\Engine\Source\Runtime\D3D12RHI\Private\D3D12Util.cpp] [Line: 810] Out of video memory trying to allocate a rendering resource UnrealEditor_D3D12RHI UnrealEditor_D3D12RHI UnrealEditor_D3D12RHI UnrealEditor_D3D12RHI UnrealEditor_D3D12RHI UnrealEditor_D3D12RHI UnrealEditor_D3D12RHI UnrealEditor_D3D12RHI UnrealEditor_D3D12RHI UnrealEditor_RHI UnrealEditor_RHI UnrealEditor_RHI UnrealEditor_RHI UnrealEditor_RHI UnrealEditor_RHI UnrealEditor_RHI UnrealEditor_RHI UnrealEditor_Core UnrealEditor_Core UnrealEditor_Core UnrealEditor_Core UnrealEditor_Core UnrealEditor_Core UnrealEditor_Core kernel32 ntdll

Hello there @Oski29!

Checking with my peers, this looks like a VRAM conflict, made a bit worse due the recent migration. The fact that your asset works in Unlit mode, but crashes in Lit mode, aims at Lumen consuming more VRAM than intended, on top of what Nanite’s asking too. Since there are multiple possible causes, let’s go one by one:

  • First thing to test, disable virtual shadow maps, as they are quite taxing, and we need every bit of VRAM we can spare. Simply go to Project Settings → Rendering → Direct Lighting, and switch the method to classic Shadow Maps

  • As for Lumen, the most direct approach is to play with the engine’s scalability settings to reduce it’s impact. Full details can be found in the UE guide for Lumen performance.

  • Yet, for testing purpouses, you can also fully disable Lumen, with the following commands:

    r.Lumen.GlobalIllumination=0
    r.Lumen.Reflections=0

  • Then we have Nanite, for which I would suggest checking your asset’s Nanite settings, and play around with the value for “Keep Triangle Percent”, set it to lower levels. You can also increase Nanite’s streaming pool capacity, with the following command:

    r.Nanite.Streaming.StreamingPoolSize = 2048

  • Finally, since you migrated from 5.3 to 5.7, there might be incompatible settings and/or stale cache values lurking around your project. To clear them up fully, just like you did before, delete the following folders from your main directory (add the “Saved” folder this time):

    /DerivedDataCache
    /Intermediate
    /Saved

  • Also, check your local AppData folder, there’s another instance of DerivedDataCache at UnrealEngine/5.7/, delete that one as well. Once completed, regenerate your project files, and allow UE to rebuild

I tried all of the things you suggested but it still does not work :frowning:
My project doesn’t even use Lumen. I don’t know what else I could do…

create a new project in 5.7

in the 5.3 version, migrate a level to 5.7, it should drag-along everything that level in-particular needs.

once migrated, close both, save in 5.7 if prompted. reopen the 5.7 project and test your level just to see if you can enable nanite period.

if it works, the issue is likely with your configuration (.ini), being this project would likely have default settings, you can test from there. from 53 to 57 would be a big leap. some settings might have been changed in name, in how they work, default-values, or even deprecated. The best i would be able to tell you is that the issue is in ‘the difference’ between the profiles of the two projects.

In my time i’ve run across some odd stuff upgrading a project vs just migrating, or even rebuilding it in the newer version.. Im not trying to tell you specifically what is broken, i am trying to validate a 53 thing you made works at-all in 57 to get you up-and-running, over-the-hump as-it-were (hat trick!).. PoC can sometimes be just-as-good (ha!) as a technical understanding..

my 2micro-currency-units