We’ve been having a lot of issues with finding the right Nvidia driver for Unreal on Linux, and after extensive testing of 570, 580, and 595, it looks like 580 is the most stable. On 570, we’d crash Unreal with ray tracing on “all the time” upon scene load, and on 595, resizing the Editor locks up the engine and it doesn’t come back (no actual crash). 580 seems to be the sweet spot, but this is the main crash we’re worried about. This doesn’t seem to affect 5.7.4, but I’ve only given it a cursory test, and given that the crash isn’t fully reproducible, it’s hard to know.
Our main curiosity is whether the crash is a known issue & fixed in 5.7.4/5.8, if there are additional things we could try to avoid it, or if it’s indicative of anything obviously concerning that we should look into with our Nanite setup, etc.
Steps to Reproduce
Struggling to reproduce this, but it has occurred a number of times for us. It seems to be related to loading medium-to-heavy scenes. I’ve gotten it to crash after loading a medium-sized scene without the Viewport open, then opening the Viewport (it was not tabbed/docked, so a fresh Viewport instance). At other times, I’ve had it happen when loading the scene with the Viewport open.
22:47:46 — Maglev level opened via LogAssetEditorSubsystem
22:47:49 — Blueprint BP_ShadowFixes_C_1 executes:
r.RayTracing.Culling 0
r.RayTracing.Nanite.Mode 1
22:57:59 — Viewport goes fullscreen/immersive
22:58:01 — Two LogRenderer: Using fallback RTPSO messages
22:58:02 — CRASH
This sounds like a known engine bug in UE5.6 with Nanite ray tracing on Linux/Vulkan. The FRayTracingManager::Update() can race or double-register a GeometryRHI handle when:
Ray tracing culling is disabled
r.RayTracing.Nanite.Mode 1 is active
A scene update occurs while primitives are still being registered
The RTPSO fallback messages in your log immediately before the crash are also a sign the Vulkan RT pipeline wasn’t fully ready.
You can try and remove the culling from the Blueprint if possible. This seems to be the main trigger and could be overloading the Vulkan RT manager. Or you can try and turn nanite mode off but that may produce undesired RT shadows if you are using those. If you need both, you can maybe try to add a small delay before enabling them in the BP.
Its been suggested that maybe the fix would be to guard with a check rather than an assert, or to clear GeometryRHI before the set in PrimitiveSceneInfo.cpp:1356. But thats a guess strictly based on log files.