Severe Visual Artifacts and Ray Tracing Memory Budget Issues after GPU Upgrade (RX 580 to RTX 5060 Ti) in Unreal Engine 5.6

Development Rendering
,
1

Subject: Severe Visual Artifacts and Ray Tracing Memory Budget Issues after GPU Upgrade (RX 580 to RTX 5060 Ti) in Unreal Engine 5.X

Hello everyone,

I’m encountering significant visual artifacts and performance issues in my Unreal Engine 5 project (Star_Wars_Lost_Echo) after recently upgrading my hardware. I’m hoping someone here might have experience with similar problems and can offer some guidance.

My Setup:

  • Previous GPU: AMD Radeon RX 580 8GB

  • New GPU: ASUS GeForce RTX 5060 Ti 16GB

  • OS: Upgraded from Windows 10 to Windows 11

  • Drivers: Latest NVIDIA Studio Drivers installed (clean installation after using DDU)

  • Unreal Engine Version: [Specify your UE version, e.g., 5.3.2, 5.4.0, etc.]

The Problem:

My project worked fine on the old RX 580 (though with lower performance, of course). After upgrading, I’m facing two main issues:

  1. “RAY TRACING GEOMETRY - REQUESTED MEMORY OVER BUDGET” Error:

  • I constantly see this error message in the editor viewport (e.g., “REQUESTED MEMORY OVER BUDGET 325,571 MB / 400 MB” or similar values).

  • I have already set r.RayTracing.Geometry.MemoryBudgetMB=8192 (or even higher) in my DefaultEngine.ini file, but the error persists, although the “REQUESTED” value sometimes changes.

  • Question: How can I definitively solve this memory budget issue? Could this error still be causing the visual artifacts even if I tried increasing the budget?

  1. Severe Visual Artifacts (Even with Ray Tracing Shadows Disabled):

  • Initially, I had many artifacts (dirty textures, wrong shadows, shimmering, glitches on distant meshes) when Ray Tracing Shadows were enabled. Disabling Ray Tracing Shadows partially resolved some artifacts.

  • New Artifacts (primarily on terrain/environment): Now, when I view parts of the scene (especially the terrain) from a distance, they look fine. However, as I move the camera closer to a specific point, unexpected shadows appear, or the terrain/objects suddenly become darker/dirtier . This effect seems to pop in as I approach. I’ve attached screenshots illustrating this (showing the same scene from far and close-up, highlighting the darkening/shadow pop-in).

  • I’ve tried the following solutions without success:

    • Cleared Intermediate and DerivedDataCache folders multiple times.

    • Increased Lumen Max Trace Distance (e.g., to 50000-100000).

    • Adjusted Dynamic Shadow Distance and Num Dynamic Shadow Cascades on my Directional Light.

    • Checked and ensured Nanite is enabled for complex meshes like the terrain.

    • Experimented with Post Process Volume settings (Ambient Occlusion, particularly).

    • Reinstalled NVIDIA Studio Drivers using DDU.

My Goal:
I want to leverage the full power of my new RTX 5060 Ti, including Ray Tracing and Lumen, for realistic rendering and better performance in Unreal Engine.

Attached Screenshots:

StarWarsLostEcho - Unreal Editor 27052026 102925
StarWarsLostEcho - Unreal Editor 27052026 1029251920×1040 720 KB

StarWarsLostEcho - Unreal Editor 27052026 102921
StarWarsLostEcho - Unreal Editor 27052026 1029211920×1040 750 KB

StarWarsLostEcho - Unreal Editor 27052026 102917
StarWarsLostEcho - Unreal Editor 27052026 1029171920×1040 566 KB

StarWarsLostEcho - Unreal Editor 27052026 102910
StarWarsLostEcho - Unreal Editor 27052026 1029101920×1040 626 KB

Any help or insights would be greatly appreciated. Thank you in advance!

2

Subject: (Solved) Severe Visual Artifacts and Ray Tracing Memory Budget Issues after GPU Upgrade (RX 580 to RTX 5060 Ti) in Unreal Engine 5.X

Hello everyone,

Thank you to everyone who took the time to read my post. I’m happy to report that I’ve managed to identify and resolve the main cause of the visual artifacts and the “RAY TRACING GEOMETRY - REQUESTED MEMORY OVER BUDGET” error.

The Solution:

After a lot of testing, I managed to isolate the problem. The artifacts (the darkening/shadow pop-in on the terrain, and likely contributing to the memory budget issue) were specifically appearing on my terrain meshes.

The root cause was the interaction of Nanite with the terrain geometry/materials.

By disabling Nanite on the problematic terrain static meshes, the visual artifacts completely disappeared, and consequently, the “RAY TRACING GEOMETRY - REQUESTED MEMORY OVER BUDGET” error also stopped appearing.

It seems that for this particular terrain setup (or perhaps a specific combination of material/mesh density/UE version/GPU driver), Nanite was causing conflicts when integrated with the Ray Tracing pipeline, leading to incorrect calculations and memory overruns. While Nanite is generally excellent for dense geometry, in this specific case, disabling it on the terrain resolved the rendering issues.

I’m now able to leverage Ray Tracing and Lumen correctly on other parts of my scene without the previous issues.

Conclusion:

This experience highlights that even powerful features like Nanite might require careful consideration and testing, especially when combined with other demanding rendering features like Ray Tracing, and potentially in specific hardware/software configurations.

Thanks again for being part of such a helpful community! I hope this solution might help anyone encountering similar issues.