[D3D12][5.8] "Out of video memory" crash with VRAM and RAM free

Summary

The editor crashes with “Out of video memory trying to allocate a rendering resource” (D3D12Util.cpp, RHISubmissionThread) in 5.8 on an RTX 2070 SUPER (8 GB). The same project, same machine, same GPU driver does not crash in 5.7, and does not crash in 5.8 under the Vulkan RHI. At the moment of the crash, both VRAM and system RAM have several GB free.

What Type of Bug are you experiencing?

Editor

Steps to Reproduce

  1. On an 8 GB GPU that reports “GPU Upload Heaps: false” (e.g. RTX 2070 SUPER /
    RTX 20-series), open a project in 5.8 using D3D12 / SM6.
  2. Open a content-heavy level that drives VRAM usage near the 8 GB limit. Our
    repro scene is a World Partition world using Nanite, Lumen, and Virtual
    Shadow Maps but we did not isolate a required feature (disabling Nanite via
    r.Nanite.ProjectEnabled=0 did not prevent the crash).
  3. With that level loaded and rendering, the editor crashes.

Expected Result

The level loads and renders without crashing as it does on the same machine
in 5.7.4, and in 5.8 under the Vulkan RHI.

Observed Result

The editor crashes with “Out of video memory trying to allocate a rendering
resource” (D3D12Util.cpp, RHISubmissionThread), while several GB of VRAM and
system RAM are still free.

rhi.DumpMemory comparison (same machine and content):

  • 5.7.4 (D3D12): Reserved Buffer Memory = 0 MB. No crash.
  • 5.8.0 (D3D12): Reserved Buffer Memory = 8–15 GB. Crash.
  • 5.8.0 (Vulkan): Reserved Buffer Memory ~= 15 GB. No crash.

An empty/minimal map does NOT crash, even though rhi.DumpMemory shows the same
~8 GB reserved buffer already present at startup.

Affects Versions

5.8

Platform(s)

Windows

For crash reports, include your callstack

Fatal error: [File:D:\build++UE5\Sync\Engine\Source\Runtime\D3D12RHI\Private\D3D12Util.cpp] [Line: 815]
Out of video memory trying to allocate a rendering resource

Unhandled Exception: 0x0000c000
Crash in runnable thread RHISubmissionThread

[Callstack] UnrealEditor-D3D12RHI.dll!UnknownFunction (10 frames)
[Callstack] UnrealEditor-Core.dll!UnknownFunction (2 frames)
[Callstack] KERNEL32.DLL!UnknownFunction

Additional Notes

VERIFIED FIX
In Engine/Source/Runtime/D3D12RHI/Private/Windows/WindowsD3D12Device.cpp, reserved-resource
support is enabled on tiled-resources tier alone, with no condition on GPU Upload Heap support:

GRHIGlobals.ReservedResources.Supported = Options.TiledResourcesTier >= D3D12_TILED_RESOURCES_TIER_2;

We built a custom engine that additionally gates this on GPU Upload Heap support:

GRHIGlobals.ReservedResources.Supported =
    (Options.TiledResourcesTier >= D3D12_TILED_RESOURCES_TIER_2) && GRHIDeviceSupportsGPUUploadHeaps;

With this change the crash no longer reproduces on the affected RTX 2070 SUPER (verified). On a
GPU without GPU Upload Heaps this disables reserved resources, reverting to the committed-buffer
path used by 5.7 (Reserved Buffer Memory returns to 0).

SECOND MACHINE (does not reproduce)
RTX 3070 Laptop (8 GB, Windows 11, GPU Upload Heaps: true) runs the same project in 5.8 D3D12
without crashing. The affected RTX 2070 SUPER (Windows 10, GPU Upload Heaps: false) crashes.
(The two machines also differ in GPU architecture and OS version.)

TESTED, DID NOT RESOLVE THE CRASH
Driver rollback (596.49 and 610.62 both crash); freeing system RAM; r.Streaming.PoolSize (incl.
3000) and r.Streaming.LimitPoolSizeToVRAM=1; r.LumenScene.SurfaceCache.AtlasSize (down to 1024);
sg.ShadowQuality / r.Shadow.Virtual.MaxPhysicalPages; r.Nanite.MaxNodes;
r.Nanite.Streaming.ReservedResources=0; disabling Nanite entirely (r.Nanite.ProjectEnabled=0).

RELATED
Commit CL 48674474 (#jira UE-351215) adds rhi.ReservedResources.VirtualSizeWarningGB → same
reserved-resource subsystem.

ENVIRONMENT
RTX 2070 SUPER (8 GB, Turing, GPU Upload Heaps: false); GPU driver 610.62 and 596.49 (both repro);
Windows 10 22H2; Engine 5.8.0-CL-55116800; D3D12 / SM6.

I have been having the same [D3D12][5.8] “Out of video memory” crash with VRAM and RAM free on my 16GB RTX 4070 Ti Super. The project loaded into 5.8 just fine. It was when I tried to go to Landscape mode (which I never had a problem with in 5.7.4) when I got the OOM crash. Tried a lot of things to lower the VRAM usage. Even when I thought, this HAS to work now. Still got the same message when switching to landscape mode. Didn’t have this issue once in 5.7.4.

Having same issue, with different projects on different machines, all crashing with Out of Video Memory error. with no fix at the moment.

Most of the time you get E_OUTOFMEMORY from a D3D call it actually means that it’s out of system memory, not VRAM. Reserved resources have nothing to do with upload heap support. Disabling them when the upload heap is not available probably just works around some other issue by preventing the feature from running on older hardware.

What’s the virtual memory usage for the entire system when the crash occurs? How big is the page file? Does increasing the page file size do anything to prevent or delay the crash? Can you attach a log generated by one of these crashes?

Total memory usage when the crash occour is about 50% , increasing page file doesn’t work (RAM size 64gb, page file around 10gb)

Also it seems to almost always crash on loading the final textures in map.

I should note switching RHI to vulkan doesnt cause it to crash, and packaged builds doesnt crash on DX12 “crash is limited to editor on dx12”.

UECC-Windows-0D3F21E14D96627E4152C4B34C30CACE_0000.zip (489.6 KB)

Hello, I’m experiencing the same issue.

I tested it on an empty map with only a Landscape and a Landscape material applied. When I reload the map, Unreal Engine crashes.

I’m using an RTX 4090 and 128 GB of RAM, so it’s definitely not a memory issue.

Has anyone found a solution?

The issue is still not fixed in the UE 5.8.1 version from GitHub.

Hey everyone,
I am running into a severe memory spike issue in Unreal Engine 5.8 while working with the Landscape Layer Blend node. Whenever I try to fill my landscape entirely with a single layer, my memory usage spikes instantly, and the engine crashes.
A few details about my setup:
Version: Unreal Engine 5.8
Optimizations Completed: I have already optimized both the landscape and the material itself to keep performance costs down.
Context: This exact same workflow and setup ran smoothly in previous engine versions. The memory spike and crash only started happening after I moved to UE 5.8.
Has anyone else come across this exact same issue with landscape layer blending in 5.8? If so, did you find a viable workaround or fix?

Thanks for the logs. It’s not running out of memory, it just runs out of residency sets (D3DX12Residency::ResidencySet::Open returns E_OUTOFMEMORY in this case, to confuse everyone). The D3DX residency implementation can handle at most 1024 in-flight command lists, which is more than enough in most cases, but in your scenario it seems that the loading code creates lots of command lists that aren’t processed quickly enough by the submission queue. We’ll look into this, please bear with us while we try to understand the root cause.

2 Likes

Out of memory while preparing landscape textures,8k landscape.

Anyone got solution for this ?

I am in middle of the project at very crucial stage.

(out of video memory trying to allocate a rendering resource unreal 5.8)

Switch to Vulkan

I know, but need DX12 Shaders.

i managed to find a specific workaround for my case with

[SystemSettings]
rhi.UseSubmissionThread=0

in the DefaultEngine.ini

UE-387332’s status has changed to ‘Ready for QA’. A member of the QA department is investigating the issue.

Experiencing the same issue.