[D3D12][5.8] "Out of video memory" crash with VRAM and RAM free

antonlicht · June 25, 2026, 2:09pm

Summary

The editor crashes with “Out of video memory trying to allocate a rendering resource” (D3D12Util.cpp, RHISubmissionThread) in 5.8 on an RTX 2070 SUPER (8 GB). The same project, same machine, same GPU driver does not crash in 5.7, and does not crash in 5.8 under the Vulkan RHI. At the moment of the crash, both VRAM and system RAM have several GB free.

What Type of Bug are you experiencing?

Editor

Steps to Reproduce

On an 8 GB GPU that reports “GPU Upload Heaps: false” (e.g. RTX 2070 SUPER /
RTX 20-series), open a project in 5.8 using D3D12 / SM6.
Open a content-heavy level that drives VRAM usage near the 8 GB limit. Our
repro scene is a World Partition world using Nanite, Lumen, and Virtual
Shadow Maps but we did not isolate a required feature (disabling Nanite via
r.Nanite.ProjectEnabled=0 did not prevent the crash).
With that level loaded and rendering, the editor crashes.

Expected Result

The level loads and renders without crashing as it does on the same machine
in 5.7.4, and in 5.8 under the Vulkan RHI.

Observed Result

The editor crashes with “Out of video memory trying to allocate a rendering
resource” (D3D12Util.cpp, RHISubmissionThread), while several GB of VRAM and
system RAM are still free.

rhi.DumpMemory comparison (same machine and content):

5.7.4 (D3D12): Reserved Buffer Memory = 0 MB. No crash.
5.8.0 (D3D12): Reserved Buffer Memory = 8–15 GB. Crash.
5.8.0 (Vulkan): Reserved Buffer Memory ~= 15 GB. No crash.

An empty/minimal map does NOT crash, even though rhi.DumpMemory shows the same
~8 GB reserved buffer already present at startup.

Affects Versions

5.8

Platform(s)

Windows

For crash reports, include your callstack

Fatal error: [File:D:\build++UE5\Sync\Engine\Source\Runtime\D3D12RHI\Private\D3D12Util.cpp] [Line: 815]
Out of video memory trying to allocate a rendering resource

Unhandled Exception: 0x0000c000
Crash in runnable thread RHISubmissionThread

[Callstack] UnrealEditor-D3D12RHI.dll!UnknownFunction (10 frames)
[Callstack] UnrealEditor-Core.dll!UnknownFunction (2 frames)
[Callstack] KERNEL32.DLL!UnknownFunction

Additional Notes

VERIFIED FIX
In Engine/Source/Runtime/D3D12RHI/Private/Windows/WindowsD3D12Device.cpp, reserved-resource
support is enabled on tiled-resources tier alone, with no condition on GPU Upload Heap support:

GRHIGlobals.ReservedResources.Supported = Options.TiledResourcesTier >= D3D12_TILED_RESOURCES_TIER_2;

We built a custom engine that additionally gates this on GPU Upload Heap support:

GRHIGlobals.ReservedResources.Supported =
    (Options.TiledResourcesTier >= D3D12_TILED_RESOURCES_TIER_2) && GRHIDeviceSupportsGPUUploadHeaps;

With this change the crash no longer reproduces on the affected RTX 2070 SUPER (verified). On a
GPU without GPU Upload Heaps this disables reserved resources, reverting to the committed-buffer
path used by 5.7 (Reserved Buffer Memory returns to 0).

SECOND MACHINE (does not reproduce)
RTX 3070 Laptop (8 GB, Windows 11, GPU Upload Heaps: true) runs the same project in 5.8 D3D12
without crashing. The affected RTX 2070 SUPER (Windows 10, GPU Upload Heaps: false) crashes.
(The two machines also differ in GPU architecture and OS version.)

TESTED, DID NOT RESOLVE THE CRASH
Driver rollback (596.49 and 610.62 both crash); freeing system RAM; r.Streaming.PoolSize (incl.
3000) and r.Streaming.LimitPoolSizeToVRAM=1; r.LumenScene.SurfaceCache.AtlasSize (down to 1024);
sg.ShadowQuality / r.Shadow.Virtual.MaxPhysicalPages; r.Nanite.MaxNodes;
r.Nanite.Streaming.ReservedResources=0; disabling Nanite entirely (r.Nanite.ProjectEnabled=0).

RELATED
Commit CL 48674474 (#jira UE-351215) adds rhi.ReservedResources.VirtualSizeWarningGB → same
reserved-resource subsystem.

ENVIRONMENT
RTX 2070 SUPER (8 GB, Turing, GPU Upload Heaps: false); GPU driver 610.62 and 596.49 (both repro);
Windows 10 22H2; Engine 5.8.0-CL-55116800; D3D12 / SM6.

LiminalArchitect · June 26, 2026, 2:51am

I have been having the same [D3D12][5.8] “Out of video memory” crash with VRAM and RAM free on my 16GB RTX 4070 Ti Super. The project loaded into 5.8 just fine. It was when I tried to go to Landscape mode (which I never had a problem with in 5.7.4) when I got the OOM crash. Tried a lot of things to lower the VRAM usage. Even when I thought, this HAS to work now. Still got the same message when switching to landscape mode. Didn’t have this issue once in 5.7.4.

AhmedSaiedZ · July 9, 2026, 1:26am

Having same issue, with different projects on different machines, all crashing with Out of Video Memory error. with no fix at the moment.

mihneabalta · July 21, 2026, 10:35am

Most of the time you get E_OUTOFMEMORY from a D3D call it actually means that it’s out of system memory, not VRAM. Reserved resources have nothing to do with upload heap support. Disabling them when the upload heap is not available probably just works around some other issue by preventing the feature from running on older hardware.

What’s the virtual memory usage for the entire system when the crash occurs? How big is the page file? Does increasing the page file size do anything to prevent or delay the crash? Can you attach a log generated by one of these crashes?

AhmedSaiedZ · July 22, 2026, 11:32pm

Total memory usage when the crash occour is about 50% , increasing page file doesn’t work (RAM size 64gb, page file around 10gb)

Also it seems to almost always crash on loading the final textures in map.

I should note switching RHI to vulkan doesnt cause it to crash, and packaged builds doesnt crash on DX12 “crash is limited to editor on dx12”.

UECC-Windows-0D3F21E14D96627E4152C4B34C30CACE_0000.zip (489.6 KB)

GX234 · July 23, 2026, 3:01pm

Hello, I’m experiencing the same issue.

I tested it on an empty map with only a Landscape and a Landscape material applied. When I reload the map, Unreal Engine crashes.

I’m using an RTX 4090 and 128 GB of RAM, so it’s definitely not a memory issue.

Has anyone found a solution?

The issue is still not fixed in the UE 5.8.1 version from GitHub.

Rahul_Raul · July 23, 2026, 5:34pm

Hey everyone,
I am running into a severe memory spike issue in Unreal Engine 5.8 while working with the Landscape Layer Blend node. Whenever I try to fill my landscape entirely with a single layer, my memory usage spikes instantly, and the engine crashes.
A few details about my setup:
Version: Unreal Engine 5.8
Optimizations Completed: I have already optimized both the landscape and the material itself to keep performance costs down.
Context: This exact same workflow and setup ran smoothly in previous engine versions. The memory spike and crash only started happening after I moved to UE 5.8.
Has anyone else come across this exact same issue with landscape layer blending in 5.8? If so, did you find a viable workaround or fix?

mihneabalta · July 24, 2026, 10:18am

Thanks for the logs. It’s not running out of memory, it just runs out of residency sets (D3DX12Residency::ResidencySet::Open returns E_OUTOFMEMORY in this case, to confuse everyone). The D3DX residency implementation can handle at most 1024 in-flight command lists, which is more than enough in most cases, but in your scenario it seems that the loading code creates lots of command lists that aren’t processed quickly enough by the submission queue. We’ll look into this, please bear with us while we try to understand the root cause.

Rahul_Raul · July 24, 2026, 7:58pm

Out of memory while preparing landscape textures,8k landscape.

Anyone got solution for this ?

I am in middle of the project at very crucial stage.

(out of video memory trying to allocate a rendering resource unreal 5.8)

GX234 · July 25, 2026, 3:39am

Switch to Vulkan

Rahul_Raul · July 25, 2026, 6:09am

I know, but need DX12 Shaders.

urosrajsic · July 28, 2026, 9:04am

i managed to find a specific workaround for my case with

[SystemSettings]
rhi.UseSubmissionThread=0

in the DefaultEngine.ini

facundosanquiez · August 11, 2026, 12:06am

Experiencing the same issue.

vms119 · August 13, 2026, 12:38pm

I have the same issue. I´ve been able to create the landscape and loading all the weights map for the paint layers in 5.7 without any issue. And in 5.8 in a empty scene just with the landscape, everytime i load the weight map for a layer it crashes with “out of memory” message. This is what Claude told me why it is crashing in 5.8 and not in 5.7:

"I compared the code from both installations (UE_5.7 and UE_5.8 are both located in C:\\Program Files\\Epic Games). In 5.8, Epic rewrote D3D12 resource residency handling. There are three changes that do not exist at all in 5.7:

D3D12.ResourcesStartResident (D3D12Adapter.cpp:38; new in 5.8, defaults to false): All D3D12 resources are now created in an EVICTED state—using `D3D12_HEAP_FLAG_CREATE_NOT_RESIDENT`—and residency is resolved during each command list submission. In 5.7, they started out resident, and none of this logic existed.
Expanded tracking (D3D12Resources.cpp:147):
// 5.7: GPU-only resources only
bRequiresResidencyTracking = IsGPUOnly(InHeapType, HeapProps) && !bExternal && !bBackBuffer;
// 5.8: On single-GPU setups, EVERYTHING is tracked (including CPU upload/readback)
const bool bShouldTrackResource = (GNumExplicitGPUsForRendering == 1) || IsGPUOnly(…);
On your machine (1 GPU), this means that upload and readback buffers are also included in the residency set for each command list. The landscape weightmap merge operation generates heaps of these (one `FLandscapeEditLayerReadback` per weightmap, plus the uploads).
Landscape 5.8 (LandscapeEditLayers.cpp): Each component now requests its 3 neighbors as part of the input area as well (`const FIntRect InputArea(FIntPoint(-1,-1), FIntPoint::ZeroValue)`), which bloats the size of each merge batch.

Result: The batched weightmap merge is the operation that introduces the highest number of tracked resources into a single submission, overwhelming the residency manager’s internal structure. That is why it crashes specifically when loading a weightmap or performing a “Fill Layer” operation."

Claude gave me some console variables but none of them worked.

Flak · August 17, 2026, 1:23pm

Hey folks, this is scheduled to be addressed in 5.8.2.

AhmedSaiedZ · August 25, 2026, 7:25pm

Crash persists in 5.8.2

ftrx · August 27, 2026, 11:49am

same for me, crash still persists. But now instead of the “Out of memory” crash i get this:

[2026.08.27-10.21.12:279][  0]LogD3D12RHI: Error: pSet->Open() failed 
 at D:\build\++UE5\Sync\Engine\Source\Runtime\D3D12RHI\Private\D3D12Residency.h:160 
 with error E_FAIL
No command list slot was available. Too many residency sets are open concurrently. 								    Consider raising MAX_NUM_CONCURRENT_CMD_LISTS in d3dx12residency.h
[2026.08.27-10.21.26:946][  0]LogWindows: Error: appError called: Fatal error: [File:D:\build\++UE5\Sync\Engine\Source\Runtime\D3D12RHI\Private\D3D12Util.cpp] [Line: 1062] 
pSet->Open() failed 
 at D:\build\++UE5\Sync\Engine\Source\Runtime\D3D12RHI\Private\D3D12Residency.h:160 
 with error E_FAIL
No command list slot was available. Too many residency sets are open concurrently. 								    Consider raising MAX_NUM_CONCURRENT_CMD_LISTS in d3dx12residency.h

[2026.08.27-10.21.26:947][  0]LogWindows: Windows GetLastError: The operation completed successfully. (0)
[2026.08.27-10.21.26:949][  0]LogWindows: Error: === Critical error: ===
[2026.08.27-10.21.26:949][  0]LogWindows: Error: 
[2026.08.27-10.21.26:950][  0]LogWindows: Error: Fatal error: [File:D:\build\++UE5\Sync\Engine\Source\Runtime\D3D12RHI\Private\D3D12Util.cpp] [Line: 1062] 

[2026.08.27-10.21.26:950][  0]LogWindows: Error: pSet->Open() failed 
[2026.08.27-10.21.26:950][  0]LogWindows: Error:  at D:\build\++UE5\Sync\Engine\Source\Runtime\D3D12RHI\Private\D3D12Residency.h:160 
[2026.08.27-10.21.26:950][  0]LogWindows: Error:  with error E_FAIL
[2026.08.27-10.21.26:950][  0]LogWindows: Error: No command list slot was available. Too many residency sets are open concurrently. 								    Consider raising MAX_NUM_CONCURRENT_CMD_LISTS in d3dx12residency.h
[2026.08.27-10.21.26:951][  0]LogWindows: Error: 
[2026.08.27-10.21.26:951][  0]LogWindows: Error: 
[2026.08.27-10.21.26:951][  0]LogWindows: Error: 
[2026.08.27-10.21.26:976][  0]LogExit: Executing StaticShutdownAfterError
[2026.08.27-10.21.27:025][  0]LogWindows: FPlatformMisc::RequestExit(1, WindowsErrorOutputDevice::Serialize.!GIsGuarded)
[2026.08.27-10.21.27:026][  0]LogWindows: FPlatformMisc::RequestExitWithStatus(1, 3, WindowsErrorOutputDevice::Serialize.!GIsGuarded)
[2026.08.27-10.21.27:026][  0]LogCore: Engine exit requested (reason: Win RequestExit)

the

[SystemSettings]
rhi.UseSubmissionThread=0

Still solves the editor crash for me, but makes the editor laggy with a large map and lot of contents.
Builds work fine even without the “fix” above.

chunfeng_tsin · August 30, 2026, 3:48am

this solution is greate !!

Bug-Reporter · August 31, 2026, 11:00pm

UE-387332’s status has changed to ’Needs Peer Review’. A fix has been submitted, and is awaiting review by another team member.