VirtualHeightfieldMesh crashes editor and game in FVirtualTextureFeedback::TransferGPUToCPU

I found an issue with using VirtualHeightfieldMesh.
I am trying to follow Youtube tutorial: https://youtu.be/cV2YhjAiogY
But I encountered engine crashes after I made an VirtualHeightfieldMesh visible in editor.
When it is visible only in game - game works for some time (longer then editor), but eventually also crashes.
I started with UE 5.5.0 where it crashed nearly immediately, then updated to 5.5.3 and now it crashes after 15-20s in editor.

In my example I am not writing to “NewRuntimeVirtualTexture”, but it doesn’t matter - I can initialize it all with zeros, with some real level data, but nothing changes.

Generally this VirtualHeightfieldMesh works! I can see for a moment proper shape in editor as well as in game.
What helps - setting Platforms->Windows->DefaultGraphicsRHI to DX11, but as UE5 goes into DX12 it do not looks like valid solution.

Any ideas why it happens and how to fix it?
I know that feature is experimental, but looks like it works for others.

What I did to reproduce problem:

  • create empty game project “blank” (I can zip it all and send - it is 25MB zip)

  • add plugin “virtual heightfield mesh”

  • enable “virtual texture support”

  • add to level “VirtualHeightfieldMesh”

  • created “NewRuntimeVirtualTexture” with virtual texture content set to “World height”

  • Landscape instance “Draw in Virtual Textures” set to “NewRuntimeVirtualTexture”, executed “Create Volume” to create “RuntimeVirtualTextureVolume”

  • “RuntimeVirtualTextureVolume” bounds set to “Landscape” with “expand bounds” set to 100

  • in “VirtualHeightfieldMesh” heightfield set to my virtual texture

  • built minmax texture

  • disable “Actor Hidden In Editor” for “VirtualHeightfieldMesh”

  • error message appears nearly immediately in log:
    LogOutputDevice: Error: === Handled ensure: ===
    LogOutputDevice: Error: Ensure condition failed: Query->SyncPoint [File:D:\build++UE5\Sync\Engine\Source\Runtime\D3D12RHI\Private\D3D12Query.cpp] [Line: 324]
    LogOutputDevice: Error: Attempt to get result data for an FRHIRenderQuery that was never used in a command list.
    LogOutputDevice: Error: Stack:
    LogOutputDevice: Error: [Callstack] 0x00007ffc4a308ffd UnrealEditor-D3D12RHI.dll!FRHICommandListBase::GetScratchShaderParameters'::6’::<lambda_1>::operator()() [D:\build++UE5\Sync\Engine\Source\Runtime\D3D12RHI\Private\D3D12Query.cpp:324]
    LogOutputDevice: Error: [Callstack] 0x00007ffc4a2343da UnrealEditor-D3D12RHI.dll!FD3D12DynamicRHI::RHIGetRenderQueryResult() [D:\build++UE5\Sync\Engine\Source\Runtime\D3D12RHI\Private\D3D12Query.cpp:324]
    LogOutputDevice: Error: [Callstack] 0x00007ffc6f06714a UnrealEditor-Renderer.dll!FVirtualTextureFeedback::Map() [D:\build++UE5\Sync\Engine\Source\Runtime\Renderer\Private\VT\VirtualTextureFeedback.cpp:259]
    LogOutputDevice: Error: [Callstack] 0x00007ffc6f066e24 UnrealEditor-Renderer.dll!FVirtualTextureFeedback::Map() [D:\build++UE5\Sync\Engine\Source\Runtime\Renderer\Private\VT\VirtualTextureFeedback.cpp:356]
    LogOutputDevice: Error: [Callstack] 0x00007ffc6f0a198e UnrealEditor-Renderer.dll!FVirtualTextureSystem::BeginUpdate() [D:\build++UE5\Sync\Engine\Source\Runtime\Renderer\Private\VT\VirtualTextureSystem.cpp:2690]
    LogOutputDevice: Error: [Callstack] 0x00007ffc6f0a27fe UnrealEditor-Renderer.dll!FVirtualTextureSystem::BeginUpdate() [D:\build++UE5\Sync\Engine\Source\Runtime\Renderer\Private\VT\VirtualTextureSystem.cpp:2820]
    LogOutputDevice: Error: [Callstack] 0x00007ffc6de84f84 UnrealEditor-Renderer.dll!FDeferredShadingSceneRenderer::Render() [D:\build++UE5\Sync\Engine\Source\Runtime\Renderer\Private\DeferredShadingRenderer.cpp:1390]
    LogOutputDevice: Error: [Callstack] 0x00007ffc6ec65236 UnrealEditor-Renderer.dll!RenderViewFamilies_RenderThread() [D:\build++UE5\Sync\Engine\Source\Runtime\Renderer\Private\SceneRendering.cpp:5431]
    LogOutputDevice: Error: [Callstack] 0x00007ffc6ec1c29a UnrealEditor-Renderer.dll!UE::RendererPrivateUtils::Implementation::TPersistentBuffer<UE::HLSL::FCellBlockData,UE::RendererPrivateUtils::Implementation::FStructuredBufferTraits>::ValidateGPUData<FSceneCullingBuilder::UploadToGPU’::51'::<lambda_1> >'::12’::<lambda_1>::operator()() [D:\build++UE5\Sync\Engine\Source\Runtime\Renderer\Private\SceneRendering.cpp:5734]
    LogOutputDevice: Error: [Callstack] 0x00007ffcd3dec2e5 UnrealEditor-RenderCore.dll!FRenderCommandPipeRegistry::StopRecording'::2’::<lambda_1>::operator()() [D:\build++UE5\Sync\Engine\Source\Runtime\RenderCore\Private\RenderingThread.cpp:1541]
    LogOutputDevice: Error: [Callstack] 0x00007ffcd3e19fef UnrealEditor-RenderCore.dll!TGraphTask<TFunctionGraphTaskImpl<void __cdecl(void),1> >::ExecuteTask() [D:\build++UE5\Sync\Engine\Source\Runtime\Core\Public\Async\TaskGraphInterfaces.h:634]
    LogOutputDevice: Error: [Callstack] 0x00007ffcc1d92722 UnrealEditor-Core.dll!UE::Tasks::Private::FTaskBase::TryExecuteTask() [D:\build++UE5\Sync\Engine\Source\Runtime\Core\Public\Tasks\TaskPrivate.h:504]
    LogOutputDevice: Error: [Callstack] 0x00007ffcc1d87d4a UnrealEditor-Core.dll!FNamedTaskThread::ProcessTasksNamedThread() [D:\build++UE5\Sync\Engine\Source\Runtime\Core\Private\Async\TaskGraph.cpp:779]
    LogOutputDevice: Error: [Callstack] 0x00007ffcc1d881ee UnrealEditor-Core.dll!FNamedTaskThread::ProcessTasksUntilQuit() [D:\build++UE5\Sync\Engine\Source\Runtime\Core\Private\Async\TaskGraph.cpp:668]
    LogOutputDevice: Error: [Callstack] 0x00007ffcd3e4893c UnrealEditor-RenderCore.dll!RenderingThreadMain() [D:\build++UE5\Sync\Engine\Source\Runtime\RenderCore\Private\RenderingThread.cpp:317]
    LogOutputDevice: Error: [Callstack] 0x00007ffcd3e4d9d4 UnrealEditor-RenderCore.dll!FRenderingThread::Run() [D:\build++UE5\Sync\Engine\Source\Runtime\RenderCore\Private\RenderingThread.cpp:443]
    LogOutputDevice: Error: [Callstack] 0x00007ffcc23bc0bd UnrealEditor-Core.dll!FRunnableThreadWin::Run() [D:\build++UE5\Sync\Engine\Source\Runtime\Core\Private\Windows\WindowsRunnableThread.cpp:159]
    LogOutputDevice: Error: [Callstack] 0x00007ffcc23b35df UnrealEditor-Core.dll!FRunnableThreadWin::GuardedRun() [D:\build++UE5\Sync\Engine\Source\Runtime\Core\Private\Windows\WindowsRunnableThread.cpp:79]
    LogOutputDevice: Error: [Callstack] 0x00007ffd12f77374 KERNEL32.DLL!UnknownFunction
    LogOutputDevice: Error: [Callstack] 0x00007ffd14d7cc91 ntdll.dll!UnknownFunction
    LogStats: SubmitErrorReport - 0.000 s
    LogStats: SendNewReport - 0.681 s
    LogStats: FDebug::EnsureFailed - 0.899 s

  • then editor crashes after some time with various reasons (access violation or some asserts about reference counter below zero etc), but always from FVirtualTextureFeedback::TransferGPUToCPU()

example “access violation”:

LoginId:6c8b81404ff48cfb7b458c8d4b15ff05
EpicAccountId:aa82e69b669446e6aaf4ad73cc89be27

Unhandled Exception: EXCEPTION_ACCESS_VIOLATION reading address 0x0000000000000070

UnrealEditor_D3D12RHI!FD3D12DynamicRHI::RHIEndRenderQuery_TopOfPipe() [D:\build++UE5\Sync\Engine\Source\Runtime\D3D12RHI\Private\D3D12Query.cpp:212]
UnrealEditor_Renderer!FVirtualTextureFeedback::TransferGPUToCPU() [D:\build++UE5\Sync\Engine\Source\Runtime\Renderer\Private\VT\VirtualTextureFeedback.cpp:207]
UnrealEditor_Renderer!TRDGLambdaPass<RuntimeVirtualTexture::FShader_VirtualTextureCopy::FParameters,RuntimeVirtualTexture::AddCopyPass<0>'::2’::<lambda_1> >::Execute() [D:\build++UE5\Sync\Engine\Source\Runtime\RenderCore\Public\RenderGraphPass.h:703]
UnrealEditor_RenderCore!FRDGBuilder::ExecutePass() [D:\build++UE5\Sync\Engine\Source\Runtime\RenderCore\Private\RenderGraphBuilder.cpp:3360]
UnrealEditor_RenderCore!FRDGTraceGraphMessageFields::Initialize'::2’::<lambda_1>::operator()() [D:\build++UE5\Sync\Engine\Source\Runtime\RenderCore\Private\RenderGraphBuilder.cpp:2903]
UnrealEditor_RenderCore!UE::Tasks::Private::FTaskBase::TryExecuteTask() [D:\build++UE5\Sync\Engine\Source\Runtime\Core\Public\Tasks\TaskPrivate.h:504]
UnrealEditor_RenderCore!LowLevelTasks::TTaskDelegate<LowLevelTasks::FTask * __ptr64 __cdecl(bool),48>::TTaskDelegateImpl<LowLevelTasks::FTask::Init<UE::Tasks::Private::FTaskBase::Init’::2'::<lambda_1> >'::13’::<lambda_1>,0>::CallAndMove() [D:\build++UE5\Sync\Engine\Source\Runtime\Core\Public\Async\Fundamental\TaskDelegate.h:171]
UnrealEditor_Core!LowLevelTasks::FTask::ExecuteTask() [D:\build++UE5\Sync\Engine\Source\Runtime\Core\Public\Async\Fundamental\Task.h:627]
UnrealEditor_Core!LowLevelTasks::FScheduler::ExecuteTask() [D:\build++UE5\Sync\Engine\Source\Runtime\Core\Private\Async\Fundamental\Scheduler.cpp:245]
UnrealEditor_Core!LowLevelTasks::FScheduler::WorkerLoop() [D:\build++UE5\Sync\Engine\Source\Runtime\Core\Private\Async\Fundamental\Scheduler.cpp:514]
UnrealEditor_Core!LowLevelTasks::FScheduler::CreateWorker'::2’::<lambda_1>::operator()() [D:\build++UE5\Sync\Engine\Source\Runtime\Core\Private\Async\Fundamental\Scheduler.cpp:75]
UnrealEditor_Core!FThreadImpl::Run() [D:\build++UE5\Sync\Engine\Source\Runtime\Core\Private\HAL\Thread.cpp:69]
UnrealEditor_Core!FRunnableThreadWin::Run() [D:\build++UE5\Sync\Engine\Source\Runtime\Core\Private\Windows\WindowsRunnableThread.cpp:159]

other reason:
Assertion failed: (OldPacked & DeletingBit) == 0 [File:D:\build++UE5\Sync\Engine\Source\Runtime\RHI\Public\RHIResources.h] [Line: 154]

UnrealEditor_RHI!FRHIResource::MarkForDelete() [D:\build++UE5\Sync\Engine\Source\Runtime\RHI\Private\RHIResources.cpp:49]
UnrealEditor_Renderer!FFeedbackGPUFencePool::Allocate() [D:\build++UE5\Sync\Engine\Source\Runtime\Renderer\Private\VT\VirtualTextureFeedback.cpp:114]
UnrealEditor_Renderer!FVirtualTextureFeedback::TransferGPUToCPU() [D:\build++UE5\Sync\Engine\Source\Runtime\Renderer\Private\VT\VirtualTextureFeedback.cpp:204]

1 Like

Hi, for me virtual heightfield mesh plus runtime virtual texture worked up to 5.4, when I updated to 5.5 the game crashes after around less than a second (I stayed at 5.4 due to this bug). The editor ran stable though. But play in editor as well as standalone crashes. So it feels to me that virtual heightfield mesh and/or runtime virtual texture has simply become broken in the engine in 5.5.

3 Likes

Thank you very much for your input - at least I know I am not alone here :slight_smile:
For me that functionality is not critical, so I’ll simply wait for fix.

2 Likes

Same here. After digging into the matter, this issues related to 5.5 new RDG async task code.

In 5.5, FVirtualTextureFeedback::TransferGPUToCPU could be called from two separate source in different WorkerThread, one for VT system and other is VHM, in which case causes data racing.

In 5.4 two sources are calling from RenderThread. If you set r.RDG.ParallelExecute 0, the crash would not longer happen.

One easy fix is to duplicate the global GVirtualTextureFeedback to use new GVirtualTextureFeedbackVHM for the only VHM API in FVirtualTextureFeedback::TransferGPUToCPU.

TGlobalResource< FVirtualTextureFeedback > GVirtualTextureFeedback;
TGlobalResource< FVirtualTextureFeedback > GVirtualTextureFeedbackVHM; // Fix VHM crash when r.RDG.ParallelExecute > 0

0001-VirtualHeightMesh-Fix-VHM-crash-when-r.RDG.ParallelE.patch (2.6 KB)
0002-VirtualHeightMesh-Fix-VHM-crash-when-there-are-more-.patch (2.0 KB)

2 Likes