When I try to use the blueprint debugger with an Unreal 4.19.1 source build, the Unreal Editor process starts consuming excessive amounts of memory (and cpu) with it rapidly hitting a 64G footprint and crashing. This is only a few seconds after hitting the breakpoint in the blueprint (I paused the editor in VS2017 to catch this)
I started doing some memory profiling, but figured it would be faster to ask here since I’ve never tried to break apart any of the c++ tick loops in Unreal. One thing I noticed with the first thing ‘leaked’ was a difference in the way it is normally freed vs. what happens while in the blueprint breakpoint. The first item that popped up was a long list of ever growing GpuProfilerEvents. Looking at the Call stack for normal clearing of GpuProfilerEvents:
UE4Editor-Engine.dll!FRealtimeGPUProfilerFrame::Clear(FRHICommandListImmediate * RHICommandListPtr) Line 378 C++ Symbols loaded.
UE4Editor-Engine.dll!FRealtimeGPUProfiler::EndFrame(FRHICommandListImmediate & RHICmdList) Line 574 C++ Symbols loaded.
UE4Editor.exe!`FEngineLoop::Tick'::`50'::EURCMacro_EndFrame::DoTask(ENamedThreads::Type CurrentThread, const TRefCountPtr<FGraphEvent> & MyCompletionGraphEvent) Line 3528 C++ Symbols loaded.
UE4Editor.exe!TGraphTask<`FEngineLoop::Tick'::`50'::EURCMacro_EndFrame>::ExecuteTask(TArray<FBaseGraphTask *,FDefaultAllocator> & NewTasks, ENamedThreads::Type CurrentThread) Line 829 C++ Symbols loaded.
[Inline Frame] UE4Editor-Core.dll!FBaseGraphTask::Execute(TArray<FBaseGraphTask *,FDefaultAllocator> & CurrentThread, ENamedThreads::Type) Line 498 C++ Symbols loaded.
UE4Editor-Core.dll!FNamedTaskThread::ProcessTasksNamedThread(int QueueIndex, bool bAllowStall) Line 665 C++ Symbols loaded.
UE4Editor-Core.dll!FNamedTaskThread::ProcessTasksUntilQuit(int QueueIndex) Line 574 C++ Symbols loaded.
UE4Editor-RenderCore.dll!RenderingThreadMain(FEvent * TaskGraphBoundSyncEvent) Line 331 C++ Symbols loaded.
UE4Editor-RenderCore.dll!FRenderingThread::Run() Line 482 C++ Symbols loaded.
UE4Editor-Core.dll!FRunnableThreadWin::Run() Line 76 C++ Symbols loaded.
UE4Editor-Core.dll!FRunnableThreadWin::GuardedRun() Line 25 C++ Symbols loaded.
AND then comparing it to the callstack where it was being allocated (FSlateRHIRenderer::DrawWindow_RenderThread) but never freed, suggests that the loop that is calling this DrawWindow_RenderThread is not properly doing end-of-tick cleanup as seen in the normal flow above.
UE4Editor-Engine.dll!FRealtimeGPUProfilerFrame::PopEvent(FRHICommandListImmediate & RHICmdList) Line 370 C++ Symbols loaded.
UE4Editor-Engine.dll!FRealtimeGPUProfiler::PopEvent(FRHICommandListImmediate & RHICmdList) Line 610 C++ Symbols loaded.
[Inline Frame] UE4Editor-SlateRHIRenderer.dll!FScopedGPUStatEvent::{dtor}() Line 253 C++ Symbols loaded.
UE4Editor-SlateRHIRenderer.dll!FSlateRHIRenderer::DrawWindow_RenderThread(FRHICommandListImmediate & RHICmdList, FSlateRHIRenderer::FViewportInfo & ViewportInfo, FSlateWindowElementList & WindowElementList, bool bLockToVsync, bool bClear) Line 891 C++ Symbols loaded.
[Inline Frame] UE4Editor-SlateRHIRenderer.dll!FSlateRHIRenderer::DrawWindows_Private::__l29::<lambda_43e86ee7c51f39979e9a39d40b280024>::operator()(FRHICommandListImmediate &) Line 1050 C++ Symbols loaded.
[Inline Frame] UE4Editor-SlateRHIRenderer.dll!TEnqueueUniqueRenderCommandType<`FSlateRHIRenderer::DrawWindows_Private'::`29'::SlateDrawWindowsCommandName,<lambda_43e86ee7c51f39979e9a39d40b280024> >::DoTask(ENamedThreads::Type) Line 190 C++ Symbols loaded.
UE4Editor-SlateRHIRenderer.dll!TGraphTask<TEnqueueUniqueRenderCommandType<`FSlateRHIRenderer::DrawWindows_Private'::`29'::SlateDrawWindowsCommandName,<lambda_43e86ee7c51f39979e9a39d40b280024> > >::ExecuteTask(TArray<FBaseGraphTask *,FDefaultAllocator> & NewTasks, ENamedThreads::Type CurrentThread) Line 829 C++ Symbols loaded.
[Inline Frame] UE4Editor-Core.dll!FBaseGraphTask::Execute(TArray<FBaseGraphTask *,FDefaultAllocator> & CurrentThread, ENamedThreads::Type) Line 498 C++ Symbols loaded.
UE4Editor-Core.dll!FNamedTaskThread::ProcessTasksNamedThread(int QueueIndex, bool bAllowStall) Line 665 C++ Symbols loaded.
UE4Editor-Core.dll!FNamedTaskThread::ProcessTasksUntilQuit(int QueueIndex) Line 574 C++ Symbols loaded.
UE4Editor-RenderCore.dll!RenderingThreadMain(FEvent * TaskGraphBoundSyncEvent) Line 331 C++ Symbols loaded.
UE4Editor-RenderCore.dll!FRenderingThread::Run() Line 482 C++ Symbols loaded.
UE4Editor-Core.dll!FRunnableThreadWin::Run() Line 76 C++ Symbols loaded.
UE4Editor-Core.dll!FRunnableThreadWin::GuardedRun() Line 25 C++ Symbols loaded.
For a ‘quick’(not) test, I tried removing the new line added to that method that was collecting GPU stats:
//SCOPED_GPU_STAT(RHICmdList, SlateUI);
– doing this eliminated the leaking gpu profile data, but that wasn’t the only leak. It feels as though the loop around FSlateRHIRenderer::DrawWindow_RenderThread is missing some critical per-call cleanup.
Posting here hoping that someone who knows this code better has an aha moment.
Thanks.