We’ve noticed an intermittent crash within the Virtual Texture Updater since upgrading to UE5.5. It crashes at the following line when trying to call the desctructor for one of the objects allocated via
FConcurrentLinearBulkObjectAllocator Allocator;
[Image Removed]
We don’t have a reliable repro yet and haven’t been able to probe this further - only a hunch that it seems like a memory stomp/write overflow. I did try running the game with `-stompmalloc` but it doesn’t look like that is hooked up to `FConcurrentLinearBulkObjectAllocator`.
Have you experienced anything like this or have tips on how to isolate the issue?
> [Inline Frame] UnrealEditor-Renderer.dll!TConcurrentLinearBulkObjectAllocator<FDefaultBlockAllocationTag>::BulkDelete() Line 755 C++ [Inline Frame] UnrealEditor-Renderer.dll!TConcurrentLinearBulkObjectAllocator<FDefaultBlockAllocationTag>::{dtor}() Line 746 C++ UnrealEditor-Renderer.dll!TDefaultDelete<FVirtualTextureUpdater>::operator()(FVirtualTextureUpdater * Ptr) Line 66 C++ UnrealEditor-Renderer.dll!FDeferredShadingSceneRenderer::Render(FRDGBuilder & GraphBuilder) Line 3591 C++ UnrealEditor-Renderer.dll!RenderViewFamilies_RenderThread(FRHICommandListImmediate & RHICmdList, const TArray<FSceneRenderer *,TSizedDefaultAllocator<32>> & SceneRenderers) Line 5439 C++ UnrealEditor-Renderer.dll!FRendererModule::BeginRenderingViewFamilies::__l96::<lambda_3>::operator()(FRHICommandListImmediate & RHICmdList) Line 5742 C++ [Inline Frame] UnrealEditor-RenderCore.dll!UE::Core::Private::Function::TFunctionRefBase<UE::Core::Private::Function::TFunctionStorage<1>,void __cdecl(FRHICommandListImmediate &)>::operator()(FRHICommandListImmediate &) Line 470 C++ UnrealEditor-RenderCore.dll!FRenderThreadCommandPipe::EnqueueAndLaunch::__l5::<lambda_1>::operator()() Line 1541 C++ [Inline Frame] UnrealEditor-RenderCore.dll!UE::Core::Private::Function::TFunctionRefBase<UE::Core::Private::Function::TFunctionStorage<1>,void __cdecl(void)>::operator()() Line 470 C++ [Inline Frame] UnrealEditor-RenderCore.dll!TFunctionGraphTaskImpl<void __cdecl(void),1>::DoTaskImpl(TUniqueFunction<void __cdecl(void)> & Function, ENamedThreads::Type) Line 1720 C++ [Inline Frame] UnrealEditor-RenderCore.dll!TFunctionGraphTaskImpl<void __cdecl(void),1>::DoTask(ENamedThreads::Type) Line 1713 C++ UnrealEditor-RenderCore.dll!TGraphTask<TFunctionGraphTaskImpl<void __cdecl(void),1>>::ExecuteTask() Line 644 C++ UnrealEditor-Core.dll!UE::Tasks::Private::FTaskBase::TryExecuteTask() Line 510 C++ [Inline Frame] UnrealEditor-Core.dll!FBaseGraphTask::Execute(TArray<FBaseGraphTask *,TSizedDefaultAllocator<32>> &) Line 492 C++ UnrealEditor-Core.dll!FNamedTaskThread::ProcessTasksNamedThread(int QueueIndex, bool bAllowStall) Line 779 C++ UnrealEditor-Core.dll!FNamedTaskThread::ProcessTasksUntilQuit(int QueueIndex) Line 668 C++ UnrealEditor-RenderCore.dll!RenderingThreadMain(FEvent * TaskGraphBoundSyncEvent) Line 317 C++ UnrealEditor-RenderCore.dll!FRenderingThread::Run() Line 443 C++ UnrealEditor-Core.dll!FRunnableThreadWin::Run() Line 159 C++ UnrealEditor-Core.dll!FRunnableThreadWin::GuardedRun() Line 79 C++ kernel32.dll!00007fff336c7374() Unknown ntdll.dll!RtlUserThreadStart() Unknown
The intermittent crash you’re encountering when calling the Allocation’s destructor could stem from multi-threading issues, such as another thread may already have deleted Allocation, causing the destructor to operate on an invalid memory location or Allocation->Next may point to invalid memory if the linked list managed by FAllocation* gets corrupted (e.g., through invalid writes or concurrent modifications). As far as I understand, using std::memory_order_acquire in Next.exchange() ensures that changes made by other threads are visible, but doesn’t prevent race conditions on FAllocation (std::memory_order_seq_cst can prvodie stricter synchronization, but is also slower).
These are hard to debug, but you could use a conditional breakpoint at the crash line to break when the Allocation is invalid. At that point, you can inspect Visual Studio’s Parallel Stacks and Parallel Watch windows (this link has more info).
Please let me know if this helps or if you have further questions. If you manage to find a reliable repro for the crash, I can help debug futher.
Thanks for your response. Unfortunately we don’t have a reliable repro. Since all crash instances are from the editor, I turned off `r.VT.AsyncPageRequestTask` in DefaultEditor.ini. I’ll keep an eye out to see if that helps.
Anecdotally, these crashes happen when you change viewmodes or launch PIE, etc.
Thank you for the update. Changing view modes/PIE launch can provide some clues as to why this crash is happening.
One more question: you mentioned that the crash is happening since upgrading to UE5.5. Does that mean you did not see this crash in earlier versions? If so, which minor version of 5.5 are you using?
thanks for that. Unfortunately I haven’t found anything in the change logs for 5.5 that would provide a clue regarding this crash. I will escalate the ticket to Epic so someone with more expertise on this matter can have a look.
I was unable to find an known issue or fix for a similar the callstack. The closest was this CL that fixed an issue with the staging buffer size.
CL#40680882 (5682f0e) Fix crash with incorrect sized staging buffer.
You may want to try integrating this if you find a way to reproduce the crash, but also take a look at the history of changes in Engine/Source/Runtime/Renderer/Private/VT/VirtualTextureFeedback.cpp