Crash on Lumen Sparse Span Array

We observe the crash described in the callstack on UE 5.6 preview (last sync from //ue5/release-5.6 at head on May 20th, 2025) after playing an internal game prototype for about one hour with a standard engine distribution. Unfortunately, we can’t yet pinpoint the reason or trigger; at this point, the only thing we have to identify the crash is the callstack.

Creating a ticket in case other people run into this crash and have more information. Also worth noting we had reported this crash on 5.5 as well, here: [Content removed]

Please let us know if there are any patches or changes you’d like our team to try out.

Thank you!

Crash call stack:

LogThreadingWindows: Error: Runnable thread Background Worker #18 crashed. LogWindows: Error: === Critical error: === LogWindows: Error: LogWindows: Error: Assertion failed: (Index >= 0) & (Index < ArrayNum) [File:UE\Engine\Source\Runtime\Core\Public\Containers\Array.h] [Line: 1067] LogWindows: Error: Array index out of bounds: 1141492723 into an array of size 3831 LogWindows: Error: LogWindows: Error: LogWindows: Error: [Callstack] 0x00007ff7603c0dd8 FDebug::CheckVerifyFailedImpl2() [UE\Engine\Source\Runtime\Core\Private\Misc\AssertionMacros.cpp:728] LogWindows: Error: [Callstack] 0x00007ff763cdf7a5 TSparseSpanArray<FLumenMeshCards>::AddSpan() [UE\Engine\Source\Runtime\Renderer\Private\Lumen\LumenSparseSpanArray.h:46] LogWindows: Error: [Callstack] 0x00007ff763cddc06 FLumenSceneData::AddMeshCardsFromBuildData() [UE\Engine\Source\Runtime\Renderer\Private\Lumen\LumenMeshCards.cpp:997] LogWindows: Error: [Callstack] 0x00007ff763cdd724 FLumenSceneData::AddMeshCards() [UE\Engine\Source\Runtime\Renderer\Private\Lumen\LumenMeshCards.cpp:896] LogWindows: Error: [Callstack] 0x00007ff763ea087d UpdateSurfaceCachePrimitives() [UE\Engine\Source\Runtime\Renderer\Private\Lumen\LumenSceneRendering.cpp:1463] LogWindows: Error: [Callstack] 0x00007ff763e2b7c1 FDeferredShadingSceneRenderer::BeginUpdateLumenSceneTasks’::2'::<lambda_1>::operator()() [UE\Engine\Source\Runtime\Renderer\Private\Lumen\LumenSceneRendering.cpp:2092] LogWindows: Error: [Callstack] 0x00007ff763e65124 UE::Tasks::Private::TExecutableTaskBase<FRDGBuilder::AddSetupTask<FDeferredShadingSceneRenderer::BeginUpdateLumenSceneTasks'::2’::<lambda_1>,TArrayView<UE::Tasks::Private::FTaskHandle const ,int> >‘::2'::<lambda_1>,void,void>::ExecuteTask() [UE\Engine\Source\Runtime\Core\Public\Tasks\TaskPrivate.h:904] LogWindows: Error: [Callstack] 0x00007ff7600c64b3 UE::Tasks::Private::FTaskBase::TryExecuteTask() [UE\Engine\Source\Runtime\Core\Public\Tasks\TaskPrivate.h:528] LogWindows: Error: [Callstack] 0x00007ff760096a7f LowLevelTasks::TTaskDelegate<LowLevelTasks::FTask * __cdecl(bool),48>::TTaskDelegateImpl<LowLevelTasks::FTask::Init<UE::Tasks::Private::FTaskBase::Init'::2’::<lambda_1> >‘::13'::<lambda_1>,0>::CallAndMove() [UE\Engine\Source\Runtime\Core\Public\Async\Fundamental\TaskDelegate.h:171] LogWindows: Error: [Callstack] 0x00007ff7600a4a1e LowLevelTasks::FTask::ExecuteTask() [UE\Engine\Source\Runtime\Core\Public\Async\Fundamental\Task.h:627] LogWindows: Error: [Callstack] 0x00007ff7600a47ca LowLevelTasks::FScheduler::ExecuteTask() [UE\Engine\Source\Runtime\Core\Private\Async\Fundamental\Scheduler.cpp:364] LogWindows: Error: [Callstack] 0x00007ff7600c9fa1 LowLevelTasks::FScheduler::WorkerLoop() [UE\Engine\Source\Runtime\Core\Private\Async\Fundamental\Scheduler.cpp:724] LogWindows: Error: [Callstack] 0x00007ff76008e56a LowLevelTasks::FScheduler::CreateWorker’::2'::<lambda_1>::operator()() [UE\Engine\Source\Runtime\Core\Private\Async\Fundamental\Scheduler.cpp:188] LogWindows: Error: [Callstack] 0x00007ff76029c713 FThreadImpl::Run() [UE\Engine\Source\Runtime\Core\Private\HAL\Thread.cpp:69] LogWindows: Error: [Callstack] 0x00007ff7606fbdd8 FRunnableThreadWin::Run() [UE\Engine\Source\Runtime\Core\Private\Windows\WindowsRunnableThread.cpp:159] LogWindows: Error: [Callstack] 0x00007ff7606f491f FRunnableThreadWin::GuardedRun() [UE\Engine\Source\Runtime\Core\Private\Windows\WindowsRunnableThread.cpp:79] LogWindows: Error: [Callstack] 0x00007fff1af47374 KERNEL32.DLL!UnknownFunction [] LogWindows: Error: LogWindows: Error: Crash in runnable thread Background Worker #18 LogExit: Executing StaticShutdownAfterError LogWindows: FPlatformMisc::RequestExit(1, FRunnableThreadWin::GuardedRun.ExceptionHandler) LogWindows: FPlatformMisc::RequestExitWithStatus(1, 3, FRunnableThreadWin::GuardedRun.ExceptionHandler) LogCore: Engine exit requested (reason: Win RequestExit)

Hi, Chris!

We’re working on preparing a project you can receive for testing this and some of the other crashes we’re experimenting with 5.6; in the meantime, here is a ticket for another crash we see frequently in our build:

[Content removed]

We’ve fixed a crash that most surely was the reason behind this one. More details in this thread:

[Content removed]

Hi, after looking at the other memory stomp issue you posted I’m inclined to agree with you - have you been able to repro this crash after implementing that fix?

We haven’t been able to repro yet, and some people who had always seen crashes happening within the first ten to twenty minutes have now played continuously with no issues. This fix went to our codebase just today, but I think we can confidently say (with fingers crossed) that it solved this and our other sporadic memory addressing issues.

Hello there,

I’ve looked more into this issue, and I’d like to know if it would be possible to reproduce this in a non-packaged game with a breakpoint on the AddSpan function. It’s not immediately obvious to me how the array index can become so far out of bounds (Actually having 1.14 billion cards in the scene seems somewhat unlikely) without something going awry.

Are these crashes occuring at a similar point in the game/level? I’m wondering if there is a particular mesh that is causing this to occur.

Interestingly, the prior crash and the current one have slightly different callstacks. The current crash site involves creating a single additional card. Given that I cannot see how a literal could change, it would appear to indicate that FSpanAllocator::Allocate is returning a bad index.

Best regards,

Chris

Unfortunately we haven’t been able to isolate this crash in a standalone project, and the repro has been challenging to track down. We have one developer that gets it regularly with about 5-10 minutes of play, and he’s been able to work around it with these shadow settings:

r.ShadowQuality 0

r.Shadow.CSM.MaxCascades 1

r.Shadow.MaxResolution 512

r.Shadow.RadiusThreshold 0.06

r.Shadow.DistanceScale 0.6

r.Shadow.CSM.TransitionScale 0

r.Shadow.Virtual.Enable 0

The main setting that seems to make the difference is disabling VSM, the other settings are so the game doesn’t run out of memory with VSM disabled.

We might be able to bundle up the prototyping project and send it to a private / secure file drop if that would be helpful. The content uses virtual assets however, and I’m having issues trying to migrate the entire content folder.

Interesting that it would appear to be related to VSMs.

Would you mind if I ask for a complete log file for one of these crashes?

There is one case where this seems like it could occur. That being if the number of cards in an asset that seems to come from the DDC. Would it be possible to clear the local DDCs of the most crash-happy system to rule out a DDC issue?

I’ve reached out to Epic for guidance on safely receiving a project in this form. I’ll reply on this front when I receive said guidance.

Best regards,

Chris

We’re seeing the crash on cooked builds - would DDC be a factor in this case?

The build produced on 5.6 would have been using the same shared DDC, but as I understand building against a new version of the engine typically will force every asset to rebuild as opposed to using a potentially corrupt cache?

We’ve sent a build via DropBox that will hopefully help to repro/debug this issue. Jeremy Keays has the details.

I’m going to reassign this back to Epic as they may be able to better handle the repro project for the issue.

Best regards,

Chris