SetLastRenderTime runs on primitives scheduled for deletion

Development Rendering
, ,
1

This seems to be a clear thread data race, where primitives are scheduled for deletion by a task, and a relevance update task tries to set the render time, resulting in a null pointer access on an std::atomic.

Putting a sleep between the nullptr check and using the value provokes the crash to happen more deterministically, indicating one thread is killing what OwnedLastRenderTimePtr is pointing to in parallel with it being accessed.

Using an atomic exchange to guard OwnedLastRenderTimePtr with a spinlock fixes the problem.

[Attachment Removed]

2

Steps to Reproduce
Put a little sleep before accessing OwnedLastRenderTimePtr->SetLastRenderTime in PrimitiveSceneInfoData.cpp, load a level, and most of the times (in our case) you get a null pointer access on the atomic that resides within what OwnedLastRenderTimePtr points to.

[Attachment Removed]

3

Hello,

Thank you for reaching out.

I’ve been assigned this issue, and we will be looking into this nullptr reference for you.

We were unable to reproduce the crash as described.

Can you please provide more detailed reproduction steps, and what CL you are testing on?

The guide for EPS submissions:

[Content removed]

[Attachment Removed]

4

We are using 5.7.1 I believe, we sync the engine periodically because we don’t just work in-engine.

It’s difficult to track down what it is exactly that is triggering this to happen, but I am getting about 18k relevance tasks, so I’m assuming the system has to be stressed a bit to trigger it.

But when we put that sleep in, multiple threads trigger the nullptr access so it’s not difficult for us to reproduce it with our level.

I just need to load the level to reproduce it, if I have put that sleep in the code. We also got it a few times when we were playtesting.

Thanks,

Gustav

[Attachment Removed]

5

We are hitting this crash as well. Not easily reproducible on demand.

Unhandled Exception: EXCEPTION_ACCESS_VIOLATION writing address 0x0000000000000000

if (OwnerLastRenderTimePtr)

{

-> OwnerLastRenderTimePtr->SetLastRenderTime(InLastRenderTime);

}

Unhandled Exception: EXCEPTION_ACCESS_VIOLATION writing address 0x0000000000000000

WFOSteam!FPrimitiveSceneInfoData::SetLastRenderTime() [E:\Octo\Engine\Source\Runtime\Engine\Private\PrimitiveSceneInfoData.cpp:21]

WFOSteam!FPrimitiveSceneInfo::UpdateComponentLastRenderTime() [E:\Octo\Engine\Source\Runtime\Renderer\Private\PrimitiveSceneInfo.cpp:2548]

WFOSteam!FProjectedShadowInfo::AddSubjectPrimitive_AnyThread() [E:\Octo\Engine\Source\Runtime\Renderer\Private\ShadowSetup.cpp:2291]

WFOSteam!FGatherShadowPrimitivesPacket::FilterPrimitiveForShadows() [E:\Octo\Engine\Source\Runtime\Renderer\Private\ShadowSetup.cpp:5062]

WFOSteam!FGatherShadowPrimitivesPacket::AnyThreadTask() [E:\Octo\Engine\Source\Runtime\Renderer\Private\ShadowSetup.cpp:4879]

WFOSteam!`FGatherShadowPrimitivesPrepareTask::DoTask’::`13’::<lambda_1>::operator()() [E:\Octo\Engine\Source\Runtime\Renderer\Private\ShadowSetup.cpp:5180]

WFOSteam!TGraphTask<TFunctionGraphTaskImpl<void __cdecl(void),0> >::ExecuteTask() [E:\Octo\Engine\Source\Runtime\Core\Public\Async\TaskGraphInterfaces.h:697]

WFOSteam!UE::Tasks::Private::FTaskBase::TryExecuteTask() [E:\Octo\Engine\Source\Runtime\Core\Public\Tasks\TaskPrivate.h:519]

WFOSteam!LowLevelTasks::TTaskDelegate<LowLevelTasks::FTask * __cdecl(bool),48>::TTaskDelegateImpl<`LowLevelTasks::FTask::Init<`UE::Tasks::Private::FTaskBase::Init’::`2’::<lambda_1> >‘::`13’::<lambda_1>,0>::CallAndMove() [E:\Octo\Engine\Source\Runtime\Core\Public\Async\Fundamental\TaskDelegate.h:171]

WFOSteam!LowLevelTasks::FTask::ExecuteTask() [E:\Octo\Engine\Source\Runtime\Core\Public\Async\Fundamental\Task.h:627]

WFOSteam!LowLevelTasks::FScheduler::ExecuteTask() [E:\Octo\Engine\Source\Runtime\Core\Private\Async\Fundamental\Scheduler.cpp:396]

WFOSteam!LowLevelTasks::FScheduler::WorkerLoop() [E:\Octo\Engine\Source\Runtime\Core\Private\Async\Fundamental\Scheduler.cpp:757]

WFOSteam!`LowLevelTasks::FScheduler::CreateWorker’::`2’::<lambda_1>::operator()() [E:\Octo\Engine\Source\Runtime\Core\Private\Async\Fundamental\Scheduler.cpp:220]

WFOSteam!FThreadImpl::Run() [E:\Octo\Engine\Source\Runtime\Core\Private\HAL\Thread.cpp:69]

WFOSteam!FRunnableThreadWin::Run() [E:\Octo\Engine\Source\Runtime\Core\Private\Windows\WindowsRunnableThread.cpp:159]

[Attachment Removed]

6

Hello,

We have been unable to reproduce this testing on 5.7, CL 51070971, and 5.7.1, CL: 48512491, both with the “sleep()” you suggested.

Have you been able to reproduce this in a vanilla install, plus the “sleep()”?

Do you have any engine modifications around this or Compute View Relevance?

[Attachment Removed]

7

We are on 5.7.1.

Apologies - I just saw this:

  • UE-341565 [CrashReport][Crash] Engine!FPrimitiveSceneInfoData::SetLastRenderTime() [PrimitiveSceneInfoData.cpp]

I will look into this fix. Thanks.

[Attachment Removed]

8

Hello,

Thank you for sharing this finding.

The CL related to that report is: 49431941 in //UE5/Release-5.7 (part of 5.7.2 hotfix)

GitHub link for convenience: https://github.com/EpicGames/UnrealEngine/commit/263b1d2b6046b3123ec698ecd8bea1482e1eec21

Please let us know if it solves the crash for you.

[Attachment Removed]