Deadlock after updating to UE 5.6

Development Platform & Builds
, ,
1

The DeadLock came with the new UE 5.6.1 update and the reason are the UE changes in UObject* StaticAllocateObject(…) function, there are three new input arguments

int32 SerialNumber, FRemoteObjectId RemoteId, FGCReconstructionGuard* GCGuard of which for interest for us is FGCReconstructionGuard* GCGuard. The big change is on Line 3732 UObjectGlobals.cpp

if (ensureMsgf(IsInGameThread(), TEXT("GC lock can only be acquired on the game thread. If you hit this ensure then an object is being reconstructed on a worker thread which is not thread-safe")) && GCGuard)
{
	GCGuard->Lock();
}

This will lock GCGuard and might block all threads waiting on GCGuard, including RenderThread.

Before final destruction the UObject must released all resources, rendering resources are released on RenderThread.

GameThread wants to be sure that all resources are released before the final destruction of the object and for that reason it sets a Rendering Fence. Fence is just a very simple task which should run on RenderThread and will be executed after all object resources are released.

In this case before releasing resources RenderThread executes render tasks which are using textures, to access these textures Renderer/RenderTask has to Pin texture WeakPtr. When WeakPtr is Pinned it calls

TStrongObjectPtr<UObject> FWeakObjectPtr::Internal_Pin(bool bEvenIfGarbage) const
{
	FGCScopeGuard GCScopeGuard;
	FUObjectItem* const ObjectItem = Internal_GetObjectItem();
	return TStrongObjectPtr<UObject>(((ObjectItem != nullptr) && GUObjectArray.IsValid(ObjectItem, bEvenIfGarbage)) ? (UObject*)ObjectItem->GetObject() : nullptr);
}

The FGCScopeGuard constructor will call FGCCSyncObject::Get().LockAsync(); which will stay in infinite waiting because FGCCSyncObject waits to be signaled from the GameThread, this waiting is blocking the task execution on the RenderThread. As result our Fence set from the GameThread will never be executed and with that GamThread will stay in infinite While loop

while (!Obj->IsReadyForFinishDestroy()) 
{
				// If we're not in the editor, and aren't doing something specifically destructive like reconstructing blueprints, this is fatal
				if (!bPrinted && !GIsEditor && FApp::IsGame() && !GIsReconstructingBlueprintInstances)
				{
					StallStart = FPlatformTime::Seconds();
					bPrinted = true;
				}
				FPlatformProcess::Sleep(0);
}

Obj-&gt;IsReadyForFinishDestroy() will never return True because the render fence task will never be executed on RenderThread

2

Steps to Reproduce
Each time running the game in PIE

3

Hi,

apologies for the long wait, I’ve been swamped a bit and only just got to catch up.

In this case the wait might have been useful, since a colleague just submitted a change to that logic today, specifically restricting StaticAllocateObject() to only lock GCGuard in async loading scenarios.

I think this should also work for you, since in your case you’re spawning a BP actor by duplicating a template and no loading is involved.

The change should be available on Github soon as 3b35031d. Right now it looks like it hasn’t been synced from our repo yet, but the change itself is trivial, just introducing an additional if around the lock:

// Take the GC lock if the object is still being async loaded to avoid any race condition, otherwise we don't care about the lock
if (Obj->HasAnyInternalFlags(EInternalObjectFlags_AsyncLoading))
{
	GCGuard->Lock();
}

Could you try this out and let us know if it solves the deadlock for you?

Thank you!

Best,

Sebastian

4

Hi [mention removed]​

Thanks for looking into this and for the offered solution.

I integrated that into our code base, seems like it works fine and solved the problem.

Regards

/Todor

5

Perfect, thanks for letting us know!

This submit just missed the 5.7 cutoff, so it will likely only be released with 5.8, so you might have to stick with the local fix for a bit.

Best,

Sebastian