ke (KismetEvent) command can deadlock if it flushes async loading (with fix)

This is something we ran into while debugging a different hang. To hit it, you need some blueprint event that, when run, will end up calling FlushAsyncLoading to wait for something that will create a new UObject.

We saw it like this:

  1. Use the ke (kismetevent) command to invoke an event on loaded objects of a particular class.
  2. This will execute UEngine::HandleKismetEventCommand. It uses FThreadSafeObjectIterator to iterate the object array, looking for matching objects. When it finds one, it calls the named event on it using CallFunctionByNameWithArguments.
  3. The event (in our case) calls LoadSynchronous to load something from a soft object pointer. This starts loading on the async loading thread, and the game thread waits for this with FlushAsyncLoading.
  4. On the async loading thread, it loads something that needs to create a new object, which gets added to the engine’s object array.
  5. Adding the object attempts to lock the array (FUObjectArray::LockInternalArray).
  6. The game thread already holds the lock (due to the FThreadSafeObjectIterator that’s still active), so the async loading thread blocks until it’s free. But the game thread will never release it, because it’s waiting for async loading to complete.

I have a fix in UEngine::HandleKismetEventCommand that gathers the matching objects first (using FThreadSafeObjectIterator), then calls CallFunctionByNameWithArguments on them in a separate loop that doesn’t hold the object array lock. The relevant code looks like this:

// Call it on all instances of the class
TArray<UObject*> MatchingObjects;
for (FThreadSafeObjectIterator It(ClassToMatch); It; ++It)
{
	UObject* const Obj = *It;
	UWorld const* const ObjWorld = Obj->GetWorld();
	if (ObjWorld == InWorld)
	{
		MatchingObjects.Add(Obj);
	}
}

int32 NumInstanceCallsSucceeded = 0;
for (UObject* Obj : MatchingObjects)
{
	const bool bSucceeded = Obj->CallFunctionByNameWithArguments(Cmd, Ar, nullptr, true);
	NumInstanceCallsSucceeded += bSucceeded ? 1 : 0;
}

Are there any gotchas here I might be missing? This does solve the deadlock in our repro.

This is a fairly specific issue, but sent us down the wrong path for a while when investigating a different hang, so I also wanted to share it here in case anyone else runs into it.

[Attachment Removed]

Hello! Thanks for reporting this. I was able to repro the deadlock using your info.

I filed a public bug UE-373054 (visible later) and submitted an engine fix for it. It applies your fix, that addresses

ke /Script/MyModule.MyClass MyFuncdeadlocking if MyFunc flushes async loading, but also the variant

ke * MyFuncwhich would attempt to call MyFunc on any UObject.

The issue tracker page should be visible later this week, while the engine fix shouldn’t be far off either. Thanks for reporting this!

[Attachment Removed]

The fix has been merged in //UE5/Main/ at CL 52512373

[Attachment Removed]

Thanks! I hadn’t even looked at the ke * case here.

[Attachment Removed]