Game freeze when many tasks are launched

Development Platform & Builds
, ,
1

In our current project, we are experiencing frequent freezes in our windows packaged build.

I managed to reproduce the issue 100% with UE5.6 from the Epic launcher (see included project)

In short, the call to FTaskGraphInterface::Get().WaitUntilTaskCompletes never return.

We don’t seem to have the problem in Editor.

2

Steps to Reproduce

  1. Start the provided 5.6 project in Editor
  2. Check that you can start and stop playing
  3. Generate a package for Windows development
  4. Start the packaged build
  5. It is frozen
  6. Attach a debugger to confirm the freeze
3

Hi Mathias,

Thanks for sharing this. I’ve narrowed it down to this using the new task API.

It boils down to a background priority task having a normal priority task as its prerequisite.

`UE::Tasks::FTask Task =
UE::Tasks::Launch(
TEXT(“Task”),
{ FPlatformMisc::LowLevelOutputDebugStringf(TEXT(“Subsequent Task\n”)); },
UE::Tasks::Launch(TEXT(“Task”), { FPlatformMisc::LowLevelOutputDebugStringf(TEXT(“Initial Task\n”)); }),
LowLevelTasks::ETaskPriority::BackgroundNormal
);

while (!Task.IsCompleted())
{
}`It’s generally not very good for performance to have long series of tasks ping-ponging between priorities like in your example. But obviously, this is a bug in the taskgraph that’s we’ll fix. I’ll let you know as soon as there is a CL to fix this.

In the meantime, you can try this simple change and confirm to us if it fixes your issue.

In TaskPrivate.h, change the Close() implementation to always wake up a worker.

It will bypasses an optimization where we schedule the next subsequent to be run by the same thread. In this case, the condition is wrong because the thread executing the task is a normal thread, and the subsequent is a background task so it cannot be executed by the current thread and we need to wake up another one instead.

Ideally, the final fix should preserve the optimization, but will need to be aware of tasks priorities.

`void Close()
{
TASKGRAPH_VERBOSE_EVENT_SCOPE(FTaskBase::Close);
checkSlow(!IsCompleted());

// Push the first subsequent to the local queue so we pick it up directly as our next task.
// This saves us the cost of going to the global queue and performing a wake-up.
// But if we’re a task event, always wake up new workers because the current task could continue executing for a long time after the trigger.

// BEGIN EDIT
bool bWakeUpWorker = true; //ExtendedPriority == EExtendedTaskPriority::TaskEvent;
// END EDIT`Ideally, the final fix should preserve the optimization, but will need to be aware of tasks priorities.

Thanks

Danny

4

Hi Mathias,

Thanks again for reporting this. It was a pretty nasty bug that as gone under the radar for far too long!

The fix is now in UE5/Main @ 44661181.

Thanks

Danny

5

You can revert the workaround once you have the real fix to get your performance back :slight_smile:

6

Hi Danny!

Yesterday I integrated your workaround. So far, it seems to fix the issue. Thanks!

I’ll integrate 44661181 soon.

Cheers,

Mathias