Hi all,
We started working on a new framework for asynchronous programming, which is located in the Core module. The goal is to simplify the process of writing code that executes asynchronously or in parallel. The Engine already had several mechanisms for this in place for a long time (threads, thread pool, task graph, etc.), but they required a fair amount of boilerplate code.
The first iteration of this new framework attempts to reduce the amount of boilerplate to an absolute minimum. The two most important additions are the Async<T>() function template and the TFuture<T> type. If you used Java, C# or the newer version of the C++ STL then you may already be familiar with these concepts.
What is Async<T>?
The new Async<T> template function allows you to execute functions on another thread without having to write a lot of boilerplate code. There are currently three different execution methods available: TaskGraph, Thread and ThreadPool. All three methods will execute your function asynchronously, allowing the function to run in parallel with the calling thread. Which method you chose depends on the nature of your asynchronous tasks. Functions that are executed through Async<T> will return immediately. This means that the result of the function is actually not available right away, and some mechanism is needed to retrieve the result at a later time. This mechanism is called a Future.
What is a Future?
A future is a variable whose value will be set in the future. If you write a function that returns, say, a TFuture<int> instead of just an int, then your function tells its users that the integer return value is not available right away, but it promises to set the value at some time in the future. While your function is computing the result, the caller can meanwhile work on other things, but as soon as the caller attempts to access the actual value of TFuture<int>, its thread will block until that value has actually been set by you. Futures therefore provide a low-level mechanism to return results from functions that are executed asynchronously.
When should I use Futures?
Futures are most useful when your code requires results from one or more other functions that execute asynchronously, or in parallel. For example, consider the following scenario where the function Foo() computes and returns a result using three other functions that are computationally expensive:
int Foo()
{
int A = CalculateA();
int B = CalculateB();
int C = CalculateC();
// other code here
....
return A + B + C;
}
Note that the computation of the final result requires all of A, B and C. Traditionally, the order of execution would be sequential:
Thread 1: |____CalculateA____|______CalculateB_____|___________CalculateC___________|__other code__|___A + B + C___|
With the Async() template function we are able to launch each of CalculateA, CalculateB and CalculateC asynchronously, which allows us to parallelize most of the work:
int Foo()
{
Future<int> A = Async<int>(EAsyncExecution::Thread, CalculateA);
Future<int> B = Async<int>(EAsyncExecution::Thread, CalculateB);
Future<int> C = Async<int>(EAsyncExecution::Thread, CalculateC);
// other code here
....
return A.Get() + B.Get() + C.Get();
}
The order of execution now looks something like this:
Thread 1: |___Async()___|__other code__|///sleep///|___A + B + C___]
Thread 2: |____CalculateA____|
Thread 2: |_______CalculateB______|
Thread 3: |_____________CalculateC____________|
Note that, here the three functions do not return integer results, but futures that will eventually hold the results. When Foo() has completed all its work and goes on to compute the result, it will block until all of A, B and C are actually available (indicated by āsleepā in the diagram above).
When should I NOT use Futures?
When your calling function does not actually care about the results of the asynchronous operations and does not need to block until the operations complete, you should not use futures. You can still execute such units of work asynchronously, and if some other code in your system needs to know about when they complete then it is generally better to use a mechanism using callbacks or delegates or instead. The key here is that the results of the async operations may be needed somewhere, but not in your calling code.
When should I use TaskGraph vs. Thread vs. ThreadPool for async execution?
Unreal Engine provides several means of parallelizing execution of tasks.
The TaskGraph is shared by many other systems in the Engine and is intended for small tasks that are very short-running, never block, and must complete as soon as possible. Launching graph tasks is very cheap as compared to starting up threads, but you must ensure that your code does not block the TaskGraph ever. In particular, you should not set up Async() functions on the TaskGraph that in turn create other Async<T>() calls or may wait on some external event.This is very important, because if all worker threads are waiting then nothing else gets done in the Engine. If your code may block or create other asynchronous calls then use Thread or ThreadPool instead.
Threads are quite expensive to create and best suited for long running tasks or tasks that may block. Operating systems generally impose limits on the number of threads that can be created, and they also slow down considerably once too many threads are alive at the same time. If you have many tasks (hundreds) or only want to maximize CPU utilization and do not care about all your tasks actually running in parallel at the same time, use ThreadPool instead.
The ThreadPool is another set of worker threads that is independent from the TaskGraph system. It allows you to queue up an arbitrary number of threads, which will then be completed one after another based on the availability of worker threads. If your tasks do not fit into either TaskGraph or Thread, then execute them here.
Note: A fourth mechanism for parallel execution, OS processes, is available in the Engine, but it is not exposed in Async<T>(). Use Thread or ThreadPool instead.
Does this mean my algorithms are parallel now?
No, futures and Async<T>() are low-level primitives that help reduce the boilerplate code required for asynchronous programming. They do not offer anything for automatically parallelizing your algorithms (although they may be used for a parallel programming library that we might implement in the future, but this is still pie in the sky).
Does this mean my algorithms are thread-safe now?
No, you are still responsible for ensuring that any code being executed asynchronously is completely thread-safe. Futures only guarantee thread-safety for the return values of your functions.
[HR][/HR]
Note: Examples of Async and TFuture can be found in /Runtime/Core/Tests/Async/AsyncTest.cpp. The implementation of Async itself also uses futures.