Announcement

Collapse
No announcement yet.

New Core Feature: Async Framework (Master Branch and 4.8)

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • New Core Feature: Async Framework (Master Branch and 4.8)

    Hi all,

    We started working on a new framework for asynchronous programming, which is located in the Core module. The goal is to simplify the process of writing code that executes asynchronously or in parallel. The Engine already had several mechanisms for this in place for a long time (threads, thread pool, task graph, etc.), but they required a fair amount of boilerplate code.

    The first iteration of this new framework attempts to reduce the amount of boilerplate to an absolute minimum. The two most important additions are the Async<T>() function template and the TFuture<T> type. If you used Java, C# or the newer version of the C++ STL then you may already be familiar with these concepts.


    What is Async<T>?

    The new Async<T> template function allows you to execute functions on another thread without having to write a lot of boilerplate code. There are currently three different execution methods available: TaskGraph, Thread and ThreadPool. All three methods will execute your function asynchronously, allowing the function to run in parallel with the calling thread. Which method you chose depends on the nature of your asynchronous tasks. Functions that are executed through Async<T> will return immediately. This means that the result of the function is actually not available right away, and some mechanism is needed to retrieve the result at a later time. This mechanism is called a Future.


    What is a Future?

    A future is a variable whose value will be set in the future. If you write a function that returns, say, a TFuture<int> instead of just an int, then your function tells its users that the integer return value is not available right away, but it promises to set the value at some time in the future. While your function is computing the result, the caller can meanwhile work on other things, but as soon as the caller attempts to access the actual value of TFuture<int>, its thread will block until that value has actually been set by you. Futures therefore provide a low-level mechanism to return results from functions that are executed asynchronously.


    When should I use Futures?

    Futures are most useful when your code requires results from one or more other functions that execute asynchronously, or in parallel. For example, consider the following scenario where the function Foo() computes and returns a result using three other functions that are computationally expensive:

    Code:
    int Foo()
    {
        int A = CalculateA();
        int B = CalculateB();
        int C = CalculateC();
    
        // other code here
        ....
    
        return A + B + C;
    }
    Note that the computation of the final result requires all of A, B and C. Traditionally, the order of execution would be sequential:

    Code:
    Thread 1:   |____CalculateA____|______CalculateB_____|___________CalculateC___________|__other code__|___A + B + C___|

    With the Async<T>() template function we are able to launch each of CalculateA, CalculateB and CalculateC asynchronously, which allows us to parallelize most of the work:

    Code:
    int Foo()
    {
       Future<int> A = Async<int>(EAsyncExecution::Thread, CalculateA);
       Future<int> B = Async<int>(EAsyncExecution::Thread, CalculateB);
       Future<int> C = Async<int>(EAsyncExecution::Thread, CalculateC);
    
        // other code here
        ....
    
        return A.Get() + B.Get() + C.Get();
    }
    The order of execution now looks something like this:

    Code:
    Thread 1:  |___Async()___|__other code__|///sleep///|___A + B + C___]
    Thread 2:  |____CalculateA____|
    Thread 2:    |_______CalculateB______|
    Thread 3:       |_____________CalculateC____________|

    Note that, here the three functions do not return integer results, but futures that will eventually hold the results. When Foo() has completed all its work and goes on to compute the result, it will block until all of A, B and C are actually available (indicated by "sleep" in the diagram above).


    When should I NOT use Futures?

    When your calling function does not actually care about the results of the asynchronous operations and does not need to block until the operations complete, you should not use futures. You can still execute such units of work asynchronously, and if some other code in your system needs to know about when they complete then it is generally better to use a mechanism using callbacks or delegates or instead. The key here is that the results of the async operations may be needed somewhere, but not in your calling code.


    When should I use TaskGraph vs. Thread vs. ThreadPool for async execution?

    Unreal Engine provides several means of parallelizing execution of tasks.

    The TaskGraph is shared by many other systems in the Engine and is intended for small tasks that are very short-running, never block, and must complete as soon as possible. Launching graph tasks is very cheap as compared to starting up threads, but you must ensure that your code does not block the TaskGraph ever. In particular, you should not set up Async() functions on the TaskGraph that in turn create other Async<T>() calls or may wait on some external event.This is very important, because if all worker threads are waiting then nothing else gets done in the Engine. If your code may block or create other asynchronous calls then use Thread or ThreadPool instead.

    Threads are quite expensive to create and best suited for long running tasks or tasks that may block. Operating systems generally impose limits on the number of threads that can be created, and they also slow down considerably once too many threads are alive at the same time. If you have many tasks (hundreds) or only want to maximize CPU utilization and do not care about all your tasks actually running in parallel at the same time, use ThreadPool instead.

    The ThreadPool is another set of worker threads that is independent from the TaskGraph system. It allows you to queue up an arbitrary number of threads, which will then be completed one after another based on the availability of worker threads. If your tasks do not fit into either TaskGraph or Thread, then execute them here.

    Note: A fourth mechanism for parallel execution, OS processes, is available in the Engine, but it is not exposed in Async<T>(). Use Thread or ThreadPool instead.


    Does this mean my algorithms are parallel now?

    No, futures and Async<T>() are low-level primitives that help reduce the boilerplate code required for asynchronous programming. They do not offer anything for automatically parallelizing your algorithms (although they may be used for a parallel programming library that we might implement in the future, but this is still pie in the sky).


    Does this mean my algorithms are thread-safe now?

    No, you are still responsible for ensuring that any code being executed asynchronously is completely thread-safe. Futures only guarantee thread-safety for the return values of your functions.




    Note: Examples of Async and TFuture can be found in /Runtime/Core/Tests/Async/AsyncTest.cpp. The implementation of Async itself also uses futures.
    Last edited by gmpreussner; 01-13-2015, 07:37 PM.

  • replied
    bump


    Error
    • Please enter a message with at least 10 characters (please_enter_message_x_chars)



    Leave a comment:


  • replied
    I use a simple bool set/reset in DoWork() what can be checked in the caller class when it has a tick function, probably not the most elegant but works.
    As an example, I reworked a plugin to make cover calculations async:
    https://github.com/s-ivan/CoverGener...oversAsync.cpp
    An example of using delegates would be cool. Probably I could do it, if I were not so lazy...

    Leave a comment:


  • replied
    Originally posted by Lordink View Post

    Can you elaborate? How do you "message" game thread to create AActors, for example? Is there a pipe/stream messaging API already built-in ue4, or are you making your own?
    I built for my projects a class with static mechanisms and static delegates that can be linked from AsyncTasks using "FSimpleDelegate::CreateStatic" like Mike Preussner suggested we should do in this same thread.

    There's usage samples in engine code, they are really simple to use together with AsyncTasks.
    They wrap parameters into structs and send them to "named" Game Thread, but I usually send object pointers instead because I need access to UProperty* objects (when I read/write user defined blueprint structs or create new UObjects within the DoWork() function.

    Leave a comment:


  • replied
    Originally posted by BrUnO XaVIeR View Post
    What you can't do:

    - Create Objects.
    - Destroy Objects.
    - Change Object Allocation Size.
    - Invoke or Broadcast Delegates.
    - Broadcast BP Events that belong to Game Thread.
    - Create Slate Widgets, etc, etc...


    What you CAN do:

    Setup your thread/task to avoid 'race condition'.
    Then you can do things like...

    - Message Game Thread to create a UObject "when possible".
    - The same to destroy an Object, add a message to queue.
    - Directly change allocation size and value of UProperty containers when using UProperty Helpers.
    - Message Game Thread to invoke Delegates "when ready".

    Depending on the size of your "task", when you message Game Thread, it may lock or slowdown if your thread is sending tons of messages every frame because then they won't in essence be asynchronous threads anymore.
    Can you elaborate? How do you "message" game thread to create AActors, for example? Is there a pipe/stream messaging API already built-in ue4, or are you making your own?

    Leave a comment:


  • replied
    With "UObject" I mean AActors, UActorComponents, etc.. these more complex classes.
    If you invoke multicast delegates from outside game's thread that runs functions within Game Thread all you'll get is a call to a check(IsInGameThread()) and Unreal gonna crash.

    I am currently writing multi-threaded modules (some for plugins too).
    If you create basic UObjects, it's fine, but more complex classes will pretty much always cause fatal issues.

    Leave a comment:


  • replied
    Care to reference where you're getting this information from? I'm curious because re. creation/destruction you've directly contradicted what an Epic engineer has written just a few posts up. Also the idea that you can't invoke delegates outside of the game thread makes no sense at all.

    Leave a comment:


  • replied
    What you can't do:

    - Create Objects.
    - Destroy Objects.
    - Change Object Allocation Size.
    - Invoke or Broadcast Delegates.
    - Broadcast BP Events that belong to Game Thread.
    - Create Slate Widgets, etc, etc...


    What you CAN do:

    Setup your thread/task to avoid 'race condition'.
    Then you can do things like...

    - Message Game Thread to create a UObject "when possible".
    - The same to destroy an Object, add a message to queue.
    - Directly change allocation size and value of UProperty containers when using UProperty Helpers.
    - Message Game Thread to invoke Delegates "when ready".

    Depending on the size of your "task", when you message Game Thread, it may lock or slowdown if your thread is sending tons of messages every frame because then they won't in essence be asynchronous threads anymore.

    Leave a comment:


  • replied
    I've found it impossible to get any real information out of Epic staff on even just a basic idea of what is and isn't safe regarding UObjects and threads.

    Still, I think it's fair to say that even reading properties isn't safe to do from another thread unless those properties are your own and you can guarantee they're not being written to on some other thread. In the case of things like component transforms this clearly isn't the case - the engine may be updating them, so you can't just read them from a background thread. What you'd want to do is have some code being run on the game thread that grabs the data you need and hands a copy of it off to the worker thread to process.

    Leave a comment:


  • replied
    Any news on the continuations? Coming from web programming and C# (where we already have both chained promises and the awesome async-await), it feels incredibly hard and time-consuming to implement asynchronous and optionally-parallel computations in UE4. We're in C++, though, and I understand how hard it most certainly is to implement such a thing so that it works on at least most platforms consistently.

    Question - I understand it's not desirable and safe to change any properties of uobjects in async tasks and other threads, but how about just accessing their properites?
    Example: I have a complex perception and goal prioritization AI system, which takes all the available (overlapping, for example) actors, extracts possible actions from their components and ranks them based on their weight (which is determined by const function calls, so no side effects, only calculations). In crowds, it becomes incredibly slow to spawn any new AIs, cause they start studying and ranking all 100+ of other guys around them and pretty much freeze the game thread.

    I feel like it would be a perfect thing to offload the whole evaluation on separate thread(s) and call delegate to make AI decision as soon as all evaluation is done. However, the evaluation "reads" overlaps, actor locations etc. I have not yet attempted to try and make it work, but do you think there are any problems with this approach?

    Leave a comment:


  • replied
    Should I used Async when I don't care about the result?

    Leave a comment:


  • replied
    I'm really interested in this feature. I am trying to create 8 TaskGraphs and in each one, I create 4 TaskGraphs. Do you think this way may cause some problems? So can I create some TaskGraphs in another TaskGraph?
    Last edited by YuchenMei; 06-13-2017, 08:28 AM.

    Leave a comment:


  • replied
    [MENTION=1986]gmpreussner[/MENTION], can you elaborate a bit on what that means in practice?

    You're saying it's okay to call NewObject outside of the game thread? If so that's news to me. Is there some synchronization going on to prevent the GC from snaffling the object before we have chance to root it or reference it somewhere?

    And then what about safely setting a UObject* property somewhere, since the GC could be reading those at any time, right?

    I'd previously assumed the only potentially safe way to use a UObject in a background thread was to pass a pointer from the game thread to an existing object that you know is guaranteed to remain alive, and even then to pretty much stick to only accessing code and member variables that we control ourselves.

    Leave a comment:


  • replied
    UObject creation and destruction is thread-safe. However, this doesn't mean that UObject usage in general is thread-safe. You are still responsible for writing thread-safe code yourself. It is probably safe to assume that none of the built-in methods are thread-safe.

    Leave a comment:


  • replied
    Originally posted by iniside View Post
    Is there any info about UObject being able to be accessed from other threads/run functions on other threads and being created from other threads ?
    The last information about it I found is from 2014 and it "working on making UObject thread safe". but it doesn't seem to have any progress since then.
    I'd like to know if anything has changed as well... Last time I checked UObjects can't be touched outside game thread.

    Leave a comment:

Working...
X