I have several asynchronous nodes to load various classes assets (Async Load Class Asset), and I want to continue my code only when all these nodes have completed.
Is there a simple way of doing this with an existing flow control node, or macro ?
I can see how this can be achieved with a custom graph involving an “AsyncDone” boolean and recursively calling itself… but I am wondering if there is a more idiomatic way to do this. Basically, this is a delayed gate, that resumes execution once all input control paths are triggerred at least once.
Attached is a simplified picture of what I want to do
Yes, I see. You mean simply chaining loads using the “completed” output (and not using the regular output).
But doesn’t this break potential optimization here ? suppose I have a bunch (a big array) of assets to load, and not only 2 like in the picture, chaining them together on completion would load them sequentially, while - I suppose (?) - chaining them with the regular path would give opportunity to load them in different parallel tasks.
In fact the question I am asking is more general: suppose I have a set of async/delayed nodes (could be timers/etc.), which might be started / finished at different points in time. Is there any control flow node that is able to receive all the completed outputs and start only when all have been finished ?
Thanks for your answer. Indeed, that’s possible async load operations are not multi-threaded (I thought they would be!), I’d be interested to hear also
So finally, I’ve made my own macro: AsyncLoadClassArray
Basically, it takes an array of soft class references, async load each item in the array, and only returns once ALL have been loaded. Hence acting as a “barrier”.
The actual implementation is the following (debug print code have been stripped in the screenshot):
I use a local variable “load pending” that gets assigned to the number of items in the array once the ForEach loop is completed, and decremented each time an async load is completed. Once the load pending is below zero, we can safely assume everything have been loaded and exit. There is also another local variable to store the result classes.
As you may see, async loads are not waiting for the previous load to be completed, allowing for potential parallelism under the hood. Whether this is really the case is something to be checked though, I don’t know really.
The macro is used like this:
Tested on the Cropout Sample project for the Spawn Data, and it works fine:
I think I have found a bug in UE though: the macro is not marked as “Latent” (no clock icon), even though it use a latent node. This will cause issue if you try to put the macro call in a function. The only way I found to flag the macro as latent was to add an unused latent node in the graph. For some reason, the Async Load Class Asset function does not do the job.
I could be wrong, but don’t think for loops plays well with latent nodes. There is a good chance those classes are already in memory and it immediately returned the reference.
Hmmm… I don’t know. The behavior looks like a bit confusing at a first grlance, as once the For loop ends, the latent nodes in the loop body are “restarted” at a later point in time (when they have completed). But this is the intended behavior in that case.
I’ll test a bit more though (the case with classes already in memory => does it completes immediatly ?), and will look in the forums for potential issues between latent nodes and for loops. Thanks for your advice/warning !
About the Asset Manager: I tried to give a peek at it (with the Blueprint API), but I am not sure about what to do about all those functions using PrimaryAssetId structure. My soft classes references do not have valid Primary Asset Id.
Well, for sure my knowledge of UE is not yet good enough to really understand all the subtleties of Asset Management
So I’ll consider trying to do differently, because due to your feedback, it is possible that what I am trying to do is too much at the bleeding-edge for BP scripting.
In fact, all of this started from analysis at the Cropout Exampe sample (for learning purposes) where they use async loading of assets using Blueprint code, but I found it was a bit strange: basically they are calling Async Load stuff in a recursive maneer (an event that calls itself until all elements of the array have been traversed), and at each completion they increment a value. When this value is above the array size, they put an AsyncComplete bool to true. Somewhere else, there is a timer that checks for every secs if the bool is true, and then, they proceed with the script. I found this to be a bit of bad/weird design, so I was trying to find a better solution (= a gate that would block until everything is loaded, instead of a timer that periodically checks if a boolean is true )
@pezzott1 Yeah, there is quite a lot of spawning code in this project. Look in the “BP_Spawner” blueprint (in the plugins)
There is the AsyncLoadClasses event (the famous recursive async loading thing that puts a bool to true). Basically it tries to load all the BP assets of the SpawnTypes array (which contains Soft Class References). The BP itself have this array empty.
But they put an instance of this Spawner in the World, with the array set to BPs [Shrub, Rocks, Tree] or something like that.
This just load the classes. The code that is being done upon the “AsyncComplete” operation is then in charge to spawn these classes in the world following a distribution
Hey there @pezzott1! Just chiming in that, in my experience you’re correct that Async doesn’t always play nice with loops, so I’m interested if this works with more scale?
Did some digging and found that async tasks are added in LatenActionManager.h. I’m sure this is not the class doing the actual loading but it’s the one handling the tasks (tested by forcing some errors to log). UObject are not limited to one action so I guess that’s why the async works in a for loop. The node is referenced directly so it gets called for every completed action.
I tested adding a soft ref of an actor with a large data table:
The loop executes normally without any delays. The completed pin of the async node then gets called when all actions for this objects are finished. It made no difference if the ‘bulky’ class was first or last, the async node was called in order after all classes were loaded.
These are the times for comparison:
Sequentialy (no for loop): BP_CSW takes longer here for some reason. My guess is that other are queued and competing for resources when done sequentialy rather than pushing all same frame.
For loop w/ original 3 actors.
Added class at second index.
Added class at last index.
All test done after restart and multiple time to be sure timer are consistent. Did not test adding more tasks to see if it extended the duration of the completed executes. Also measured with the computers date and time in case the for loop was messing with the profile node, it was the same give or take a couple of ms.
Classes that are loaded can be accessed by resolving the soft ref even when the async node hasn’t returned anything. The cool thing is that classes seem to be loaded in parallel and not sequentialy as they are returned. When I move the big class to second index the other 3 were able to be resolved before the Completed was ever executed.
Don’t know . I will stay with tried and true methods. Memory managment is a ■■■■■…
Hope I didn’t get this wrong. If anyone has any insight or reads something that needs correcting then they are more than welcome.
I did similar tests yesterday, and reached the conclusion that there is some form of parallelism involved under the hood.
In the cropout sample, I used my for-based macro for async loading, and replaced one of the three models with a blueprint referencing a big nanite model. So I have [Beach_Cliff, Shrub, Stone] as classes to load.
In the following screenshot, at the beginning of the process the second class (the Shrub) is already loaded and the others two (the Beach_Cliff and Stone) are unloaded. The 3 are async loaded following the original order (the big Beach_Cliff first) . But upon completion, the Shrub class have completed first !
Because of the loop, they all start at the same time, in parallel. But you have to do some extra coding if you want to know when they have all finished.
. I will stay with tried and true methods. Memory managment is a ■■■■■…