Help with Multi-Threading a RuntimeMeshComponent update.

Development Programming & Scripting C++
1

I have a RuntimeMeshComponent that is procedurally generated. Currently this is a component on an actor. This actor class also houses the containers associated with the vertex positions the RMC is using. I have a TArray<FVector> for the vertex locations, TArray<int32> for the face assignments, and a TArray<myGameData> that holds additional vertex information (material types, offsets, etc…). Now recently I’ve built a menu system to allow me to visually edit the procedural mesh generation parameters and see the update occur. But, this causes a small hitch when “generate” is pressed as it runs through updating the RMC vertex positions, and their normals. So Ideally I would move this process to its own thread so that it doesn’t cause this hitch. I’ve run through Rama’s thread tutorial and I’ve seen how computing 50000 prime numbers while this is all going on and still having 90 FPS is great so I figure I could move the RMC vertex position update (and normal calculation) to another thread to achieve this. I just finished reading through this but I must admit its a little above my head. Still I’d like to push forward and get my feet wet. So I have a couple questions about multi-threading and the construction of all related.

There’s the render thread, and the game thread right? When I completed the Prime number tutorial I was adding that process to an entirely new thread, outside the game thread and the render thread? If my data is contained inside an actor class will I be able to modify the data in my containers within that class inside a new thread (the one I’m trying to create to do this heavy lifting)? If not how would I modify the vertex positions on a RuntimeMeshComponent->BeginMeshSectionUpdate() reference? After all this RMC is currently a component attached to an Actor class.

2

So I’ve set it up so that my Character class calls a BeginProceduralMeshUpdate() which fetches two references, one to the procedural mesh vertex array, and one to the procedural mesh vertex data (normals and such). Then I call my function to create a thread, and process the vertices into their new location, and new normals. This takes place one the new thread. In order for me to see this work completed, I need to call EndProceduralMeshUpdate() from where I called Begin. But how do I know the thread I created finished the work load? Do I have to periodically check inside it? Maybe twice a second? Wouldn’t it be cleaner if the work thread notified the game thread it was finished by calling an end function in the game thread from the work thread? Then I wouldn’t need to check on it periodically. Is it standard practice to just monitor the work thread until it completes? Or is there a more efficient way to have the work thread notify the game thread?

3

So I found a great tutorial on delegates and figured it might be a good way to notify the character class that the work had completed. So on my Work Thread, when it completes the work it calls the delegate function on my character class, which calls the EndMeshSectionUpdate(). I really thought this would work. I can see that it works in terms of the work thread notifying the game thread that it completed calculating the new vertex data. I have it print to log when the EndMeshSectionUpdate() function is executed (from within the delegate function). So I know it’s getting there at least. When it crashes it opens up the IConsoleManager.h line 789

// faster than GetValueOnAnyThread()
T GetValueOnGameThread() const
{
// compiled out in shipping for performance (we can change in development later), if this get triggered you need to call GetValueOnRenderThread() or GetValueOnAnyThread(), the last one is a bit slower
checkCode(ensure(GetShadowIndex() == 0)); // ensure to not block content creators, #if to optimize in shipping
return ShadowedValue[0];
}

So I’m lost again. I feel like this is part of the challenge with Multi-threading and I’m not getting it, so any insight is welcome. thanks!

4

@Thumper You’ve started down a path that I’ve been debating how to handle for a while now. I see it appears you’ve figured out basic threading within UE4. So I’ll explain the problem you’ll have specific to the RMC (and this applies to the PMC as well to my knowledge).

First up, the RMC does not currently support what you’re trying to do. It assumes all interaction with it is from the game thread, and this is primarily due to the engine as a whole not really supporting/wanting you to touch a UObject from another thread. I’m not an expert in UE4, but this is how I understand the reasoning behind not touching a UObject from custom threads. UE4 runs a custom precise garbage collection system to handle UObjects, and for this to run it has to be able to stop things from accessing the memory it’s tasked with managing. For UE4 this is assumed to just require stopping the game thread, and then it can go do it’s work and then resume the game thread. So if you have another thread that’s accessing a UObject, the garbage collector (GC) doesn’t know anything about it and can potentially delete the object while your other thread is using it.

Next, I’m not sure if you’re using collision, but there’s currently no way to offload that from the game thread so you’ll still get a significant hit from that with large meshes.

The above reason is why the RMC, and PMC, don’t support this type of interaction natively. I’ve been thinking about ways to support it while being safe and playing nicely with the UE4 GC. I’d be interested to know more about your needs, but I have no guarantee I’ll ever really try to add support and even if I do when it will be.

5

Ah, this is a bummer, but thanks for the information.

Currently I’ve built a sphere like planet that generates procedurally (Using the UnrealFastNoise Plugin). I’ve also built a photoshop layer blend system and exposed it all to a UMG menu. This allows the user to create noise modules with specific settings, create composite layers that allows the blending of modules (add, subtract, multiply, etc…) so that you can very specifically modify the noise and create a composite of many noise modules blended together. It works great. But when you click any “generate” or “update” button it hitches for about 2 seconds (more with each additional layer added to a composite hierarchy). My intentions were to dump this process to a background thread. My RMC mesh has subdivision levels to it, so the background thread would solve maybe 2 subdivision level below the final output level, bring that up to render, solve the next subdivision higher (the remaining verts, because the first vertices were solved in the previous subdivision layer), bring that one up to render, hide the previous, and finally solve the final subdivision level (the remaining vertices in the master vertex list). There are additional advantages to solving the previous subdivisions because you get larger lateral information (because the vertices are further spaced, so knowing your neighbors, means you know further away) and so you can do work on the horizontal aspect (like prescribe certain color information for the materials).

I could split the world into sections, and do the sections one by one. I was going to do that anyhow, I’ve been putting it off because I want to be sure about how I want to split it. Maybe it’s time. But honestly I’m not done on the multi-threading. I’m very curious about it. So if you don’t mind, I have a couple questions about it. Is it possible at all to use the RMC? It’s not that my game is crashing due to a memory error. It’s stopping due to an assert of some sort. For instance, last night I decided I would use the command

ENQUEUE_UNIQUE_RENDER_COMMAND_FOURPARAMETER(
AKoreChar,
AIcosahedron*, icoSahedron, icoSahedron,
int32, index, 0,
ERuntimeMeshBuffer, grouped, grouped,
FBox, BoundingBox, BoundingBox,
{
icoSahedron->rtm_Base->EndMeshSectionUpdate(index, grouped, BoundingBox);
});

to call the EndMeshSectionUpdate() which is the last command I give before the game stops. I thought that ENQUEUE command sends the code to the render thread and tells it that when it gets to it, run it. So I would imagine that is safe, or at least safer. But I get the same game stop as above and it brings up the code GetValueOnGameThread(). So, let me ask this question in two ways, one, if I were going to hack this (force the engine) to allow it where would I start? And Two, if I were to modify the RMC what should I look into as you see it as the core problems?

6

@Thumper I’ve actually done some similar things as my main project, and reason for initially building the RMC, is a voxel engine which for right now is dedicated to terrain. I use alternate threads to generate everything, then use a MPSC (Multiple Provider - Single Consumer) mode on TQueue to pass the result back to the game thread. This is how I’ve personally gotten around the threading issue. I still get hit by collision cooking but I’m slowly working on fixing much of that.

Now specifically for your above command…

EndMeshSection (and well any of the RMC’s public functions) expect to be called only from the game thread. What you’re doing there will try to call it from the render thread. The reason for needing to be on the game thread has a few parts to it, first it updates bounding information for culling, then it also updates collision if needed, then also for certain section types will ask the engine to recreate the scene proxy. Then from there it will either let the proxy recreation take care of moving the mesh to the render thread, or it will use that same type command you have above to send updates over to the render thread.

7

This has been very informative. For a while I’ve wanted to bypass the hitches generated by the substance plugin when you refresh a substance’s parameters. I’ve thought for a while about doing that on another thread but wasn’t sure about what data access problems I’d have, getting data to and from the game thread.

8

I use the same approach as Koderz for multi-threading using the RMC.

I have two queues, one for generation tasks, one for rendering. The game thread adds a new task to the generation queue, which spawns a thread and calculated all the geometry, and then when it’s finished adds a task to the render queue. I check the render queue each tick and if there’s a task in there, it calls UpdateMeshSection.

9

Sorry for delayed response, I’ve been busy splitting my RMC into several RMCs. It definitely helps doing updates on smaller lists. So I’m trying to make sure I follow you mid_gen. You have two additional threads you use. One for calculating, and processing the RMC data, and a second render thread? Do you mean a render thread on top of the render thread that already exists? Are you using the MPSC mode for Tqueue as well? When you add a task to the render queue what task is that? When you call UpdateMeshSection, you’re calling it from the game thread right? and doesn’t that add a task to the render queue?

10

I only have one additional thread. I have two queues :

Processing Queue
Rendering Queue

When I need to update a section of mesh, I add a job to the processing queue. I check this queue on tick (game thread) and if there’s a job, I spawn a thread to process the data.

When the thread finishes, it puts a job into the render queue.

I check the render queue on tick (game thread), and if there’s a job, I call UpdateMeshSection with the data set.

11

Sounds like you’re getting mixed up between threads and queues. A queue is just a data structure, one that is often shared between multiple threads. From what I understand, I don’t think you need to be doing anything from UE4’s render thread in this case.

Essentially, you just have to decouple your worker thread from your game thread objects, since UObjects are not threadsafe. So in this case, presumably what is taking a noticeable amount of time is the generation/subdivision stuff. So your worker thread needs to be told ‘Generate data for this region at this subdivision level’, and it goes and does the processing, storing the data in it’s own structure as opposed to directly onto the RMC or any other game thread object. Once it’s done, it hands that data back over to the game thread, which just has to update the RMC with it. That will likely involve just some mem copies or possibly even just handing off of pointers. Either way, it will be minimal so won’t hitch the game thread.

To hand the data back, you can use a queue as mid_gen says. Your queue element may be nothing more than a pointer to the generated data. The game thread can check the queue in tick (there are other ways but this is the simplest), and if there is something there, it applies the new data to the RMC.

12

Great explanation, thanks! Let me see if I have this straight. The scenario is I’m updating a RMC, modifying the vertex locations from where they currently are to where I’m telling them to be now. BeginProceduralMeshUpdate hands a reference to my worker thread which I’m creating for the purpose of calculating the new positions. Worker thread does those calculations, but instead of modifying the vertices at the reference I store the modified vertices in a new array. Then I add a pointer to this array of modified vertices into a TQueue. Then I call a delegate function back on the game thread (can I do that?) to notify that it has completed processing the vertices. This delegate function calls EndMeshSectionUpdate. This is where I’m a little lost. EndMeshSectionUpdate doesn’t take any arrays. So should I process through the modified verts and copy them over the BeginProceduralMeshUpdate vertices reference (seems redundant) or can I just swap out the root pointer for my new modified vertices list pointer?

Originally I had it setup so that my worker thread just modified the vertices reference at their location which I’m guessing was the problem because the RMC is a UObject and a separate thread cannot do that?

13

You shouldn’t pass a UObject reference to your worker thread at all really, you should copy the vertices (whilst still on the game thread) into some non-UObject structure and pass that to the worker thread. Whatever you later add to the queue when done has to remain alive of it’s own accord, so something like a shared/unique pointer to a struct or vertex array would usually make sense.
If you execute a delegate from the worker thread once it’s done, that will still be executing on the worker thread so it’s not much use. You could use the task graph system or custom synchronization to achieve this, but it’s really not necessary - just checking to see if there’s anything in the queue each tick on the game thread is fine and much simpler.

This is a high level outline of how to do something like this. I haven’t used the RMC and have no idea about these particular functions, so I can’t help you with those details I’m afraid.

14

@mid_gen, @Thumper I have something the two of you might be interested in…

I’ve had this idea for a while about supporting threading like this directly for the RMC. Well, I’m about to… It will still be up to you to manage the threads and how you want to get the threads to do work, but the part of then passing the data back to the RMC is pretty much solved now. This isn’t in GitHub yet, but likely will be late tonight.

9d98d595ae.png
9d98d595ae.png914×49 7.09 KB

Basically, when you tell your thread to do something, you pass it a TWeakObjPtr to the RMC it needs to call back to when it’s done, then you can use the static helper FRuntimeMeshAsync to do most of the heavy operations like create/update mesh, create/update collision meshes. Some things like batched updates, and in-place updates won’t be supported through this due to complexity working around the GC.

15

Does anyhone have a example on how to use threads and make a progress bar on generation? Thanks