Adding and removing physics bodies in control rig dynamically

[anonymous-edc (Anonymous User)]

This question was created in reference to: [Adding physics to Modular characters by “Leader Pose Component” [Content removed]

Hello, as recommended in my previous question, I am experimenting with control rig physics to implement a character with physicallized parts of clothing. In short, the goal is to use Leader Pose Component method to animate the clothing pieces based on the character mesh and then control rig (CR) to simulate their physicalized parts. I am using “Instantiate From Physics Asset” (IFPA) to construct the physics bodies from the physics assets ¶ - authored by artists - of the clothing and then simulate them in the rig. It seems promising so far, but there are issues with dynamically rebuilding the CR.

1) Can I set control rig variables used in Construction Event dynamically when the rig is part of animBP?

We want to set which PA to use dynamically (based on what clothes the character has on) -> I need to “parametrize” the Construction Event by the right PA to pass it on to the IFPA. Is it possible to somehow do this when CR is used as a node in animBP? Perhaps during the InitializeAnimation event of the animinstance? Anyway, even if possible, my next problem is that as the clothes can change during gameplay, I need to be able to re-initialize the rig. And that is not an option for animBP CR node, right? I would have to re-initialize the whole animinstance I suppose (which even if possible is not a great solution)?

2) How to use CR as a component for animation?

The next option is to have the CR as a component of the character actor. This allows me to use “Initialize” node on the CR component to re-trigger the construction and using the OnPreInitialize event + SetVariable on the CR to pass in the PA I want to the rig to use it in Construction Event. This works, but I have not found out how to setup the component to act basically the same way as when the CR is the last node in the anim graph of my skeletal mesh component (SMC). I can use the “Add Mapped Skeletal Mesh” node to set the CR as either the input or output of the SMC, but I would sort of like both: have the SMC process its animBP, send result to the CR and then read back from the CR and update bone positions accordingly. ChatGPT advised to use two SMCs, set one to be invisible, do the animation on it, map it as output to the CR and then have the second SMC be visible and mapped as input for CR and use that for rendering. Surprisingly, it sort of works, but feels very wrong and I can see it being a source of all kinds of issues, not to mention memory overhead. Is there some more reasonable way to achieve this “two way” mapping of the CR component?

3) How to reconstruct/remove only chosen physics bodies?

What I would actually like to do when clothes change is to only remove physics bodies that are no longer used and create new ones, but keep the solver and bodies that don’t need to be changed as they are (both for performance and also for visuals - e.g. so that physical motion of my coat is not reset because I put a new hat on my head). For that I would like two things:

1. the ability to say that my CR custom user event can call construction-only nodes (like IFPA) - I assume there would have to be some restrictions on when such event can be triggered
2. a node for despawn of physics body

Looking at the IFPA’s implementation in FRigUnit_HierarchyInstantiateFromPhysicsAsset_Execute(), I see new components can be added and removed by URigHierarchyController’s Add/Remove component, but what are the pitfalls of using this after construction? Likely it needs to not be done during animation update, are there other restrictions? I am ok with not having any collisions between assets from different PAs.

Note: I can think of an alternative approach using one CR for headwear, one for torso etc. This way I always have only 1 or 0 physics bodies per CR, so there is little to gain by doing this per-parts re-initializion and I could just reset it whole. But that means several rigs and several physics solvers instead of just one, would have to measure if it brings a significant performance overhead. And still would need question #2 resolved for that to work.

[Euan_Carmichael]

Hi, based on what you said here:

> In short, the goal is to use Leader Pose Component method to animate the clothing pieces based on the character mesh and then control rig (CR) to simulate their physicalized parts.

It sounds like you’ve effectively got an ‘uber’ skeleton setup for the lead component where the bones of that skeleton contain all the different possible attachments, etc, and only certain bones are then skinned in the follower meshes depending on the current configuration, is that correct?

Assuming it is, I think I would look at using a data asset to specify the physics asset within the control rig construction event. Essentially, you would create a data asset type that just has an array of physics assets. Then, create an asset of that type that contains the different physics assets that you may want to instantiate from in your control rig. You can then specify the data asset via user data on your mesh asset - it would be a Data Asset Link entry within the Asset User Data property. You can then use the Get User Data node in control rig to retrieve the physics asset and pass that to the IFPA node. All you would then need is a pin on the control rig that changes the path on the Get User Data node to get the correct physics asset for your current character.

The other part of this is that it requires that you call the construction event on your control rig each time that you change character configuration. Re-calling the construction event after the rig has been initialized is possible, and it’s something that we already do in certain situations (like a LOD change). You would trigger the construction event to run by calling UControlRig::RequestInit. Currently re-running the construction event is a hard limitation within control rig as you can’t modify the rig hierarchy outside of that event. So you can’t add or remove bodies in other ways, although that is something we potentially want to support in future.

Having said all of this, I did try and test the user data setup quickly and ran into what seems to be a bug with accessing certain types from the Get User Data node (it failed to retrieve the physics asset type) so if this approach is of interest, let me know and I can talk to the dev team about why this isn’t working.

Another option here is to have a single physics asset that contains all the bodies for the different character configurations. You would create the CRP setup using the IFPA node just as you are at the moment. But then, for each attachment/body that you don’t want to be active in the current configuration, make those bodies kinematic, set them not to collide and disable all joints and controls on them. You can do this via the forward solve at the moment so you wouldn’t need to re-run the construction event. But there will be some overhead for having the extra bodies that aren’t active.

I’ll go through the specific questions that you had.

> Can I set control rig variables used in Construction Event dynamically when the rig is part of animBP?

I think the data asset approach mentioned above is likely the best option to support the approach that you mentioned here. You don’t need to reinitialize the anim bp, just re-run the construction event if you go with this.

> How to use CR as a component for animation?

I would strongly advise against using control rig component for gameplay animation. It was designed for offline usage in VFX and other industries rather than games at runtime. The update of the control rig happens directly on the game thread so it’s particuarly inefficient compared to using the control rig animation node.

> How to reconstruct/remove only chosen physics bodies? I see new components can be added and removed by URigHierarchyController’s Add/Remove component, but what are the pitfalls of using this after construction?

This just isn’t supported at the moment for the reasons that I mentioned. Although it may be supported in future. The equivalent would be disabling the bodies that you don’t want in the current character configuration, as I mentioned.

> Note: I can think of an alternative approach using one CR for headwear, one for torso etc. This way I always have only 1 or 0 physics bodies per CR, so there is little to gain by doing this per-parts re-initializion and I could just reset it whole. But that means several rigs and several physics solvers instead of just one, would have to measure if it brings a significant performance overhead. And still would need question #2 resolved for that to work.

This is the more standard approach to solving the problem in the engine at the moment. And it doesn’t just apply to your control rig setup; it would also apply to your skeleton. The usual approach is to have an ‘invisible’ base skeleton that has a simplified set of core locomotion bones. Then, have more complex visible mesh(es) that have more complex hierarchies with all of the cosmetic bones, etc. You could then, in theory, have different control rigs for the different mesh parts. This is similar to the setup that we went for in the witcher demo. But you then run into performance problems when running multiple control rigs and multiple skeletal meshes. So, at that point, you would likely end up going back to thinking about mesh merging to reduce complexity (back to the previous question that you raised). Unfortunately, the engine doesn’t yet have a magic bullet solution for the problem of composable characters made from different mesh parts that can be switched, although the hope is that Mutable will make this easier.

[Euan_Carmichael]

No problem, I’ll close out this thread.

[anonymous-edc (Anonymous User)]

> It sounds like you’ve effectively got an ‘uber’ skeleton setup for the lead component where the bones of that skeleton contain all the different possible attachments, etc, and only certain bones are then skinned in the follower meshes depending on the current configuration, is that correct?

Yes, exactly

Using the data asset + re-triggering construction event is something I want to try, thanks for the suggestion. However, I got stuck right from at the beginning: you’re saying to use UControlRig::RequestInit and that I should stick to using CR inside an animgraph rather then component. But I don’t see a way to expose RequestInit in such a way that I could call it on the instance of the CR that the animgraph uses. I could maybe create a RigUnit node that I call from inside the evaluation of the control rig in response to some bool variable set from the outside and I imagine there is a way to get the ControlRig instance from its ExecutionContext and call RequestInit on it. But even if I make this work, it means I will be at least 1 frame delayed before the actual re-init happens (since I will be requesting it from within the forward solve step, so I guess it will only be done afterwards). Do you see some better way to do it?

[Euan_Carmichael]

I think you’ve got a couple of options here. The first one I would look at is modifying FAnimNode_ControlRig::Update_AnyThread so you can queue a request to rerun the construction event at that point. You would also need a new property on the node, that you could expose as a pin, to trigger this. Similarly, another option is to create a derived version of the node, if you want to keep the engine implementation clean.

The second approach would be to use UControlRig::FindControlRigs to find control rigs that are active on the current mesh/actor. The issue with this approach is that it uses a TObjectIterator, which is slow, so I would be wary about doing this regularly. But it would avoid needing to go via the anim instance.

In either case, looking at the code again, you can probably use UControlRig::RequestConstruction rather than UControlRig::RequestInit as I don’t think you’ll need to reinitialize the VM.

If you go with either of the approaches above, the construction event should be requested by the point that FAnimNode_ControlRig::Evaluate_AnyThread is run, so it should be processed then.

[Euan_Carmichael]

Quick follow-up on this. I’ve tracked down the cause of the issue with the GetUserData rig unit not retrieving physics assets from data assets linked to meshes. The following code in FRigDispatch_GetUserData::Execute isn’t correct:

const FString CPPType = RigVMTypeUtils::GetCPPTypeFromProperty(PropertyResult);
const FRigVMRegistry& Registry = FRigVMRegistry::Get();
const TRigVMTypeIndex ResultTypeIndex = Registry.GetTypeIndexFromCPPType(CPPType);
const TRigVMTypeIndex UserDataTypeIndex = Registry.GetTypeIndexFromCPPType(UserData->GetCPPType().ToString());

The rigvm registry doesn’t allow raw uobject type ptrs to be added, these are always sanitised to the object ptr equivalent. So when “PhysicsAsset” is added to the registry, it will actually be added as “TObjectPtr<UPhysicsAsset>”. But that code in the rig unit isn’t sanitising the strings in the same way. So instead, that code should be:

const FString CPPType = RigVMTypeUtils::GetCPPTypeFromProperty(PropertyResult);
const FRigVMRegistry& Registry = FRigVMRegistry::Get();
const TRigVMTypeIndex ResultTypeIndex = Registry.GetTypeIndexFromCPPType(CPPType);
 
// Object type will be null for non-uobjects. We deal with this in PostProcessCPPType
UObject* CPPObjectType = RigVMTypeUtils::GetCPPTypeObjectFromProperty(UserData->GetProperty());
const FString UserDataCPPType = RigVMTypeUtils::PostProcessCPPType(UserData->GetCPPType().ToString(), CPPObjectType);
const TRigVMTypeIndex UserDataTypeIndex = Registry.GetTypeIndexFromCPPType(UserDataCPPType);

With that, you should be able to retrieve the physics assets from the data asset if you decide to go down that route.

[anonymous-edc (Anonymous User)]

Thank you. I am trying it out, but struggling, I have never worked with asset user data. Could you please give me more step-by-step instructions for dummies? I was unable to find any useful documentation on the topic

> Essentially, you would create a data asset type that just has an array of physics assets. Then, create an asset of that type that contains the different physics assets that you may want to instantiate from in your control rig. You can then specify the data asset via user data on your mesh asset - it would be a Data Asset Link entry within the Asset User Data property.

I think I follow this part - create a new class derived from UDataAsset; create an asset of it and fill it with data; in the skeletal mesh component of my actor’s blueprint add new entry to the “Asset user data” array of type Data Asset Link. Choose the asset I created as the “Data Asset” and set some namespace (I assume that’s some arbitrary string of my choice that I will then use in Get User Data).

> You can then use the Get User Data node in control rig to retrieve the physics asset and pass that to the IFPA node. All you would then need is a pin on the control rig that changes the path on the Get User Data node to get the correct physics asset for your current character.

This is where I get lost: where do I get the “path” for Get User Data? Trying to understand it from the implementation, it looks like it’s getting the data from some map on the control rig instance, so I also need to somehow pass the data asset from the skeletal mesh to the control rig and only then I can get it using the GetUserData? So is there some ControlRig->SetUserData(myMesh->GetUserData(“mypath”, “myNamespace”)) I should be calling somewhere?

[Euan_Carmichael]

No problem, the user data workflow isn’t obvious, and I don’t think we’ve documented it anywhere yet, unfortunately.

It sounds like you have a lot of this working already but I’ll go through all of the steps just in case. First, you need to create the asset data type and the data asset itself. That’s documented here.

Once you have the data asset, you need to add it to the Asset User Data on the mesh via a Data Asset Link. (I didn’t modify the namespace from the autogenerated one when doing this.) You need to do that for the mesh that the control rig was created against or from which it was imported, and - I assume - any other meshes that you might want to use with this rig.

Once you do that, the Get User Data node should pick up on the user data that’s been added to the mesh. The node should populate the namespace and path for you. You shouldn’t need to enter any of this yourself.

Then it’s just a case of connecting it up and with the previous changes, the node should find the correct physics asset.

If the paths aren’t auto populating for you, I would double check that the data asset has been added to the correct meshes. If the preview mesh is different to the mesh the control rig was created against, you may run into issues. So the other thing you could check is just quickly create a new rig from scratch on your mesh to see if that works.

The other thing, when I was testing, is that I created my data asset as a primary data asset just for convenience. So possibly, if it’s a regular data asset type, that type hasn’t been loaded for some reason so it can’t be found by control rig. If you think something like that’s going on let me know and I can investigate.

[anonymous-edc (Anonymous User)]

Thank you, I was assigning the data asset to the skeletal mesh component that uses the mesh rather than the mesh itself, the way you showed works.

I think this is all I needed regarding this question, thank you for your help.