I’m an absolute beginner using UE 5.4 out of necessity for a specific case for work, so bear with me.
I need to animate a character’s body and face in real time with rokoko studio. The character is a traditional white cloth ghost, so it has a lot of cloth simulation going on, including eye and mouth holes that I need to stick to and follow the facial expressions. So essentially, I have a pre-draped cloth mesh with the eye and mouth holes, and the rigged underlying body shape mesh with the facial blendshapes that the cloth mesh rests on top of and sticks to.
This is what I’ve done so far:
-I have imported the custom character’s skeletal mesh, complete with ARKit facial blendshapes;
-I have imported the cloth object’s static mesh, used it to create a Chaos Cloth Asset and setup it up to “stick” to key parts of the character (the eyes, mouth and sleeves) by using the transferSkinWeights and addWeightMap nodes that come by default with the graph and manually painting in the parts of the mesh that follow/stick to the associated skeletal mesh.
-Also in the cloth asset graph, I took the character’s skeletal mesh and created a static mesh from it, imported it into the graph, used a transferSkinWeight to transfer the skinweights from the skeletal mesh to the static mesh, merged it with the previous default collision and simulationconfig nodes and connected it to a selfCollisionConfig node, so that the cloth asset collides with all parts of the underlying rigged character mesh. (I used this tutorial for this step)
-I have then created an animation blueprint for the skeletal mesh, for the rokoko integration with rokoko bodypose and facepose nodes to be driven by the connection to rokoko studio in realtime, and it’s working. The body and face are animating correctly in real time.
Here’s the issue though: If I drag the character into the scene with the rokoko animation blueprint, add the cloth object, and hit simulate, it all works (the cloth collides with the skeletal mesh, the eye and mouth holes, and the sleeves stick to the mesh, but the face mocap blendshapes seem to be ignored by both the collisions and the sticking process. I can see the underlying skeletal mesh moving its mouth and eyes through the holes, and I can see for example the eyebrows clip through the cloth when they move up. It’s as though, as far as the cloth object’s nodes are concerned, the character has no blendshapes and is always in a neutral face pose.
Is there something I’m missing? Is there a way to make the chaos cloth process interact with the rokoko-driven blendshapes?