Adding a facial animation to an existing body mocap animation in UE4 (not in realtime!)

Hi guys,
we are facing a problem right now, which we hope to solve with your help.
We want to add an idle face animation to an existing mocap animation to render a movie via the sequencer in Unreal Engine 4.27.

We are facing the following challenge:

The recorded mocap body animation needs to be cut in the sequencer because it is too short or too fast in some places. As a result, the character’s animation blueprint must be set to “Use Custom Mode”, which deletes the blending between face and body animation (Layered per Bone) and overwrites it with the animation used in the Sequencer

Question:
How do we get both animations (face and body) together and editable and renderable in the sequencer without re-recording the mocap animation?

Detail Problem:

Unreal Engine: UE4.27, Character Creator Tool: Reallusion, Body-Mocap: Motionbuilder 2018, Face mocap: Livelink Iphone to UE.

We used Reallusion’s Character Creater tool to create the character and used the Unreal Engine preset to export it. The character creator also automatically creates pose morphs for the face.

The body animation have been leaned into Motionbuilder 2018 and the character retargeted to the animation. So the rig in the animation already has the correct naming of the character in Unreal. Which is why there is no need to retarget in Unreal anymore. The animation is simply imported with the correct character rig.
The facemotion is recorded directly with the character using an Iphone via the livelink in Unreal. The animation and facemotion are then merged into an Animation Blueprint through the “layered blend per bone” node and the blueprint is dragged into the scene.

If a master sequence is created then and the blueprint is dragged in to play the animation, we have to cut the animation to fit a recorded voiceover (no character speech!). We noticed that we had to set the animation of the blueprint to “Use Custom Mode” so that it would render the correct animation including cuts. When we do this, the character loses its facial animation and only the body animation is rendered.

Is there a way, or do you know of a way, that we can blend the existing two mocap shots together and render them in the sequencer, even though we still have to make cuts in the animation in the sequencer?

A little help would be hugely appreciated! We’ve been looking for a solution for a while now and are a bit stumped.

1 Like