MetaHuman - Blending ARKit Facial Motion Head Movement with Body Animation

Is there a way to retain head rotation from a recorded ARkit facial motion capture whilst keeping the rest of the body animation on a MetaHuman.

To explain better - I have an animated MetaHuman in sequencer with the Face and Body ControlRigs removed with the base “walking” animation added to the body component.
I also have a pre-recorded facial animation from ARkit (livelink) of this Metahuman whistling and looking around…

Now when I add the facial animation the the “Face” component in sequencer, the facial expressions work just fine, I just lose the head rotation from the ARKit capture, the level sequence just respects the head roation from the walk animation.

Is there any way to have the facial ARKit animation either blend or replace the head rotation from the body animation. Without being able to do this I do not see the use of recording the head movement in ARKit so there must be a way, I am just missing it.

If I need to edit the Face_AnimBP or edit other parts I am not adverse to doing that, just there doesn’t seem to be any documentation on how to do this (in either 4.27 or UE5) and Youtube, despite heaps of great videos, never seem to show “blending” the two animations…

Thanks in advance for any advice!

PS. This is in UE5

Haven’t tested that yet, but the Face_AnimBP has a setup where is getting the body animation from the main skeleton, and then it applied the facial animation blending the wo together.
It uses a Layered blend per bone node, which is using 3 joints, as you see in the image below

I think that, in order for the neck animation to work from the data from the iPhone, you need to use the neck_01 joint, instead of the ones you see in the Layered node.
By doing so, you get the animation from the iPhone from the neck_01 up, while the rest of the body is driven byt the body animation.

4 Likes

@Enter_Reality - You are a great human being and legend!

That immediately solved it, I had even poked around in the face animbp thinking it had to be in there but couldn’t find any references to head or neck, and there they were tucked away under Layer Setup (collpased)

I really appreciate you taking the time to respond :slight_smile:

1 Like

Hi There! I wanted to ask you if you could post a picture of your node setting as I am the same you were but cannot find the way to apply it the way you did.

Just a picture of the final node setting would help a lot, thanks!

Thanks a lot. You really saved my life. Searched for months and tried many ways and this is best solution.