Download

Realtime Animation retargeting from Mocap data

Hi, I am new to the community and to the Unreal Engine, so I hope my question will make sense:)

I am using a MoCap suit from Noitom to drive the animation of a character in realtime. I won’t be recording any animations, realtime performance is what I am after. With a normal human proportion character(Screen_01.png), it works okay, but when I use it on highly stylized characters(Screen_02.png), the Noitom plug-in treats everything as Skeletal animation and does not support Animation Scaled for the hips to adjust leg movement according to the height (leg length) of the stylized character.

So I am wondering if I could use UE’s retargeting mechanism the way you would normally retarget animations, but do it in realtime to retarget animation coming in from MoCap suit onto the skeleton with normal proportions to the stylized skeleton. Basically to do a correction pass on MoCap Data to better fit the stylized character. Both characters share the same skeleton.

I assume I need to create a different animation blueprint for the stylized character and somehow reference the animation of the other character and do some adjustments to it?

I think Ikinema used to do something like that.

Thank you! Ary

You need to make sure that within Axis settings Displacement is not checked ( see below ), as this can cause issues with the character proportions.

I also strongly advise to create a custom Body Size setup, so that you can literally input the limbs length, so that the mocap will fit your character needs.

01.JPG

In general, if you want to do realtime rescaling, you do need to get the data from the suit and rescale it accordingly…but since the suit gives you rotation data mostly, you will either create an IK setup for the entire body, so that you could eventually adjust the hands position as well to avoid clipping, or just get the rotations from Axis itself using the Body Size Manager setup mentioned above.
Good luck :slight_smile:

Hi, thank you for your response. That is actually a quite original solution, so I did measure precisely the bones of my stylized character and added it to the Body Size Manager. Overall it helped to retarget leg animation much better, but it has introduced a new problem - registering foot contact is worse now and rotating hips results in sliding feet on the ground. I’ve tried different sizes of feet and heights for ankle position but nothing seems to be fixing the problem. I guess I won’t be able to fix this without a proper IK setup for legs?

Sliding recorded here:
https://f.io/WwavEhjK

Thank you very much!
Ary

I guess that, even though Axis is doing something to lock the feet on the floor when they touch it, you can create an additional setup that does something similar, meaning that if the ankle ( the value coming from Axis I mean ) is below a specific height, the feet are locked on the ground ( only position, rotation is still free ), and you can then send that “locked” position to the leg IK setup, which will remove the sliding.
Of course, if you’re sliding on purpose, the feet won’t slide, but maybe this can solve the issue with your character :wink: