SUCCESS - Kinect 2 MoCap -> Brekel Body 2 and 3 -> Import FBX -> Retarget to Unreal Mannequin

I had previously posted my problems using the Brekel Body 2 trial software and a single Kinect 2.

My problem was that Unreal would not let me Import the the Brekel/Kinect FBX as I did not have a skeleton to hang the animation data on - the Unreal documentation says the FBX will import the skeleton included with the FBX data if you leave the Skeleton button at None in the Import Manager. After countless tries and variations, much internet searching a query here I was helped by Mssr Brekelman himself, and I was able to get the Brekel_Example file that includes the Brekel/Kinect skeleton. With that I was able to get the FBX to import.

The second challenge was to modify the Target skeleton - in my case a character rigged to the stock Unreal Mannequin which defaults as A-pose - and modify it to T-pose to match the T-pose of my incoming FBX with Brekel/Kinect Source skeleton. This could be easy but Unreal is vastly complex so nothing is ever easy for new users.

One solution is to pull my A-pose Mannequin-based character skeleton into the Skeleton Manager where I can Modify - raise the arms to T-pose - that’s pretty straightforward. But SAVING the modified skeleton and identifying the T-pose file, that’s tricky. (The Unreal documentation shows a Save button next to View, but it’s gone in 4.25.)

You have to hit the Modify button to change the Mannequin, then after changing hit the Modify button again - on the submenu choose CURRENT POSE, then Save with the Tool Bar Save button upper left. Now you have Saved a T-pose asset for retargeting.

You also have to build a Rig in the Retargeting Manager for both your Source Mocap skeleton, in my case Brekel, and for the Mannequin skeleton (this is largely automatic since the Mannequin is an Unreal asset to start with). The Brekel/Kinect skeleton is basically similar to the Mannequin, though I chose None for Fingers and Toes. Both use the Humanoid rig.

Finally, you call up your FBX animation file inside your Unreal Project and right click the FBX file in your Content Window - find on the popup menu Retarget Animation Assets - sub menu Duplicate Anim Assets and Retarget. This brings up the Skeleton Select pop up menu. Your source pane will be the skeleton of the FBX file. Identify the Target skeleton on the left side menu of this screen - your Rigged T-pose skeleton likely has an asterisk(*) icon next to the name. Click that skeleton and it should appear in the Target pane, remembering that it must be in the same A-pose or T-pose as the Source FBX skeleton to its left. Fix, if not.

Add the Prefix to the file name so you can identify the Retargeted animation, hit Retarget button lower left and you are in business.

I learned a lot from this very basic video:

Next experiment - using two Kinect 2 sensors inside Brekel Body 3 to see what improvements in motion jitter, foot tracking, and especially in occlusion - turning a full circle during motion capture and getting a clean(ish) file. Not so clean with single sensor, so far.

Also, finding out how Brekel/Kinect 2 handles facial states - from the 5-point simple read to the full face. My animated film does not have human faces, to avoid the uncanny valley…

Great info. I am trying the same with the azure Kinect. I don’t suppose you’d be willing to share a video or some screenshots with the setups? Thanks

not quite ready to do a video yet - but maybe I can help with a specific problem?

Are you using Brekel or just the Kinect SDK from Microsoft?

Have you been able to import at all?

Once you get the basics set up it is impossibly easy to record mocap and get an Unreal character moving. The goal is in sight…

.

I played around a bit more and have it working now and via live link. Getting a bit of a floating ghost (tip toe) like outcome tho, any thoughts on how to make the feet a bit better? https://www.dropbox.com/s/ub421912ua…36-25.mp4?dl=0

Hi - your animation mocap looks pretty clean, not a lot of jitter. Are you in Brekel Body 2 or is it 3? I have not worked on the foot floating yet, but I expect to firm that up in the original mocap in Brekel, rather than in Unreal.

Did you have trouble getting the ports sorted out for Live Link? I have my Brekel and Unreal working on the same machine and the ports arent talking yet. I have a feeling it is a lot more efficient to record the Mocap inside Unreal, rather than export fbx from Brekel and import/retarget constantly in Unreal.

Are you working on game animations or film/tv?

Cheers, I am working towards film/tv stuff but still learning the ropes. I tested ipistudio at the weekend and that actually worked really well too. But u have to do a T poss at the start of each record and it’s a bit more involved before bringing it in but feet seemed to work very well straight away.

I was using body 3 trial beta that Jasper sent to test…The live link worked for me but I think I needed to open the port, it only shows up in unreal once a bodies being tracked though beware. It does seem to work a bit better than an fbx import yeah and is a better workflow to re-record takes and playback. . Going to do some more digging into these feet though. Cheers

[USER=“4079472”]Tim Main[/USER] interesting. Was the kinect at the same position when you tried with both software? I am assuming that you have tried both Brekel & ipistudio .

Great tutorial. But what about real-time mocap data coming from Brekel to UE4 mannequin or any .FBX t-pose character from Mixamo? I find animation retargeting option when it’s pre-recorded animations, but you don’t see that option when it’s real time. Do you know how to do that?

Im finding it practically impossible to retarget from the brekel skeleton to the unreal, the whole thing keeps coming out all mangled, does anyone have a screenshot of the retarget settings?

Hi, I have the same problem to import the FBX files. Could share the Brekel Skeleton with me? Thanks