How do I remove or diagnose facial Mocap Jitter?

Community Community & Industry Discussion
, ,
1

Hey all

I have setup the Kite Boy Facial Mocap Demo. My Iphone X does not say its over heating. I am using Unreals Live Link Face But App. My computer is using a 3090 and a Modern Threadripper with 128 GB of ram so the FPS is pegged at 120 the whole time. I am using wifi, but over a fairly over powered Linksys WRT3200ACM that is only 15ft from the computer.

How-ever, this is what the mocap looks like in unreal.

Does anyone know how I might diagnose or remove the stutter stop jitter that is happening?

Any time and help is greatly appreciated

2

I would guess that your WiFi connection is the issue, given that everything else is the looking fine hardware wise.

1 Like
3

My apologies. I should have listed that I have a Linksys WRT3200ACM . Its a fairly powerfull router. BUT just incase I ordered the lightning cable to eithernet adapter to test that.

4

when I’m at my desk I use the lightning ethernet adapter and it helps a bit. The iPhone generally doesn’t send a very stable flow of data. One other issue can be having too low light in your space. The iPhone does use the depth sensor and video camera I believe. So some times if it’s too dark that can effect the frame rate.

1 Like
5

Ahh HA!

You were right! So it was the wifi speed.
First I tried the iphone to network cable and computer directly to cable. That fixed it.
BUT I Luckily also realized my router had a “Media Prioritization” section.

I set it to give all prioritization to my computer and my iphone.

Problem solved and still on wifi :smiley:

thank you so so much for the help :slight_smile:

6

It’s just data being sent… the best way to fix the issue would be to introduce a small delay to allow some pre-buffering on the unreal end.

Sure it wouldn’t be “live” anymore, but it would probably help with stuttering on all takes.

I know the whole concept of the demo is the “live” angle, but that’s not what we do professionally/traditionally for mocap data. In fact, 60% of the work is probably clean up to remove stutter and other issues…

Generally speaking, we record with a decent camera hooked into an helmet, dots on the face, and process the videos with a variety of things - faceware being one of the ones I like best.

I’m sure the same concept can be apply on the faceAR side of things - record first, sit at the desk and process the performance after?

7

Also my apologize. I should have clarified the use case. This is for an actually live streaming facial capture. So I can prolly get a buffer and delay the sound by a second or two as well.
BUT yeah yeah, for the actual cinema render stuff. Totally will do like you suggest and I greatly appreciate the tips :slight_smile:

8

Solution :smiley:

If anyone has same problem.

  • See if you WIFI router has a setting to prioritize the receiving computer and the iphone. If so that should do the trick.
  • Great way to test if its the wifi is to pick up one of those really cheap thunder cable to network adapters for the phone and hardwire both the phone and the computer into the router. ALSO dont add more destination sources to the ipone app. It bug s out and only tracks in unreal for about 5 seconds.
1 Like