I’ve seen a lot of videos that explain how to use an iPhone ( with a Lidar camera and ARKIT ) to control the facial movements of a Metahuman. (eg. iPhone Facial Capture with Unreal Engine | Unreal Fest Online 2020 - YouTube) The latest Arkit also supports body and hand tracking but I can’t seem to find any software solution that will track everything at once using an iPhone. Does such a thing exist? If not, is there any technical reason why ?
1 Like
Because.
How exactly do you expect 1 camera to know more than 2 dimensions?
Lidar on the camera can and does guess the depth. Take a peak at the iphone videos showing face, and body and hand mocap.
It’s still less accurate / not the norm.
You get away with faces easy because the depth is set at the beginning and kept constant / it doesn’t even matter really.
Body and hand I have never seen on an iPhone.
Vive does it. With more than 2 cameras.
The more cameras you have the more accurate it is.
Before you see people implementing this with iPhone the accuracy has to be on point, like it was for facear. Takes time to get there.
If you want to see Body and Hand on an iPhone, we are currently out with an unreal beta doing body both on an iPhone (search for PoseAI on the marketplace if you want to participate). We do not use ARKit or Lidar however.
Adding face is possibly technically and eventually something we would like to integrate, but it is not a priority for now.
Good to know there is no technical limitation. I’ll check out the beta. Still would love to find all three (face , hands, body) done through a single camera! Between AI and lidar the accuracy would be good enough for many use cases.
I could see that with a lidar scanner, but the best ones that would support a high FPS capture are far more expensive than a set of motion capture suit, or even a proper red based camera setup.
Think something like a GeoSlam scanner.
You may be onto something though.
It’s probably worth trying to use a double lidar this way instead or on top of 2 cameras.
Having the actual distance to a specific point from the camera would incredibly improve accuracy…
Though I have not seen anything like this, I also haven’t looked into cinematic mocap in a long while. They could potentially have added something the like already…