Hello!
I want to apply audio-driven facial animation synthesis technology to unreal in-game project. And I have never used game engine, UE of Unity…
In my knowledge, most of this ML approach have .obj sequences results, just a series of meshes. It can’t be applied for real-time game.
I found ‘blendshape’ is the key to real-time facial animation. But there is no talking 3D dataset including blendshape weight information. To use blendshape and ML approach both, using ARkit application and recording short samples myself is the best way to get blendshape weights and easily apply to skeleton mesh.
However, it will be worst data to be input of neural network.
So, I want to know, is there the way to get arkit 52 blendshape weights(no need to exactly 52) from normal mesh (just vertices information, npy array)?
really need your help, a little bit of related knowledge. I’m a beginner…