Advice Needed – Using MetaHumans with Audio2Face for Mobile VR

Development Character & Animation
, , , ,
1

Hi everyone,

We’re currently conducting R&D for a mobile VR experience on Unreal Engine and would love to get some insights and guidance from the community.

We’re evaluating two main solutions for character creation: MetaHuman and Character Creator. A big advantage of both is their compatibility with NVIDIA’s Audio2Face, which allows for automatic lip-sync animation based on audio input.

However, neither solution seems perfectly suited for deployment on Mobile VR, specifically Meta Quest 3. So, we have a few questions for those who might have gone down this path:

  1. Has anyone successfully used MetaHumans with Audio2Face in a mobile VR build?
  2. Does the Audio2Face plugin work reliably with lower LODs (LOD3, LOD4, or LOD5)?
  3. On average, how many MetaHumans can be realistically loaded in a mobile VR scene on the Quest 3, assuming a moderately optimized environment?

As for Character Creator, we’ve noticed that only the highest LOD (around 17K polygons) supports blendshapes for lip sync, making it less ideal for performance. While optimizing those characters would be a significant effort, we do appreciate that Reallusion offers lighter rigs and more optimized hair options.

We understand that MetaHumans were originally designed for AAA games and high-end cinematics, but if there’s a way to adapt them for mobile VR, it would be a huge win for our project.

Any tips, experiences, or best practices would be greatly appreciated!

Thanks in advance,