MetaHumanAudioToFaceRuntime
🎙️ Speech in → 🧑 MetaHuman talking. In real time. Zero baking.
Turn any audio into fully animated MetaHuman facial performance at runtime. An NNE / ONNX inference solver converts speech directly into viseme curves and drives your MetaHuman through LiveLink — no offline processing, no pre-baked animation, no waiting.
Feed it a WAV file or a live PCM stream, and your character lip-syncs instantly. Perfect for realtime voice agents, TTS-driven NPCs, and interactive digital humans.
✨ Key Features
🔊 Runtime Audio-to-Face
Speech → viseme curves, generated and played live. Nothing baked, nothing precomputed.
📡 Dual Input — Files or Live Stream
Play from WAV, or push streaming PCM with BeginAudioStream / FeedAudioStream / EndAudioStream — built for realtime voice & TTS pipelines.
🌿 Procedural Liveliness
Automatic blinks, gaze drift, and subtle head micro-motion keep your character alive in both idle and talking states.
😊 Emotion Layer
Rule-based emotion + optional ML emotion recognition (NNERuntimeORT), with full manual override when you want precise control.
🎚️ Tuning Presets
One-click looks — Realistic Dialogue, Cinematic Close-Up, Expressive Stylized, Live Stream Presenter — or go fully custom.
🧩 100% Blueprint-Driven
~90 tunable parameters and complete events (frame generated, playback state, emotion recognized, errors) — all exposed to Blueprint.
👁️ In-Editor Preview
See facial output instantly, without entering Play mode.
⚡ One-Call Function Library
Apply generated frames to a SkeletalMesh in a single Blueprint node.
📋 Requirements
PlatformWin64DependenciesLiveLink · MetaHumanCoreTech · NNERuntimeORTAssetAn existing MetaHuman character in your project
🤝 Build a Complete Digital Human
Pairs seamlessly with the RealtimeVoice plugin:
🗣️ RealtimeVoice → 👄 MetaHumanAudioToFaceRuntime
realtime voice + realtime lip-sync = a fully interactive talking digital human.
