ACE Audio2Face-3D: AnimateFromAudioSamples only works for the first audio clip, subsequent clips fail (only first frame of animation)

Development Programming & Scripting
, , ,
1

Body:

Hi everyone,

I’m working on integrating NVIDIA ACE Audio2Face-3D plugin into my Unreal Engine 5 project. I’m using FACERuntimeModule::Get().AnimateFromAudioSamples() to drive MetaHuman lip sync with runtime-generated PCM audio.

Problem:

  • First call: Works perfectly. Audio plays, lip sync animation runs correctly from start to end.

  • Second call (same or different text, same component): Only the first frame of lip sync animation plays (mouth twitches briefly), then animation stops immediately. Audio still plays correctly (I can hear the voice), but no lip movement after that initial frame.

My setup:

  • Using bEndOfSamples = true for each complete audio clip.

  • Audio format: 16kHz, 16-bit PCM, mono.

  • Same UACEAudioCurveSourceComponent* Consumer is used for all calls.

  • I’m calling EndAudioSamples() before the second call (tried both with and without it).

What I’ve tried (none worked):

  1. Calling EndAudioSamples() before each new audio clip.

  2. Adding Delay (0.05s ~ 1.0s) between calls.

  3. Destroying and recreating the component.

  4. Using different audio clips (different text content/length).

  5. Adjusting BufferLengthInSeconds on the component.

Observations:

  • Long texts (several seconds) work fine. But if try the same long texts again, the issue still happen.

  • Short texts (e.g., “Hello”, “你好”) fail consistently.

  • Audio playback is always correct — the problem is purely lip sync animation.

  • No error logs from ACE module.

Question:
Is there any hidden state or session that needs to be reset between calls to AnimateFromAudioSamples when using bEndOfSamples=true?

Is there a proper way to restart the Audio2Face session after it has finished processing a clip?

Any help or pointers would be greatly appreciated!

UE Version: 5.4
ACE Plugin Version: Latest (from NVIDIA GitHub)
Platform: Windows

Thanks in advance.