Subtitles-based Lip Sync

If officially doesn’t support Android. I can explain how to compile it for Android if you need.

Please use @YuriNK to mention me here. Otherwise, I don’t receive notifications about new messages in this topic. Or just write at e-mail.

@YuriNK I don’t appear to have phonemes, rather just a bunch of very specific morph targets. Is there any way to simplify them into more usable morphs within UE4 blueprints or does an external modeling software need to be used

Can you show a list or screenshot with existing morph targets?

@YuriNK HI~dose this plugin support Chinese?

Hi Yuri,

Hope you are doing well. Huge fan of your plugins. Using two of the right now.

I’m messaging in regards to the lip sync plugin. Getting crashes in the game. The error message is TextLipSync.OnSpeaker_AudioSingleEnvelopeValue

That’s the last thing that pops up before it crashes. Any idea on what the issue might be and how to fix it?

Thanks for your patience and for reading this message.

Looking forward to your response.

Warmest Regards,

Marcus

Just as a reference this is the issue I get from the crash reporter.

LoginId:91b4c5d9470a0338f353ffb476d4eece
EpicAccountId:4e61fe71a1fb457880f7d7dcfb02c492

Assertion failed: (Index >= 0) & (Index < ArrayNum) [File:D:\RocketSync\4.22.0-5660361+++UE4+Release-4.22\Working\Engine\Source\Runtime\Core\Public\Containers/Array.h] [Line: 611] Array index out of bounds: -1 from an array of size 41

UE4Editor_Core
UE4Editor_Core
UE4Editor_Core
UE4Editor_Core
UE4Editor_Text2LipSync!DispatchCheckVerify<void,<lambda_153d5433ab26277b95d537c6705d781e> >() [d:\rocketsync\4.22.0-5660361+++ue4+release-4.22\working\engine\source\runtime\core\public\misc\assertionmacros.h:162]
UE4Editor_Text2LipSync!UTextLipSync::StretchPhraseData() [d:\build++portal+dev-marketplace+full\sync\localbuilds\plugintemp\hostproject\plugins ext2lipsync\source ext2lipsync\private extlipsynccomponent.cpp:1657]
UE4Editor_Text2LipSync!UTextLipSync::OnSpeaker_AudioSingleEnvelopeValue() [d:\build++portal+dev-marketplace+full\sync\localbuilds\plugintemp\hostproject\plugins ext2lipsync\source ext2lipsync\private extlipsynccomponent.cpp:1488]
UE4Editor_Text2LipSync!UTextLipSync::execOnSpeaker_AudioSingleEnvelopeValue() [d:\build++portal+dev-marketplace+full\sync\localbuilds\plugintemp\hostproject\plugins ext2lipsync\source ext2lipsync\public extlipsynccomponent.h:258]
UE4Editor_CoreUObject
UE4Editor_CoreUObject
UE4Editor_Engine
UE4Editor_Engine
UE4Editor_Engine
UE4Editor_Engine
UE4Editor_Engine
UE4Editor_Engine
UE4Editor_Engine
UE4Editor_UnrealEd
UE4Editor_UnrealEd
UE4Editor
UE4Editor
UE4Editor
UE4Editor
UE4Editor
kernel32
ntdll

YuriNK

@YuriNK Hope this works to @ you

Hi,

I’ll answer via e-mail.

So I’m also a Character Creator 3 user. If there are any ways to make it easier for CC3 like another branch without stepping on other’s toes, that would be very welcome. The CC3 use is growing exponentially because of the new skinGen shaders and the easy UE export, so it might be of benefit to you in the long run. I would also approach Reallusion with this. One question I have is if there is a way to make this work with streaming text input, like an avatar helper with AI text output?

Does this work with the default SK_Mannequin?

This works well. One thing I did notice though, possibly a bug, is that if you have a volume slider that the minimum audio doesn’t adjust for low values and if it’s very low the mouth just won’t move. That can be worked around but figured I would mention it as it may not be a known issue.

Looking at your metahumans doc, it appears that you have added the option to use poses instead of morphs. but you have not updated the store page.
It is right?