I got a fully set up model with every morph targets I need (see attachment 1) and I want to lipsync her in my real time motion capture system when somebody speak to the mic.
What I’ve tried is a plugin called Ovrlipsync and I set up in the exact same way as follow tutorial (sorry it’s Japanese because unfortunately I couldn’t find any official/unofficial tutorials to teach me
how to draw the blueprint):
https://pafuhana1213.hatenablog.com/…8/11/26/011057
As you can see the attachment 2 and 3, for test, when I press 1, the audio should be played and my character should be lipsynced but it just doesn’t work. Can anyone help me to find out where the problems are?
1 Like
You may have to edit a source file in the plugin and rebuild the plugin after:
Plugins/OVRLipSync/Source/OVRLipSync/Private/OVRLipSyncActorComponentBase.cpp
At the bottom of the .cpp is a list of VisemeNames, you should replace the default names after “(TEXT(” with your custom ones that match each said case the best…
thank you so much I rewrite those text and it worked! You saved my life!
And sadly now I got 2 new problems:
1.Every time after I restart unreal engine the lipsync doesn’t work any longer when I play and if I delete the audio and the lipsync sequence, reimport and recreate them, It fixed. But what is going on earth? It shouldn’t work in this way isn’t it?
2. I still need someone with a warm heart to help me to figure out how to draw the BP to lipsync when someone speak to mic. I just copy the BP from the sample project and rewrite it base on my understand but it just doesn’t work.
The error in having to re-create the audio files/lipsync sequences is in the format of the original audio source. I usually run all of the .wavs through Audacity, cleanup, and re-save at 44100 mono. Then within UE4, open the audio file, and check streaming in the options. The way UE4 manages the length of longer audio files is broken unless you do this. Then UE4 can sync which ms of audio is playing for OVRLipsync. So after doing that in UE4, save the wav and right-click, create ovr lipsync sequence. Then it should work with the longer audio tracks, without having to re-create it every time.
I’ve checked the format of my audio and it is 44100 mono as you told me and after I checked streaming in option and then create lipsync sequence, ue4 crashed and here is the report. Tried several times and it was same.
One more thing is after the crash if I restart ue4 and try to create sequence once again, it is not like usual takes few seconds and show me a progress bar until it finishes, it just created the sequence instantly. But the lipsync doesn’t work any more once again.
I told you the order wrong, first create the lip sequence from the audio, then check streaming on within the audio file, after the lip sequence has been created from it.
I change the order and the problem still occurs, I restart my project, lipsync doesn’t work any more. Is it possible for I made any other mistakes?
I am not really sure, sorry. I use UE4 4.21 engine version and it all works fine after doing the above. Maybe it is the extra Blueprint stuff it looks like you are doing there, have you tried it without changing anything except the mesh within the example project from Oculus, while doing the above?
finally I figured out how to solve the proble(probably):
there’s another check which is called seekable below streaming, I tried to check this one, and it worked.
Anyhow, thanks for your patience to help me. I appreciate that.
and I got another question. Do you know how to draw the BP of Live Capture? I try to realize that someone speak to the mic, and my character can be lip synced.
I replaced every part of OVRLipSyncPlaybackActor with OVRLipSyncActor and it doesn’t work. I also add following text to make sure my mic can be used:
[Voice] bEnabled=True
[OnlineSubsystem] bHasVoiceEnabled=False
Can you please help me once again to figure out where the problem is?