It’s time to give something back to the community which I’m glad to see is always willing to help everyone very quickly and efficiently, so this is my small contribution to the developers who wants to add something to their game.
So, what’s all this about?
Introducing the Lipsync Tool, and as you may already guess, it’s a tool that allows you to create lipsync animation for your characters!
Not to be confused with the way other lipsync tools work, meaning that this tool do not analyze the audio/text and gives you phonemes, instead this tool gives you a library of ( english ) animations, which are basically spoken words ( more or less 58000 animations ) for you to apply to your character.
How does it work?
You need to create an Animation Montage where you basically drag&drop the words/animations you need and string them accordingly to create whatever sentence you want, based on the audio you have, and since you can also add audio to the Notifies you have a realtime audio/video feedback of the character moving his/her lips while talking.
What you might need to tweak is the play rate of each animation, so that the word is spoken slower/faster, and it’ll match properly with the audio.
It is a one click solution? What do I need to do to make it work with my character?
This system is heavily inspired by the tool used in the good old Source Engine, meaning that you need to create a set of 19 mouth shapes ( using Maya/Max/Whatever ) which corresponds to a set of phonemes needed for the animation to be applied to your character.
In the test I did I’m using Blend Shapes for the phonemes, but I’m planning to also support joints in the future.
A custom Blueprint will be given so that the only thing you need to do inside UE4 will be to add the reference to your character so that you can use the animations.
How much it’ll cost?
5$? 900$? 1500$? It’ll be completely FREE! Why? Because will still be rough around the edges and lots of tweaks could be made to this tool, but that will require coding, which is something I’m not familiar at all, so that’s why I’ll release this tool completely free.
However if you’ll find this tool usefull you’ll have the possibility to donate whatever amount you want, since I’ll be using that money for hardware upgrade and other custom solutions for UE4
Room for improvement
As said above, a customized UI inside Persona would be great, with the ability to:
- Filter all the words you want to use
- Possibly drag&drop all of them in the anim montage so that they are layered accordingly
- Something else for sure
This will require someone who can code that, so I’ll leave to the community to refine this tool in order to be more user-friendly
When it’ll be available?
I’m planning to release the tool probably around the end of the year or early January ( if everything goes according to plan ).
If you want to know more, have any questions feel free to ask