[FEATURE REQUEST] Facial animation

[Eugen der Starke]

UE4 is great for animations. The most animations I saw though try to hide the face of characters in some way, because there is no facial animation possible, except doing it hard way... no simple lip synch either...

We know it can be easilly done with Unity plugin for Mixamo-Fuse (FacePlus). Mixamo doesn't want to do it for ue4, which is really pitty. So maybe Epic could do it?

Thanks!

[darthviper107]

Facial animation is possible, either through morphs or bone animation, it just doesn't have a tool that allows you to create it in the editor

[FrankieV]

Everything one needs to do facial animation is already built into the animation pipeline by default and layer blending.

Example.

The magic trick was simple.

I brought in one of our player models set up with 72 facial cluster into motion builder, snapped 6 target shapes, and attached a voice mocap device and then plotted the data down to the base rig. Importing it into Unreal 4 is the same process of importing any other kind of animation, a run cycle for example. To make it work a simple layer by bone was needed to target the animation to the root of the facial cluster and off it goes.

The issue is not having a engine side feature but the need for low cost authoring tools.

[Eugen der Starke]

I know the possibility is there, but it's not high level :-)
Like Assembler vs. C++, or UE4 vs. plain C++ :-)
UE4 lets me develop my game 5-10 times faster, than plain C++...
Thus animating my characters facial motions manually would cost me as much time as the rest of game.

Please take a look at FacePlus, too see what I mean ;-)

[darthviper107]

That's just a tool, 3D animation software usually have tools or a plugin that can do that type of thing. For most cases you would make your animations in another program, like Motionbuilder. In some cases you might want to avoid having tons of facial animation files so there is middleware out there that allow you to sync text with your audio so that it can then analyze the text and blend your facial poses to match, though it doesn't always look very good--that's the type of facial animation you see in games like Mass Effect, or Fallout/Elders Scrolls. UE3 had a system like that called FaceFX, but it's not included with UE4 since they wanted to strip out the middleware they would prevent them from sharing the source code for.

[ryan20fun]

Or what about using phoneme and viseme based lip syncing?
Links:
A Practical and Configurable Lip Sync Method for Games
Automated Lip-Sync for 3D-Character Animation
COMPARISON OF PHONEME AND VISEME BASED ACOUSTIC UNITS FOR SPEECH DRIVEN REALISTIC LIP ANIMATION

HTH

[NegInfinity]

That's just a tool, 3D animation software usually have tools or a plugin that can do that type of thing. For most cases you would make your animations in another program, like Motionbuilder. In some cases you might want to avoid having tons of facial animation files so there is middleware out there that allow you to sync text with your audio so that it can then analyze the text and blend your facial poses to match, though it doesn't always look very good--that's the type of facial animation you see in games like Mass Effect, or Fallout/Elders Scrolls. UE3 had a system like that called FaceFX, but it's not included with UE4 since they wanted to strip out the middleware they would prevent them from sharing the source code for.

I believe it would be worthwhile to have builtin lipsyncing tool that can produce lipsync animation track for animation. I'm speaking of morpheme based tool here.

Based on quick search on the web, while unreal does have blendshape support (which can be used for lipsync), it looks like there's no builtin solution for that kind of thing now, and you'll need to write some sort of lipsync controller yourself.

Offloading that kind of work to external source is not very good solution, because lipsync is tied to localization process. So, it is different from standard animation sequence and it should be built into engine.

[ULLS]

Just thought I'd throw this in here as it's the best example of facial animations in UE4 I've witnessed so far:

https://forums.unrealengine.com/show...ial+animations

I personally have driven facial animations using timeline nodes controlling the morph targets, not the best way to do it I know but it worked and I'm an amateur so.....

[Eugen der Starke]

Wow, so much new information for me! I definitely will follow those links and try them out!

Motion Builder - I might be mistaken, but I thought it costs a lot of money and isn't something an indie game developer could aford, especially if married and having two children :-) jocking...

Yes you are right - there is a lot of middleware, 3rd party etc... It's just when I saw UE4 for the first time I told myself - that's everything I could ever need! And in many ways it is!!! Respect to Epic!!!!!

But you wanted to know, what you could improve, so I thought it would be nice to have the facial animation possibility. And since as I've heard the unity has such a plugin - it made me believe it cannot be that hard :-)

It could be a paid plugin. Maybe 50 or even 100$. But buying the MotionBuilder for facial animations would be too much. Maybe we should start a poll how many developers need such a plugin and would buy it :-)

Thanks to all reply!!!!!!

[EoinOBroin]

I believe the ART tool by Epic will be getting an update with a facial rigger that is near completion! Keep an eye out for it!

[order66]

We know it can be easilly done with Unity plugin for Mixamo-Fuse (FacePlus). Mixamo doesn't want to do it for ue4, which is really pitty. So maybe Epic could do it?

Why don't you just use FacePlus? They do have a plugin for MotionBuilder. From there you can export the animation data to UE4.

[Steve_T]

FaceFX is also a thing but is a lot of monies...