How to prepare your model for facial animation in UE4

I am almost done with the main character for a game I am developing, and want to make sure my preparations for the character are as complete as possible so that I don’t get major setbacks down the line. For that reason I am now also preparing the character for basic animation. Have to admit though, animation is not my strongest suit and so far the animations I have done for any 3D game were limited mostly to the loops for in-game use. Nothing fancy for cutscenes. Since the game I am now working on will likely feature cutscenes, I wish to know what I need to take into account for facial animation and how I can use, for instance, voice overs and the likes in UE4. I should note that I am very likely going to rely on morph targets. I already constructed the normal rig and would prefer not to have to redo it (only when it’s really necessary). Not a big fan of rigging :stuck_out_tongue:

The technical requirements is a tough nut to crack as the set up, frame work, is an area that you can paint yourself into a corner and it’s not uncommon for an entire character development pipeline to be ripped apart so the fact that you might have to do a redo of your rigging should be included as part of your development plan. On one project I redid the rigging and character setup 4 times to take advantage of features added during the development process.

As luck would have it though there are converging technology that will help make the process easier via practical examination and one product we are taking advantage of is the Genesis 3 frame work from Daz3D and after years of consideration have licensed the use of Daz Originals as part of out development pipeline.

A quick test.

For your needs Daz Studio and the Genesis 3 frame work is available as a free download and is of studio grade that is easily reconfigured for use in a video game and the rigging supports both morph targets as well as cluster shaping that can be used with any facial tracking system as well as voice devices. At it’s very basic G3 is an excellent product to test theory quickly prior to implementation.

As an example of theory testing we decided not to use morph targets due to a huge performance loss on hi resolution characters and will be using the cluster option as demonstrated in the example as it is a feature built into the rigging that is hardware rendered and keeps the animation requirements with in the same animation pipeline. Via testing it’s clear using morphing will at some point paint our project into a corner.

Hmmm, I will have to keep this in mind. I will also keep in mind that creating a new rig may become a necessity.

Since I need to produce something playable before we can actually start implementing anything resembling narrative, I may as well just focus on the loops for now. I will definitely keep DAZ in mind though; if we get to the point where the character needs more complicated animations, I may redo the rig again while trying to incorporate DAZ into the pipeline. Considering my current experience, that may actually be for the best as mu current rig is probably not optimal.

Well the nice thing about Daz Studio and the G3 framework is it all can be pipelined via FBX so you don’t have to commit as it does not cost you anything except for add ons that you want to buy. A group I’m working with just purchased a license extension to have access to just the G3 framework, after about 6 years of humming and hawing, which solves what we needed for a few bucks.

Hope i don’t hijack this thread by asking into the G3 framework from Daz, i tried to look into this, since this post seems super relevant for anyone doing a main character and wanting to allow it to have facial animations as well. So FrankieV, can you explain a bit more what the Genesis 3 does from a practical perspective? Also i found it really confusing to try to figure out what i had to purchase if i wanted to use this with UE4 and what the price would be.

I’ve also just briefly looked into Faceware, not sure anyone has experience with that, but looked pretty cool too.

Well I’ll try to put it all into some kind of context as Daz Studio is one of those apps that at first glance seems like a toy until you find some kind of killer app, need, like say a 3ds Max or Maya as to possibilities figured out via discovery. This is the stage where DS is at the moment as it’s clear as to it’s use in the world of visual media that you will never notice as to the question “How did they do that?”

Some user stories.

Scroll to the bottom and there is more than a few stories as to use via discovery.

As to our decision to make use of at least the Genesis 3 frame work was based on the requirement of being able to jump the technical requirements of code and the need to have a pipeline to conform to the demands of content where real time results is a requirement and it does take a lot of time and effort to even develop a basic frame work and adding in functionality is a time consuming vampire as to design concepts that you would like to add that goes beyond the demands of code and content.

More and more though there is a market demand were visuals we see in visual media, like movies, needs to translate much closer to the original and with Unreal 4 changing the game so to speak can be adapted via convergence into a 3d application that supports a real time rendering and movement sub system.

Example.

What is impressive is how affordable the hardware is becoming but still expensive is creating the frame work and art assets necessary to connect the hardware up to and as to that need Daz Studio/Genesis 3 fills the gap.

Overall what you can do with Daz Studio is best left to the imagination but like most high end applications it’s of studio grade as to usability with in the visual industry and up to Epic and Unreal 4 to create the bridge as they have done for VR and ArchVis.

How much does this cost?

To get the basics, Daz Studio with Genesis 3, is a free download so as far as a test drive goes it does not cost a cent.

Daz3d’s business is selling art assets so whatever you purchase comes with a fair use license for “any” use as to visual media. So if you needed some 2D elements rendered you could stage the asset in DS, render the image, and free to us it with out further licensing.

To make use of the source asset in your game you need to purchase a license extension. In our case after discount was $200 for Daz Original under $100,000. This what makes it a bit confusing as it is vending machine licensing where you buy, via the store, the type of licensing extension you need from the merchant selling the art asset.

For me though it was a no brainer to sped $200 bucks on a frame work that works but with the added bonus of a character model that can act which is not even a design requirement.

Hi FrankieV,

I’m also exploring using Genesis 3 in UE4 and have a few questions if you don’t mind answering them.

Can you explain a bit about “cluster shaping” as an alternative to morph targets? I googled the term and couldn’t find any references to what it is.

Do you create your animations in Daz Studio or in another program such as Max?

Did you have to deal with the twist bones in the Genesis 3 skeleton? From what I can tell they would make foot ik difficult.

Thanks.

Genesis 3 is set up to use both cluster as well as morph targets so you can harvest one or the other or make use of both at the same time. Another reason G3 makes for a excellent development tool as one can test both and make decisions based on discovery rather than investing massive amounts of time necessary to make a functional asset. Playing with a studio grade asset is the best way to figure out what is what. :wink:

Cluster shaping is easy to understand as it is just another joint added to the base rigging as a child bone that only effects a fixed volume of space. As part of the frame work though clusters lends it’s self nicely as to authoring the required animations as it all works on the same channel as a run cycle or as an additive using per bone layering.

Morph targets on the other hand has to many drawbacks, that I can get into if interested, but the deal breaker is morphing is not hardware rendered so suffers from a massive and noticeable performance drop so we opted for clusters.

For our animations we use Motion Builder for authoring but the following is as good as any tutorial as to how to set up clusters using either Maya or 3ds Max. Once again a cluster is just a joint that effects an area of influence.

Setting up clusters in Maya.https://youtube.com/watch?v=jMb_ejEgKqk
To wire in the cluster shapes this is more or less the technique I used to hook up the voice device that drives the lip sync I used insider of Motion Builder as demonstrated in the test video I posted.

Using cluster shapes https://youtube.com/watch?v=V2LSJHKichg
The benefit with in a real time game engine is there is no need for actors and a single dev could produce all of the dialogue tracks necessary using just digitally clean audio samples as well all of the dialogue takes fit with in the same animation migration pathway.

Could I do the animation in Daz? Sure but I’ve been using MB since the free lancer versions so over the years I’ve collected so many animation takes that the need for creating the animations for any project is all ready done and just needs to be implemented to taste.

Twist bones was not an issue as Motion Builder deals with them as part of the auto rig and is a needed and common element as part of a base rigging frame work and is not unique as far as IK and UE4 goes. The most common problem as to IK twisting though is usually caused in how the joints are linked.

Thank you very much for the comprehensive reply and for the links. Most appreciated.

Your welcome.

DS is an interesting product that has a lot of cross app compatibility with Unreal 4 and would be nice in the future if there was tighter integration as jumping the gap is a bit awkward at the moment.