Best practices for using morph targets on modular skeletal meshes?

Development Character & Animation
,
1

I’m considering an approach for handling modular skeletal meshes and would like some insight on whether it’s a good idea in terms of both performance and workflow.

The idea is to give certain modular components (like clothing meshes) their own morph targets.
These morphs wouldn’t be for large scale deformations, but instead for subtle corrective adjustments.
For example, if a character pose causes a shirt to fold unnaturally or clip in certain ways, a morph target could be applied to restore the mesh to a more natural shape.

I previously asked a related question here:
If I connect modular meshes to the base skeletal mesh using Master Pose Component, will their morph targets play automatically with animations, or do I need to drive them manually?

Oddly, that thread hasn’t received any responses, which I found surprising — I assumed this workflow would be more common or at least more widely discussed.

So my main questions are:

  1. Does adding morph targets to modular meshes introduce significant overhead, especially when scaled across multiple assets?
    Or is the impact negligible if the morphs are only minor adjustments?
  2. Is this a standard/common or recommended workflow in production pipelines?
    Or is it generally avoided?

In short, I’m trying to figure out if this is a sensible, scalable technique, or if there are pitfalls I might not be aware of.

2

  1. Well as for overhead morph targes are low as far as impact goes and has been updated to use handwear rendering, which once had a performance hit. Since vertex movement is the only thing stored then the impact is relatively minor.

  2. The use of morph injectors is very common with in a production pipeline as it allows for unique characters built with in the same framework.

To play with the idea get your hands on Daz Studio and Genesis 9. Daz3d has been using morph injectors for years to create 1000’s of different characters on a single framework.

3

  1. It’s extra computation - no matter what you do, when you say: “this vertex moves this much because of the animation, now compute how much extra to move it because of the morph” the engine needs to perform extra math on it.
    The cost - and if you can afford it - all comes down to the system in use.

  2. No - it’s generally avoided but only because UE4 doesn’t really support both Morph and Animation - particularly on clothing. Once you enable clothing, you loose the morphs.
    This could have been altered in later engine versions (than 4.2x which is somewhat stable), but then again later engine versions aren’t a viable product to use, so if you do use it it’s highly unlikely you’ll be able to release a game that performs well enough not to get slaughtered by accurate reviews.

Overall - just make your assets outside of the engine in a DCC where you can directly bake what you need as you need, producing individual meshes for each character’s body.

This way your end product will work no matter what you do or what engine you put the assets in - all for essentially no extra work, since you have to do the work in the first place anyway.

4

Hey @MostHost_LA :waving_hand: — thanks for your comments, some of this is very interesting but I’d like to understand more about what you meant by “baking” in this case.

In my pipeline I use Blender blendshapes → export to Unreal → they become morph targets. When you say “produce individual meshes” or “bake what you need … outside of the engine,” do you mean:

1. Baking blendshapes into the base mesh (so the deformation is permanently applied) so that morph targets are no longer needed?

2. Exporting multiple static/skinned meshes each representing a different “morphed” version instead of using morphs in engine?

3. Or something else entirely (like baking vertex deltas, shape keys, etc.)?

Also, how do you handle situations where you want dynamic morphing (e.g. facial expressions or flexible body shapes) but also need good performance / compatibility with clothing and animation?

I’d appreciate if you could expound your workflow: steps, tools, what you bake, how, and how you import into Unreal.

Thanks! :blush:

5

1 and 2.

Bake as in “freeze in place” with whatever morphs applied.

To answer your other question - since in 4.x the pipeline for cloth doesn’t allow for morphs (blendshapes whatever), you can levarage the material with the same effect and some work.

Make a texture that defines the area you want to offset and how it gets offset (a heightmap essentially), then you apply it in the material to the vertex.

The End result is quite the same - you get the animation driving the object, and then a recalculation based on the shader.

The difference is, instead of hacking apart the engine to merge blendshapes/morphs and cloth you went around it.

The downside is the cloth simulation math is only applied Before the material calculations.

Now, for cloths that’s both good and bad. If fat to skinny you have issues since the clot animation is done on the fat. If skinny to fat it works because the material stretches the original animation out and the simulation will likely still fit.

For mixed situation like you are asking, the best way is to split up the mesh. Cloth is separated out. Faces are separated out. Usually even body parts if you have a modular character.

For performance sake you likely want to implement runtime C++ mesh merging on this - but usually the face has to stay separate to run morphs/expressions (mostly due to ease of use? You don’t really want to do single takes with the face involved without a Green screen room and 3 cameras to film markers. Trust me on that.)

Workflow/steps could take me a month to write - and chances are you won’t have the same tools, and neither would I recommend anyone buy them because truly they suck.

The easey part.

Make a character mesh in blender. Weight paint it. Make it look as close to the actor as possible. Make it move well.

The harder part: Rig the face with bones. And weight paint it well. Then take your actor’s picture of the 52 poses required by the system, and start creating keyframes modeled/adjust weightpaint to have the pose match the image as closely as possible. Once done, you bake those as blendshapes (there are scripts to do this publicly avaliable).

With that, disable the new unreal importer BS, enable the legacy one, then import the mode(s) into unreal and assemble - do not use the main project. You just need a take recorded set up (essentially all that the latest engine versions are good for, so engine version isn’t critical here).

With the model assembled, add livelink to the model and set up for face capture (off an iPhone 12x likely, via live link face).

Optionally and I suggest no one does because the system/quality/cost sucks and isn’t worth it. You hook up to Rokoko for motion capute. Live link will allow different options - they all mote or less suck but some are better than others. Generally the inertia stuff doesn’t woek, so rokoko, sisense, etc. are a waste of money or so low fidelity that you have to put thousands into cleanup on top.

At this point you start baking your animations or face capture scenes, save them. Export them out to filter and fix in Blender.

Then save from blender (keep all your files external to the engine). And once you are happy you start the process of importing the clean animations into your acrual project.

At thia step usually make sure the animations sit on your C drive and that you edit them/re-import them with ease.

With all of that, you start to assemble the character and putting together whatever scene you need. This part is really just whatever you need to do with it in engine…

Cloth etc is much the same as the above.

You import it in, set it up as a separate character if you want precision/perfection - or set it up as modular swaps.

I do the modular swaps with a csv sheet that drives what goes where.

Example > Tunic - no chest, no shoulders, no torso, no pelvis, no thighs, etc.

(So the csv is a list of all the parts that make up the body that get removed or included on each given asset).

Thousand ways to do this more or less right… and it’s barely scratching the surface. Still. Should give you some direction.