The new 3.0 MetaHumans: ~40% performance gains!?

[Some]

Edit 4/27/24 - Epic has decided to keep these 40% performance gains UEFN exclusive. Read my latest replies to this thread for updated information regarding this & everything else in this first post. When I wrote this first post, I wasnt too knowledgable in thia area.

Metahumans are known to cause heavy performance load. They are not, in any sense of the phrase, game-ready.

I’ve been chasing a way to make MetaHumans “game-ready” for a while now. They are an amazing tool for creators to use to make their works come alive! But placing more than 3 into a scene destroys any hope of game optimization. The reveal of the 5.3 tools & Epic boasting about the “efficiency and quality” of the new MetaHumans in the UEFN tech demo revived my hopes in a accessible solution to their performance issue.

The performance cost is due to the ridiculous amount of bones (animation calculations), not the complexity of the geometry. Long story short: Even without weighting the fancy corrective bones to any vertices, they still destroy performance at all LODs. So far (pre 5.3) the only way to fix this (theoretically) has been to export MetaHuman to external program, manually delete bones of choice, re-weight vertices, import to UE & retarget animations to new skeleton. Unfortunately, even this solution to MetaHuman game performance has no proof of concept, as everyone online claiming to have done this has not revealed any steps in their process and/or went radio silent.

I just tested these new Metahumans in UEFN 5.4 to see whats new:

Set up:

• The UEFN version is listed at 5.4.
• My local MetaHuman in the UE 5.4pre is version (2.0.3) and not labeled as legacy. Bridge does not prompt me to upgrade them.
• My local MetaHuman in UEFN is version (2.0.3-29956716) and labeled legacy in the Metahuman importer. It prompted me to update them.
• Clicked link in UEFN and updated my MetaHuman in browser to “UE 5.4”.
• My MetaHuman in UEFN is now current and version is (3.0.0-32294560)
• (If you try this by importing a MetaHuman into a new project, you may need to relaunch the program to avoid “generating indefinitely” bug.)
• My newly updated 3.0.0 MetaHuman is not importable into UE 5.4pre from Bridge. Only the old 2.0.3 one.

3.0! but what does it mean?

File sizes:

• 2.0 Low “MetaHumans” folder is 1.22 GB
• 2.0 Low “Girl1” folder is 339 MB
• 3.0 Low “MetaHumans” folder is 135 MB
• 3.0 Low “Girl1” folder is 41.8 MB
• 3.0 Med “MetaHumans” folder is 150 MB
• 3.0 Med “Girl1” folder is 48.6 MB
• 3.0 High “MetaHumans” folder is 176 MB
• 3.0 High “Girl1” folder is 67.3 MB

Keep in mind: during the UEFN tech demo, they said that their MetaHuman had an “average complexity hair style”. Mine has one of the more complex hair styles. So Epic was mostly telling the truth with their 50 MB figure. Well, they omitted the file space the “/Metahumans/Common” folder takes up. Briefly looking inside directories, Epic made the most file size improvements by consolidating 10s of hair & face uassets into just a few.

The LODs available for 3.0 MetaHumans are now organized differently per import quality (Low/Med/High):

• Low Import = 2 LODs (old LOD 5,7)
• Med Import = 3 LODs (old LOD 3,5,7)
• High Import = 4 LODS (old LOD 1,3,5,7) (might be 0 or 1, not sure)

Nothing new here at all that impacts performance, just a new organization of previous LODs available.

There is a new blueprint component available called “MetaHuman”. This comes with some promising settings! I found this component in UEFN 5.4 in the 3.0 MetaHuman blueprint by default, right under LODSync.

Maybe this component will allow us to bypass the heavy processing requirement of having hundreds of non-weighted corrective bones weighing down the skeleton. I’m cautiously optimistic. I’m not too versed in control rig, but having control rig options for compartmentalized sections of the skeleton can’t hurt performance, right? The animation gurus need to chime in on these options for potentiality.

Now onto animations. Since UEFN does not have profiling tools like UE, I did some back-alley testing:

• 2.0 & 3.0 MetaHumans perform the same by default.

• But you get ~ 40% FPS increase in editor when you turn off all “correctives” in the new 3.0 MetaHuman component.

• “Body Correctives” influenced ~ 90% FPS changes.

• “Neck Correctives” & “Neck Proc Control Rig” influenced ~5% FPS changes each.

• I did not test any of the advanced Control Rig settings. I will leave that to others experienced with Control Rig.

(Scalability: Cinematic, Ryzen 5800x / 4070 Ti Super, both mildly overclocked)

(Reminder: for Low import 3.0 MetaHumans, LOD 1 is the highest LOD possible. Equal to 2.0 LOD 7)

(I understand this bad methodoligy for performance profiling. UEFN has profiling tools disabled & no console. You cannot use the MetaHuman component inside of UE 5.4pre either.)

Hopefully UE 5.4 full release comes with the 3.0 MetaHuman component and it’s not hard-locked to UEFN. While the ~40% increase in FPS is nice to have out-of-the box, I don’t believe it will be enough, as others more knowledgeable than I have said the mere presence of the corrective bones on the skeleton is the core issue. More testing will need to be done for use-cases in populated real-time settings. I wish I could be more picky in what correctives are being used and which are not. Such as only having them for elbows but not the hands, legs or feet. At a glance, manual skeleton modification is still on the table.

The 3.0 MetaHumans now come with 25 basic locomotion animations:

• They have root animation
• They have foot step animation notifies

The 3.0 MetaHuman in UEFN comes with a big control rig:

Epic was clear on their intentions for MetaHumans to be used as NPCs only in Fortnite. Not sure if this rig will come with MetaHumans into UE 5.4 full release. Is this procedural locomotion? I’m not sure and I don’t care enough to test it. Let us know if you see anything interesting, animation people.

Python:

I’ve been practicing Python in order to mostly automate a custom modification to the MetaHuman skeleton. The new UE 5.3 API enables bone deletion, so this may be possible to do entirely in UE 5.3+. I’ve already created scripts already that return lists of bones per Mesh, LOD or whatever. So it should be possible maybe with specific skin weight copy and pasting? Python wizards that have messed around with vertex weights, please shout out.

Conclusion:

The new MetaHuman component will help, but not enough I’m afraid. We haven’t even tried animating their faces on top of this! I hope my investment into Python winds up helping me create a mostly automated solution for removing these corrective bones forever.

[Some]

UE 5.4pre Bridge prompted me to update to 3 0 MetaHumans, but they don’t come with the new component.

[Enter_Reality]

Thanks for the detailed analysis, relaly appreciate this

[Some]

Update:

My Python script prototype is near completion.

After fresh project & fresh Metahuman import: running the script goes thru every MetaHuman skeletal mesh & deletes certain groups of corrective bones. The groups of corrective bones are based on the original MH animation LODs.

But some of the mesh verticies normalize to 0 thus getting weighted 1 to the root after deleting certain corrective bones. I’m now manually identifying problem bones & scripting the copying/modifying of vertex weights pre bone deletion.

hopefully its that easy…

[Some]

Update:

After some more research, I learned that the MetaHuman performance cost is not due to the raw # of bones, but the # of vertices influenced by a large # of bones. The engine has to “think” of how to move each vertex during one frame of an animation. And if each vertex has a large number of moving bones to pay attention to, it cant think efficiently. This new understanding of skeletal mesh performance seems more realistic and tangible than simply “delete bones lul”.

In UE, vertexes can only have 8 bones maximum weighted to them. So I made a script to gather data on the worst offending bones before continuting with the bone deletion script. It looks like this:

For every vertex (~17,000 in “f_med_nrw_body”):
If the vertex has >4 bones influencing it:
Count +1 for each of those bones.
Print list of all bones, each notated with the number of times it was weighted to a vertex with 5-8 bones total.

Once I have this list, I can slightly change my targets for what bones I want to prioritize deleting. And perhaps some corrective bones that dont intruduce many stacking bone weights per vertex (elbow, knee) can remain for fidelity purposes.

This part of the saga is where subjective decisions are going to be implemented. So i’m making sure to set up mild scalability settings for other users that use this in the future. Such as bone-per-vertex tolerance (1-8) when identifying offending bones & potentially varying degrees of choice when finally deleting bones. For example: a potato mode for NPC metahumans that deletes more bones and a mode for main character metahumans that leaves some of the fancy bones, primarily in the face and arms.

[tomaaron872]

Thank you so much for your research. Do you mind sharing your python code? I’m curious how that works and I might be able to help with python as I used to program a lot with it in my uni days.

[Enter_Reality]

Very interesting.
Out of curiosity, is there a skinning swap happening between LODs? meaning if below a specific LOD the corrective joints are removed, a new skinning profile is loaded?

Regarding general performances, I think that having a “game character” kind of standard for Metahumans will help a lot of people understanding that default Metahumans are mainly for cinematics, and very few people have the experience or want to dig a bit more to see how they can optimize those characters.
The tech behind is amazing, but the way they release Metahumans was not too easy to understand for most people…then everyone started making beauty shots with their 3090/4090 and that was it

Keep us updated with the python script, I’m really interested to see the amount of improvements you added…does it come with a UI as well?
I think that the possibility to choose the degree of optimization with a simple to read UI will really be helpfull.

[Some]

Of course! Once it begins returning anything other than data. Doesn’t help that I also just started a full time job.

Currently what I have complete is a script that, when given a list of skeletal meshes and a number between 0 & 8, goes through each skeletal mesh vertex and grabs data from vertices with > x amount of bones weighted to them.

Simple on paper but I made mistakes both in learning python & using the wrong/inefficient unreal python classes. This is clearly the best option for my cause
https://docs.unrealengine.com/5.3/en-US/PythonAPI/class/SkinWeightModifier.html

Ive done very few modification tests, as the data collection script just got finished. Pruning at large based on weight thresholds is too barbaric of an option. And prioritizing bones that offend a “# bones weighted per vertex” threshold more often doesnt return a good result either.

My next step is to experiment with adding other factors to the equation. One is bone hierarchicy distance from root. Maybe this will help me prioritize hands and feet first opposed to entire body at a glance. Another is omitting certain groups of bones by choice. I have a few lists of bones from the metahunan lod settings I can see if targeting different groups at different strengths can produce a more desireable effect. And I will 100% need to program logic for skin weight transferrence, opposed to simply pruning and praying.

One the script begins returning a combination of desireable performance gains with little visual loss, I will begin sharing code. But right now I have little time and it must be spent of forward progress rather than communication & collaboration.

Thank you for sharing interest. It helps me stay fueled for a solution

[Some]

No UI plans. But the script is no more complicated than inputting this into python lines:

“Import script as script1”
highlight all metahuman skeletal meshes in content browser
script1.optimize(5, 3)

5 being the given threshold for how many bones are “allowed” to be weighted to any given vertex.
3 being what out of the box LOD bone groups, as you mentioned, to omit or include. 3 being the most agressive LOD settings for the script. Targeting as many bones as possible outside of the bare bones hierarchy

I need to put math in now, like giving skin weights to parent bones when I remove them from bones deep in the hierarchy. Something about unreals built in normalization doesnt produce desireable results.

[Enter_Reality]

I think you can get the skinning informations directly from the Maya source file, since there you also have per LOD meshes where you can get info from.
You may be able to combine that using skin weights profile.

[Some]

I’ve very quickly noticed I’m entering a realm I don’t completely understand. So for now I am researching and doing some profiling to more accurately target why they are non-performant.

I think I have all the data now? But I might be wrong. I have lists of weighted bones per vertex and the id’s, names, weight amounts all returning in a list of tuples.

[Some]

I found a lot of out-of-the-box resources for optimizing animations. Might get some return pre-script by reassembling the modularily of the metahuman in different ways.

Animation Debugging and Optimization in Unreal Engine

[Some]

The Animation Budgeting Plugin works like a charm with the MetaHuman! Honestly surprised how easy it was to implement. Just do some blueprint replacing for 5 mins and you’re good to go. Potential con are the parameters. I hope its able to be tweaked to my liking. But even on default settings, BIG game thread improvement. This was on UE 5.4 preview, 32 MetaHumans running in place, forced LOD 0 & use cards on grooms. Very promising!

Next performance improvements I’ll try with the MetaHuman:

-Animation Compression Library
-Attempting Skeletal Mesh Merge on MetaHuman skeletal meshes.

[Some]

Skeletal Mesh Merge plugin provides ridiculous gains.
Animation Budget Allocator plugin is fantastic and versatile, but not as strong.

[Some]

Default MH:

after Skeletal Mesh Merge:

32 LOD0 MetaHumans without hair

The math makes perfect sense too. Original MH needs to tick 5 skeletal meshes for every one placed. Merged only needs to tick 1 per person.

If I can find a way for other people to easily copy over morphs to newly merged mesh, this is going to be an OP solution.

[SkinnerServices]

This is fantastic work. Being new to UE (3d in general) I quickly discovered the same issues. Hence how I’m here finding your posts.

Great job, even if these plugins aren’t the final solution, it sounds quite promising.

[Some]

UPDATE:

I got the script working. It finally stopped crashing. Not sharing it yet as I’ve not figured out how to use it for the entire skeleton yet, just parts. But the hardest part is done. It works for any skeletal mesh/bones. It’s not Metahuman specific by nature of the script, but of course its compatible.

Give it a list of Skeletal Meshes and Bones; then watch it delete every bone matching the given names on every skeletal mesh given. In regards to using it on the LOD 0 Metahuman, I have only figured out what bones you can “freely” delete in the fingers of each hand (minus the thumb). I started in this area because it proved the be the most unforgiving when deleting bones. Thanks to my relentless testing, no skin weight modification needed to take place, whatsoever! I settled on what I believe to be a best-case scenario for trimming down LOD 0 to maintain 90% of its perceived deform fidelity whilst reducing the raw bone count by ~60%. The secret is in the order in which you delete the bones.

The order of operation for bone deletion took a considerable amount of time to decipher because the surrounding bones soak up the skin weights from the newly deleted bone. One mistake and the finger is going to bend like paper or a rubix cube. Hard lines, not pretty enough for LOD 0. Many permutations were manually attempted.

I eventually found a reasonable compromise (between deleting as much as possible and leaving LOD 0 fidelity intact) which involved deleting 8 bones from each finger, leaving the 3 “bulge” bones & the “palmMid” bone on each finger. This reduces each finger from 22 bones to 8 @ LOD 0 and the fingers still looks mighty beautiful with so less corrective bones involved. A worthy compromise in my eyes. LOD 1+ is of no concern due to their weighted bones containing nearly no correctives. Any any correctives involved I plan to target in my LOD 0 hit list. Here is a picture of the hand flexing @ LOD 0 after losing 60% of its bones. The hard part was making the squishy parts on the inside of each finger stay intact at many degrees of flexion & extension.

Very nice! Hopefully it actually helps performance. My animations are all running just fine, but I imagine they might need re targeting? We will see…

Tomorrow I’ll start knocking out the thumb solution and start moving up the arms. The feet should be easier than the hands. I plan to make them crude with simply toe flexing, nothing fancy like the hands.

[Some]

Script works perfectly for entire body except for the feet. The vertices that make the skeletal mesh around the toes are only weighted 1 bone: the bone segments of the toes. I got lucky with the hands, because I found if I deleted the bones in specific order, there would be no lone-vertices after bone deletion. Lone-vertices get weighted to the root bone automatically. Only way to fix this is to manually re-attach the vertex to a bone of choice.

So I’m now (hopefully) finishing the script by adding a few loops that copy the skin weights from bones to their parents before deleting them. Tricky part is not deleting bones that have children before copying their weights to the closest bone in the hierarchy that we are not planning to delete.

[Enter_Reality]

Holy cow, that’s amazing!
This thread become one of the most interesting things I’ve seen in a while, really good job Some

[mhmdamjd]

yea i noticed that too te performance is better btw im glad to see dscord user in epic forum thats the first time i saw this