MetaHuman Curve Animation Memory Size

Hi,

So we are currently looking at memory as it has become an issue on our project. I baked down down some of the MH animations to joints and was shocked at the difference

I will use the fear animation that you get through MHC as reference

[Image Removed]

[Image Removed]

We do lose quite a bit of fidelity in the face so we don’t want to necessarily go down this route.

I am wondering if there are ways to make the memory footprint at least competitive with joints and if this is something that’s being looked at for future improvement?

[Attachment Removed]

Hi, is it the editor memory usage or the memory usage in a cooked build that you’re concerned about?

To go into a bit more detail about what’s going on under the hood with the sequences, we store compressed data separately from the editor representation of the data. The compressed data is stored on the anim sequence object, however, the editor representation is stored in the data model (usually UAnimationSequencerDataModel), which is a separate object and so won’t show up in the size map comparison. The editor representation of the data is also much larger than the compressed data, since it’s stored in MovieSceneFloatChannel and MovieSceneTransformChannel formats, which are much more complex than the compressed buffers (and uncompressed).

What’s likely skewing the figures you’re seeing here is that, for the editor data, we copy the curve data back from the data model object to the anim sequence. So the editor curve data is stored in both the sequence and the data model, but the bone transforms are only stored in the data model. We do this because Montages use the raw curve data at runtime, so we have to copy it back from the compressed data. But this is stripped out of anim sequences on cook, so it shouldn’t affect sequences at runtime.

To go back to the original point about the size map, when you look at the sequence there, you should be seeing the compressed data (bones & curves) and the editor curve data. So it’s not a straight comparison to strip out the curves and compare before/after in the size map.

One alternative would be to look at the data that’s displayed within the sequence itself in the Asset Details panel. We display an estimate of the uncompressed data size and the compressed curve & bone data sizes:

[Image Removed]So you could repeat your test and look at those values, that should give you a somewhat more accurate idea of the differences in the sizes.

[Attachment Removed]

Hi Euan,

I should have clarified, yes we are only concerned about memory usage in a cooked build. If the size map only looks at the memory in editor then I don’t really see the point of it as profiling tool.

Is it just bad for animation curves specifically given the complexity of it? How should I be using it if at all if I’m only concerned about memory usage in the build?

I looked the same assets again in the compression menu and here is what I found

[Image Removed]

So essentially the 121 KB for the joint only data is just the editor memory and 853.886 KB is much closer to the memory usage would be in a build? Why the huge difference?

And for the curve animation if 595.406 KB is the more accurate figure for memory usage then going with anim curves looks like the obvious choice here. I’m I correct in concluding this?

Thanks

Robert

[Attachment Removed]

Hi Robert,

Your mileage is going to vary a lot with the size mapper tool, depending on the assets that you’re looking at. The closer the cooked data format is to the editor format, the more accurate the info is going to be for runtime. But with anim sequences, both the data is very different in editor vs runtime (compressed vs raw) but also we store the data in different objects outside of the anim sequence which the size mapper tool doesn’t track.

If you want to look at the memory usage of the sequences in a cooked build, the best option is to use the cvar Obj List Class=AnimSequence. That’ll print out info for all the sequences that are currently in memory, and it should be pretty similar to the compressed data size that you see in the asset details panel for the sequence, plus the other serialized properties. But just be aware that command won’t give you an accurate size in the editor, so you need to run it in a cooked build.

The other option would be to take an insights capture and then use memory insights to look in more detail at what’s going on in a cooked build. There is also AnimSeqStats which will give you more detail on the composition of the animation, the keys, etc. Ideally, we would have better tooling for this, that was correct for both editor and cooked builds.

In terms of the other questions you had:

> So essentially the 121 KB for the joint only data is just the editor memory and 853.886 KB is much closer to the memory usage would be in a build? Why the huge difference?

Yes, I would expect the memory usage for the sequence in a cooked build to be similar to the ~850kb reported in the asset details panel plus other serialized data. I think the reason for the difference here is similar to what I mentioned above, in that the data is stored in a different object that the size mapper isn’t tracking.

And for the curve animation if 595.406 KB is the more accurate figure for memory usage then going with anim curves looks like the obvious choice here. I’m I correct in concluding this?

It depends on the specifics of the animation, how many bones there are vs curves, how many frames have keys, etc. But generally, a curve is going to be cheaper than a bone transform because there’s less data to represent (1 float vs 7 - assuming no scale on the bone). I noticed in your example, the compression codecs are different between the first screenshot and the second. So I would also recommend always using ACL compression for both, where possible (the main limitation is that stepped keys aren’t supported properly with ACL compression). With ACL, compression of curves should be broadly in line with compression of bones (accounting for the fact that there’s more data to represent in a bone transform vs a curve).

Thanks,

Euan

[Attachment Removed]

Hi Euan,

I have some questions about memory on the editor side. Is that something worth paying attention to?

Will everything become more unstable if we start to have hundreds or thousands of curve animations?

Can anything be done to mitigate if this could become an issue?

Are there any other stats to pay attention to in general?

Thanks,

Robert

[Attachment Removed]

Also on the size map, Is the disk size reading correct at least? Or is there a Editor/Build distinction there as well?

[Attachment Removed]

Hi Robert,

Last question first:

> Also on the size map, Is the disk size reading correct at least? Or is there a Editor/Build distinction there as well?

Unfortunately, this is still more complicated than you would hope. For disk size in Size Map we rely on the package size directly, rather than running the serialize method for each of the objects. And that should include the compressed data when you’re looking at the editor asset size. But this size will also include the data model, which is misleading if you’re interested in the cooked data size. (You’ll also get a different result if you look at the size of the cooked asset in Size Map as in the cooked data, we serialize the compressed bone transforms out to bulk data instead of including them in the anim sequence package.)

This is all pretty unintuitive and not very helpful, so I’m going to raise this with the dev team to see if we can do something to better support displaying the actual values for editor vs cooked, both in terms of memory and disk usage.

> I have some questions about memory on the editor side. Is that something worth paying attention to?

Will everything become more unstable if we start to have hundreds or thousands of curve animations?

Can anything be done to mitigate if this could become an issue?

Yes, it’s possibly something to be concerned about on large projects. Both memory usage and processing times are affected when loading all the animation data. The usual way to avoid those kinds of issues within the editor is to avoid hard references that load all your animations when you load the editor and/or a level. If nothing within the level has a hard reference to the animations, then they won’t be loaded by default, and you won’t pay the cost for loading them. Something to look out for with this is that if you use Motion Matching, the normalization databases tend to end up loading all the animations that are used by a mesh when the mesh is loaded. That shouldn’t be an issue for facial animations, they would only need to be loaded when they are referenced by an anim bp, level sequence, etc, that’s loaded. But you could end up in a situation where you load a mesh because it’s placed in a level, which in turn references an anim bp that uses all your facial animations, so they are all loaded when the level is loaded. Using soft references whenever possible avoids this.

There are some cvars you can also look at related to memory usage. We have a.StripFramesOnCompression & a.StripOddFramesWhenFrameStripping which we use on Fortnite to reduce the memory usage of anim sequences by stripping out frames when generating the compressed data.

There’s also a.AnimSequencer.LazyRigHierarchyInitMode. This won’t improve memory usage, but it’ll affect processing times when loading anim sequences. By default, when sequences are loaded, a FK Control Rig is created under the hood for each sequence (the idea being that you’ll eventually be able to modify the animations using that rig directly in-engine). But this is expensive when you load a lot of animations at once. This cvar skips that on load. This option is experimental however, and there have been issues with it causing deadlocks on cook in some more complex projects. We’ve made some improvements in 5.8, which we hope have resolved that, but we haven’t tested it on Fortnite yet.

> Are there any other stats to pay attention to in general?

I think Obj List Class=AnimSequence and AnimSeqStats are going to be the most useful for tracking anim sequence memory usage. And memory insights. There is also memreport -full, which will dump out a file to your project’s /Saved/Profiling/ folder, but that seems to mostly just be generated via Obj List Classfor various types.

[Attachment Removed]

Hi Euan,

Thanks a lot for this information! I’m glad you also see how confusing all this can be when profiling.

I think a really nice change to the size map is if it looked at all the aspects of an animation sequence.

And maybe even having them as sub-squares in the main Anim Sequence square. It would be useful to see which piece is adding the most.

And also if it could display an “estimated compressed” or something in the viewer without having go into the details panel on the asset itself that would be awesome.

Thanks,

Robert

[Attachment Removed]

Hi Robert,

Yeah, I agree that we should try to improve this to make it more intuitive/useful. The issue is that size map relies on default functionality of classes/packages (ie. serialization path and package size), so it may be difficult to customize the behaviour to get it to display all of the relevant data for sequences. But I think it’s worth spending some time investigating this, and the dev team agree, so I’ve made a feature request to improve the information we’re surfacing, although I can’t say what the timeline would be for that work.

Thanks for the feedback on this!

Euan

[Attachment Removed]