Supported Frame Rates in MetaHuman Animator

Development Character & Animation
, ,
1

I would like to consult and make a request regarding MetaHuman Animator.

Currently, I am conducting research into audio analysis using MetaHuman Animator.

During this process, we encountered an issue with the FrameRate.

It appears that in the source code the FrameRate is directly set, as shown below:

UE5/Engine/Plugins/MetaHuman/MetaHumanAnimator/Source/MetaHumanToolkit/Private/MetaHumanToolkitBase.cpp

FMetaHumanToolkitBase::SetMediaTrack

// Audio tracks currently don’t have a proper display rate associated with them, so we default to 30 fps

const FFrameRate AssumedAudioDisplayRate(30’000, 1’000);

RatchetMovieSceneDisplayRate(AssumedAudioDisplayRate);

UE5/Engine/Plugins/MetaHuman/MetaHumanAnimator/Source/MetaHumanPerformance/Private/MetaHumanPerformance.cpp

FFrameRate UMetaHumanPerformance::GetFrameRate() const

{

if (FootageCaptureData && !FootageCaptureData->ImageSequences.IsEmpty() && FootageCaptureData->ImageSequences[0])

{

return FootageCaptureData\-\>ImageSequences\[0]\-\>FrameRateOverride;

}

else

{

return FFrameRate(30, 1\); // Default frame rate if no image sequence

}

}

I have two questions:

・To support 60fps, would it be sufficient to simply change these values to 60’000 and 60 respectively?

I am also wondering if there are any other code locations that need to be updated in order to fully support 60fps.

・For future enhancement purposes, could the FrameRate be made configurable via setting items (e.g., in Project Settings or Editor options)?

As it stands now, unless a programmer extends the system, artists cannot select or change the FrameRate themselves.

It would be helpful if the FrameRate setting could support not only 30 or 60 fps, but also values such as 23. 976 or 59.94 fps.

Thank you very much for your support.

[Attachment Removed]

2

Hi,

There are 3 potential framerates to consider within this process:

  • Inference rate
  • Resampling rate
  • Playback rate

Inference rate - This is the rate at which our ML model generates animation keys. This is fixed to 50hz regardless of the other rates used.

Resampling rate - As part of our generation of animation keys we resample the 50hz animation to match the “playback rate” of the sequence.

Playback rate - This is the rate the animation sequence is set to.

Currently in UE5.6 & UE5.7 we match the resampling rate to the playback rate within the Performance asset. As you mention this playback rate defaults to 30hz. Our plan in a future version is to match this better to the inference rate (50hz) so that we are not performing any re-sampling of the keys.

So the changes you suggest should change the playback rate and resampling rate to 60hz and give you 60hz animation. Just be aware that under the hood the animation would have been generated at 50hz and then re-sampled. A better approach might be to change these numbers to 50hz giving you the “raw” animation result, then do interpolation and resampling further down your pipeline.

[Attachment Removed]

3

Thanks :grinning_face:

[Attachment Removed]