Animation Samples vs. Frames Counts

Development Character & Animation
, ,
1

This question was created in reference to: [UAnimationBlueprintLibrary::GetNumFrames != [Content removed]

I’m noticing additional inconsistencies around how the animation code treats frame counts vs. sample counts that I’m hoping to better understand:

  • UAnimSequenceBase::GetNumberOfFrames - deprecated function, returns GetNumberOfSampledKeys() as the number of frames.
  • UAnimationBlueprintLibrary::GetNumFrames - the number of frames is GetNumberOfSampledKeys() - 1
  • UAnimDataModel::PostLoad - initializes NumberOfFrames to NumberOfKeys - 1
  • UAnimSequenceBase::GetFrameAtTime - the maximum frame number (0-indexed) is GetNumberOfSampledKeys() - 1 in the clamp, which implies the number of frames is actually GetNumberOfSampledKeys()
  • UAnimToTextureBPLibrary::GetAnimationFrameRange - again treats GetNumberOfSampledKeys() - 1 as the end frame index, where the start frame index is 0, again implying GetNumberOfSampledKeys() is the number of frames.
  • BlendSpaceAnalysis.h under the Persona module - uses GetNumberOfSampledKeys() - 1 as the number of sampled keys (???) but later seems to use it as the last sample key index.

Often the last sample (not frame) of an animation is identical to the 0th sample so that looped animations are seamless. The consequence of treating GetNumberOfSampledKeys() - 1 as the last frame index is that tools like AnimToTexture bake out the same data twice (first and last frame) causing one frame of freezing duration looped animations. It seems to me that it should be using 0 as the first frame index and GetNumberOfSampledKeys() - 2 (ie. number of frames - 1) as the last frame index.

Is that potentially a bug, or am I misunderstanding Unreal’s definition of samples vs. frames?

Thanks!

2

Hi, in the engine you should find that there is always one less frame in an animation sequence vs the number of keys. So there is a final key in each animation which is not necessarily the same pose as the final frame of the animation. Another way to think of it is that the frames sit between each of the keys.

I’ve included a screenshot below as an example from the final frame of one of the locomotion animations in the Gameplay Animation Sample. If you set the animation to use stepped rather than linear interpolation, you can see what the poses in each frame looks like:

[Image Removed]And in that example, the final frame (the 120th frame, which is frame number 119) has a different pose than frame 0 (and also a different pose than the final key, which is the same as the first key).

With that in mind, I’ll go through each of the APIs that you mentioned.

> UAnimSequenceBase::GetNumberOfFrames - deprecated function, returns GetNumberOfSampledKeys() as the number of frames.

This method is really old at this point and has been deprecated for a long time. But back in the UE4 days when it was used, the implementation was wrong and it caused problems since number of frames != number of keys. I wouldn’t use that function, and we’ve removed it in 5.6.

> UAnimationBlueprintLibrary::GetNumFrames - the number of frames is GetNumberOfSampledKeys() - 1

This is as expected based on the info above.

> UAnimDataModel::PostLoad - initializes NumberOfFrames to NumberOfKeys - 1

Also as expected.

> UAnimSequenceBase::GetFrameAtTime - the maximum frame number (0-indexed) is GetNumberOfSampledKeys() - 1 in the clamp, which implies the number of frames is actually GetNumberOfSampledKeys()

This one is interesting. I think we have a bug in UAnimSequenceBase::GetNumberOfSampledKeys which is called from UAnimSequenceBase::GetFrameAtTime. UAnimSequenceBase::GetNumberOfSampledKeys is just doing a straight conversion of sequence length into frame time and returning that as the number of keys. So if you have a 2 second clip at 30fps, that will return 60 keys. But that’s not correct - it’s 60 frames and 61 keys. I’ll need to confirm this with the dev team next week. That API is only used by montages, which is possibly why we haven’t seen it causing problems. Anim sequences all use the UAnimSequence::GetNumberOfSampledKeys implementation instead.

> UAnimToTextureBPLibrary::GetAnimationFrameRange - again treats GetNumberOfSampledKeys() - 1 as the end frame index, where the start frame index is 0, again implying GetNumberOfSampledKeys() is the number of frames.

There is also a bug in this method by the looks of it, but it’s not the OutEndFrame calculation. That method also returns the number of frames in the animation. But it’s calculated by doing `OutEndFrame - OutStartFrame + 1;` which means we’re effectively returning the number of keys, not frames. Back in UAnimToTextureBPLibrary::AnimationToTexture we use that to calculate the time to sample from so we’ll effectively try and sample frame 60 (which is frame 0). That would mean a duplicate frame in the texture that gets generated.

However, looking at the material function, when we sample the texture we always subtract 1 from the total number of frames stored in the data asset. So I think in practice, that means we never sample the duplicate frame when playing back the material. But if you see a duplicated frame during playback, please let me know. It’s hard to say for sure without testing a working setup.

> BlendSpaceAnalysis.h under the Persona module - uses GetNumberOfSampledKeys() - 1 as the number of sampled keys (???) but later seems to use it as the last sample key index.

I’ll need to check with the dev team and follow up on this one. The variable naming doesn’t help here, but it is effectively working with frames rather than keys.

3

Thanks for the detailed breakdown Euan!

I’ve attached a minimal UE5.4 project that should demonstrate the AnimToTexture bug I was running into. If you open up /Game/Scene it’s got a mesh animated using a VAT, another one posed on frame 33, and the last one posed on frame 49. Frame 33 and 49 are identical, causing a pause in the run cycle. I slowed down the play rate of the animated one so the pause is more obvious. The original run animation (AS_Run) is 16 frames, 17 samples long, with the last sample identical to the first. But the baked out animation is frames 33 to 49, which is 17 frames long.

4

Thanks for attaching the repro project. It’s clear from that that we do have a bug and I think that also actually repros on the mannequin setup that ships with the plugin if you run it at a low enough play rate. There’s two issues going on. The first is the one that I mentioned where we’re baking out too many frames into the texture. But the second is the one you mentioned where the end frame that gets written into the material instance is also off by one. I hadn’t realized previously this was being used within the material.

There’s two ways to fix the end frame being incorrect. One is to make the change that you mentioned where we subtract 2 to get the end frame instead of one. The other is to change the node highlighted below from the GetAutoPlayFrame material function, since end frame is effectively already the number of frames:

[Image Removed]I think that should also ‘fix’ the issue. I’ll need to discuss with the dev team how we want to fix this but I expect we’ll go with changing the native implementation since changing the material function is only really a workaround.

5

Thanks for confirming!

I’m going to make the end-frame baking fix locally and will keep an eye out for any official fixes coming down the pipe.

6

No problem. I’ve logged a JIRA that you can track here about this issue. It’ll be updated once I submit a fix. If that goes into a Fortnite branch, you may not have access to the CL, so feel free to either reopen this thread or start another one if you want access to the changes prior to the next release.