Layered blend per bone with multiple entries changes output when weight is 0 in one of the entries

Development Character & Animation
, ,
1

We just noticed this by chance in one of our ABPs. If you have 2 blend poses:

  • Blend pose 0:
    • Weight: 0
    • Animation: null
    • Branch filter: clavicle_l, depth 0
  • Blend pose 1:
    • Weight: 1
    • Animation: Some animation
    • Branch filter: clavicle_r, depth 0

I was expecting to see just the clavicle_r animation playing cleanly. Instead, the shoulder is rotated further than what the animation feeds. However if I set a valid animation and a very small weight on blend pose 0, this is solved.

It seem the blend poses per bone filter resets one of the poses to refpose if weight is 0, and it somehow still gets blended with the result. Just wondering if it’s a known issue. The workaround so far seems to be setting an animation and a small weight always.

2

Hi, I don’t think we’ve run into this issue previously. Are you running with just a regular local space blend, or are you using any of the mesh space blend options? Could you also try disabling ANIM_BLEND_POSES_PER_BONE_FILTER_ISPC_ENABLED_DEFAULT to see if that changes the behaviour at all?

The code performing the actual blend should be in FAnimationRuntime::BlendPosesPerBoneFilter. There are various branches in there depending on whether it’s a local space blend, etc. But from quickly reading through the code, I don’t see that the transforms from Blend Pose 0 should be affecting the output transforms if the weight on that pose is 0. Assuming there’s no difference between the ispc and regular code, the blend will be doing something like this:

for (const FCompactPoseBoneIndex BoneIndex : BasePose.ForEachBoneIndex())
{
	const int32 PoseIndex = BoneBlendWeights[BoneIndex.GetInt()].SourceIndex;
 
	const FTransform& BaseAtom = BasePose[BoneIndex];
	const FTransform& TargetAtom = BlendPoses[PoseIndex][BoneIndex];
	FTransform BlendAtom;
 
	const float BlendWeight = FMath::Clamp(BoneBlendWeights[BoneIndex.GetInt()].BlendWeight, 0.f, 1.f);
	MaxPoseWeights[PoseIndex] = FMath::Max(MaxPoseWeights[PoseIndex], BlendWeight);
 
	if (!FAnimWeight::IsRelevant(BlendWeight))
	{
		BlendAtom = BaseAtom;
	}
	else if (FAnimWeight::IsFullWeight(BlendWeight))
	{
		BlendAtom = TargetAtom;
	}
	else
	{
		BlendAtom = BaseAtom;
		BlendAtom.BlendWith(TargetAtom, BlendWeight);
	}
 
	OutPose[BoneIndex] = BlendAtom;
}

BoneBlendWeights should contain the per-bone weights for Blend Pose 0 and Blend Pose 1. Since the weight for Blend Pose 0 is 0, any bone that could be affected by that pose should just end up going down the code path that outputs the transform from the base pose:

if (!FAnimWeight::IsRelevant(BlendWeight))
{
	BlendAtom = BaseAtom;
}

And then any bone affected by Blend Pose 1 should just take the transform from Blend Pose 1:

else if (FAnimWeight::IsFullWeight(BlendWeight))
{
	BlendAtom = TargetAtom;
}

However, I could be missing something since I haven’t been able to get a repro yet.

3

Just a quick follow up that I’m still working on a full fix for this issue. We have a potential way forward, I just need to write the code. However, I’m going to be out of office for the rest of this week so I’ll follow up again next week.

4

Hi [mention removed]​, sorry for the delay getting back to you on this, it’s been a busy couple of weeks. I have a shelf ready for you to test - it’s 46149208.

The shelf mostly just changes how the mesh space transforms are accumulated (for both scale and rotation). Previously, like I mentioned before, we were accumulating the target mesh space transforms from a mix of local space transforms (depending on the layer that was active for each bone index). The shelf adds an extra loop where we accumulate the mesh space transforms for each layer before iterating over the bones. That means the mesh space buffer should be valid for the current layer that we’re iterating over. Then we only apply those transforms to bones that are using the current layer. I’ve updated both the C++ and ISPC implementations of all of the mesh space blend types.

As usual, I’d recommend testing these changes extensively. I’ve done some basic tests with the assets that you sent over, but this hasn’t been through a QA test.

Let me know how you get on.

5

Great, thanks for letting me know. I’m in the process of forward porting this with the aim of getting it into 5.8. I’ll close out this thread but you can always reopen it if you run into any problems down the line.

6

Hi Euan,

I managed to repro, you can find the project attached. It’s quite a simple test. Here’s also a video. You’ll see that the right arm has a different pose when the left arm has 0 blend weight. It’s indeed a mesh space relative blend.

7

And the video

8

Hi Sergio, thanks for attaching the repro project. That made it easier to track down what’s going wrong here.

The problem is in how we’re accumulating the mesh space transforms for the two blend pose buffers. If you take a look in the non-ISPC code you’ll see that we do this:

			for (const FCompactPoseBoneIndex BoneIndex : BasePose.ForEachBoneIndex())
			{
				const int32 PoseIndex = BoneBlendWeights[BoneIndex.GetInt()].SourceIndex;
				const FCompactPoseBoneIndex ParentIndex = BoneContainer.GetParentBoneIndex(BoneIndex);
 
				if (ParentIndex != INDEX_NONE)
				{
					AccumulateMeshSpaceRotation(PoseIndex, BoneIndex, SourceRotations[ParentIndex], TargetRotations[ParentIndex]);
				}
				else
				{
					AccumulateMeshSpaceRotation(PoseIndex, BoneIndex, FQuat::Identity, FQuat::Identity);
				}

And looking at AccumulateMeshSpaceRotation, it does this:

	auto AccumulateMeshSpaceRotation = [&](int32 PoseIndex, FCompactPoseBoneIndex BoneIndex, const FQuat& ParentSourceRotation, const FQuat& ParentTargetRotation)
	{
		SourceRotations[BoneIndex] = ParentSourceRotation * BasePose[BoneIndex].GetRotation();
		TargetRotations[PoseIndex][BoneIndex] = ParentTargetRotation * BlendPoses[PoseIndex][BoneIndex].GetRotation();
	};

So although we have two blend pose buffers that need to be accumulated into mesh space, we are accumulating them into a single TargetRotations buffer, intermingling transforms from the two buffers. Obviously, that isn’t going to give valid results in mesh space since part of the pose will come from one buffer and part from the other.

The reason that you saw a difference in behaviour when setting the blend weight to 0 is that, like you already noticed, when that’s the case, we use the ref pose as that blend pose buffer. When that is accumulated in this way with bones from the other blend pose, we get a different result from when the weight is 0.1 as we’re then accumulating with different transforms.

To fix this, I think we need to accumulate both blend pose buffers into mesh space first, then use the relevant pose index for each bone to determine which target rotation we should be blending to. I have something working locally that does this, but it’s not particularly efficient. I’m going to try and talk with the dev team tomorrow to see if there is anything more sophisticated that we could be doing. Updating the ISPC code may also be a bit tricky.

I’ll update you again once I have more info.

9

Thanks Euan, seems to work fine from my tests.