Nested Linked Anim Layers lead to multiple AnimInstance initializations

Development Character & Animation
, ,
1

Hello Epic Support team !

We recently noticed some hitches when spawning pawns around our project. We tracked down the problem to the linking phase of our AnimLayers. After further investigation and instrumenting a lot of what was going on around here, we found out that when linking an AnimInstance whose main AnimGraph contains a LinkedLayerNode linked to its own anim layer graph, Unreal initializes the anim layer graph twice :

  • First, through the linked AnimInstance’s InitializeAnimation() > FAnimInstanceProxy::InitializeRootNode() which leads to initializing the LinkedLayer node following the graph from the root node. This will make the linked layer node initialize its subgraph : the linked instance self layer.
  • And a second time at the end of the linked AnimInstance’s InitializeAnimation() when calling UAnimInstance::InitalizeGroupedLayers() > UAnimInstance::PerformLinkedLayerOverlayOperation() > InitializeAndCacheBonesForLinkedRoot(). Here, PerformLinkedLayerOverlayOperation() considers that the node should be linked to “self”, and therefore, initializes the graph (which has already been linked and initialized by the previous step)

This process can be repeated even further if the anim layer graph evaluates another anim graph which follows the same structure (through an LinkedAnimGraph node for example). This leads the deepest layer graph to be initialized thrice, causing several hitches when adding our pawns to the game.

Our animation / gameplay teams like to make the main graph call its layer anim graph as it allows to preview the result of the graph in the AnimBlueprintEditor easily (without requiring to layer the graph onto another AnimInstance) while also making it usable both as a linked anim layer and as a linked anim graph.

I’ve tried patching the engine to prevent these multiple initializations by making the LinkedAnimGraph & LinkedAnimLayer nodes track whether they have already initialized or uninitialized their subgraph and by checking this state when UAnimInstance::PerformLinkedLayerOverlayOperation() attempts to initialize it in order to avoid initializing an already initialized graph.

Following this investigation, I have a few questions :

  • Do you think this is an actual bug or has the issue more to do with the way we structure our ABPs ?
  • Are there some aspects that I may have overlooked that require these repeated initialization and cache-bones calls ?
  • Does the solution I’m implementing seem like a reasonable way to address to this problem ?

Thank you for your consideration,

Best regards,

Thomas

[Attachment Removed]

2

Steps to Reproduce
I’ve linked a repro project reproducing the asset structure that I described in the main ticket description :

  • BP_Pawn is the player pawn. It is a duplicate of the ThirdPerson pawn which simply links ABP_UpperBodyAdvanced onto its AnimBlueprint during BeginPlay.
  • ABP_UpperBodyAdvanced’s main graph holds a LinkedAnimLayer node to its “UpperBody” graph which contains a LinkedAnimGraph node linked to a ABP_UpperBodyBase instance.
  • ABP_UpperBodyBase’s main graph holds a LinkedAnimLayer node to its “UpperBody” graph which evaluates the CR_UpperBody ControlRig.
  • CR_UpperBody is a simple ControlRig that uses several “Import Skeleton” nodes in its Construction graph in order to make it easier to find inside Unreal Insights

How to reproduce :

  • Open the attached repro project
  • Load /Game/LVL_Main
  • Setup the insight trace by enabling : Stat Named Events and the Objects, Counters & Stats channels
  • Start the trace
  • Hit Play
  • Stop the trace
  • Open the recorded trace and look for “Actor BeginPlay” events
  • Under BP_Pawn’s BeginPlay event, we can see that ControlRig is executed three times, which is caused my the multiple initializations of the UpperBody graph.
    [Attachment Removed]
3

Hi,

Sorry for the delay getting back to you on this issue. We’re still in the process of catching up after Unreal Fest last week.

Unfortunately, this is a known issue with the linked layer/graph system within anim blueprints and it’s not trivial to fix in a general way that doesn’t have some kind of impact on edge cases. The linked anim graph/layer system didn’t originally support nesting in the way that you have setup, and when support was added for Fortnite, we didn’t do the major refactor that would have been required to prevent the need to potentially initialize/cache bones multiple times on the linked graphs.

The reason that we initialize and cache bones more than once on the linked graphs is because the usual flow via InitializeAnimation only touches nodes within the graph which are deemed relevant (ie. they have a non-zero weight). Because of that we force initialization and cache bones to run again on the subgraph which should then touch all nodes in the graph. Without this, you would get crashes or unexpected behaviour if some nodes in the subgraph weren’t relevant at link time but then became relevant later.

In terms of a fix for this, it’s a tricky problem because it is valid in certain circumstances for initialize or cache bones to be called multiple times before update is called on the graph (this is mostly via CMC for replaying moves in a networked setup). So if you were to add a gate on the linked anim graph/layer nodes to only recurse into the sub-graph once, you might break behaviour that relies on being able to call those functions multiple times.

One thing you could look at doing is gating execution of the control rig, as that’s the particularly expensive part of the graph by the looks of it. That would be a trivial change in FAnimNode_ControlRig::CacheBones_AnyThread:

void FAnimNode_ControlRig::CacheBones_AnyThread(const FAnimationCacheBonesContext& Context)
{
	DECLARE_SCOPE_HIERARCHICAL_COUNTER_FUNC()
 
// FIX START
	FBoneContainer& RequiredBones = Context.AnimInstanceProxy->GetRequiredBones();
	if (RequiredBonesSerialNumber == RequiredBones.GetSerialNumber())
	{
		return;
	}
// FIX END
 
	// make sure the inputs on the node are evaluated before propagating the inputs
	GetEvaluateGraphExposedInputs().Execute(Context);

Here, I’m caching the Required Bones serial number on the control rig anim node and only doing the cache bones work when that value has changed. And that value should only change when cache bones actually needs to run (ie. lod change, etc). This should be a pretty low-risk change. You could also look at doing something similar in the linked anim graph/layer nodes, but that would be a riskier change.

As a side note, I would also recommend disabling Tick Animation On Skeletal Mesh Init in the project settings, if you haven’t already, to prevent a game thread update of the anim graph when the meshes are registered.

Happy to discuss this further, or take a look at the changes that you mentioned you’d implemented if you want to share those. Just a heads-up, our offices are closed for the next couple of weeks, but if you want to discuss things more, I’ll follow up again as soon as we’re back.

Thanks,

Euan

[Attachment Removed]

4

Hello Euan,

Thank you for you answer ! I looked a bit into FAnimNode_ControlRig and FAnimNode_ControlRigBase and it seems that the ControlRigBase node is already gating the ControlRig execution similarily with its LastBonesSerialNumberForCacheBones. The only issue is that AnimInstance linking calls Initialize() before each CacheBones() call, clearing the cached serial number an making the node always run ControlRig execution.

Following your inputs I’ll try to implement my workaround a bit differently, by targetting the AnimInstance layer linking process specifically instead of trying to address the issue in the AnimNodes. In this context, I should be 100% sure that each subgraph should be initialized only once. This should allow me to gate everything properly while keeping the risk relatively low…

I’ll dig a bit further into this direction and I’ll keep you posted if I end up with something promising.

Best regards,

Thomas

PS: “Tick Animation On Skeletal Mesh Init” seems to be ON indeed, I’ll tell the project to set it off, thank you !

[Attachment Removed]

5

Hi Thomas,

Just following up on this since I’m back in office now. Let me know how you get on with the changes that you’re implementing and whether you run into any problems. This thread will stay open for a couple of weeks and then auto-close.

Also, just regarding the change to Tick Animation On Skeletal Mesh Init, when you make that change, you’ll need to hide the meshes on the frame that they spawn, since, without the animation update, they’ll spawn in ref-pose. That’s the only reason we haven’t turned this off by default.

Thanks,

Euan

[Attachment Removed]