Severe UE5 Memory leak BP visual debugging

Hyaniner · August 6, 2022, 1:54pm

Here is what I’ve been digging in this whole week.

Summary

If there is a container(TArray, TMap, Tset) with very many items (in my case, over 1,000~10,000) that visible in Blueprint,
or, if too many properties in Blueprint, specially with struct that have many properties
or, if Blueprint has a reference to Blueprint like that I said above,
if you use Blueprint debugging functionary,
hitch or freezing or crash of editor may happen, because of out of memory.
Because at every tick, Blueprint Debugger read every single Blueprint visible items of container and Blueprint visible properties in the classes and structs that has item set as watching value, not only properties set as watching value.
Updated: By applying meta keyword “DebugTreeLeaf” to UCLASS (in c++), It is possible to prevent spreading of those recursive function calling to read properties. This could not be complete solution, but it could make problem less harmful. I will comment about this at last chapter of this post.

Detail

If there is some property to display by Blueprint debugging functionary of the Unreal Engine, Those functions are called.
- FKismetDebugUtilities::GetDebugInfoInternal
- FPropertyInstanceInfo::FindOrMake
- FPropertyInstanceInfo::PopulateChildren
- And, FPropertyInstanceInfo::PopulateChildren call these
  - FPropertyInstanceInfo::FindOrMake Again,
  - And FPropertyInstanceInfo::PopulateChildren Again…
  - It happen recursively, for all Blueprint-visible properties referenced.
    - This includes each items in containers like TArray, TMap, TSet,
    - This includes each properties of UStruct, too.
If there is some reference between instances of BP or c++ class with blueprint visible property, this process could be spreading upon all related things.
This is called at every single tick.
- Because I wondered why this is not happened before, I checked UE 4.27.
  In 4.27, properties were displayed at only PIE is paused. It was not real time.

Example

I made small project to reproduce this:
GitHub - cutycutyhyaline/BPDebugReproduce

This is BP_Sub.

This has TMap with integer and struct HitResult, named “BP_Sub_TMap”.
At the event BeginPlay, it add 1000 items in this TMap.
- Why I choose HitResult is, this has many properties.
  I think it will be good example for this.
- If 1000 is too heavy or too light to your computer, adjust it.

This is BP_Main.

TestVarToWatch is just integer variable to example as Watching value.
Please remember I set this as watching.
At Event BeginPlay, BP_Main spawn a instance of BP_Sub, and store it in its reference variable, (named “RefSub”)
I will comment later about BP_the_Other.

Turn on Hitches and UnitGraph in viewport stat display.

And Turn on Data Flow tab, in the Blueprint Debugger.
You can this with “Window” menu in Blueprint debugger.
After that, run PIE. At now, everything is ok.

Expand BP_Main in Data Flow tab, of Blueprint Debugger.
Blueprint Debugger start to display the value of integer variable, set as watching value above.
With it, hitch is started.

This is FPropertyInstanceInfo::PopulateChildren, I commented in the Overview above.
Path is : Engine\Source\Editor\UnrealEd\Private\Kismet2\KismetDebugUtilities.cpp

It has branching based on if statement, by UObject, TArray, UStruct, TMap, and TSet.
I set the breakpoint at code which is checking TMap, line 2202.
And this breakpoint hit.
Please remember I expanded the instance of “BP_Main”, in Blueprint Debugger.
- But this breakpoint hit because “BP_Sub_TMap”.
- This is the property of BP_Sub, not property of “BP_Main”.

In call stack, this process is started with BP_Main, and,

RefSub, this is property of BP_Main, reference for BP_Sub,

And it reached BP_Sub_TMap, the property of BP_Sub.

And If you press the Continue button in the visual studio,
you can see the breakpoint is hit at every tick.

I think, “Ok, If I does not touch anything in the Data Flow tab of Blueprint Debugger UI, It will be ok.”. But it was wrong. Let’s see “BP_TheOther”

This is BP_TheOther. It is varibale to reference BP_Main, It can be set at spawn. Except this variable and static mesh of sphere to identify existance, there is nothing.

In Blueprint editor of BP_Main, after spawn, store the spawned BP_TheOther in another reference variable. And then, Set it as Watching Value, named “RefTheOther”

In Blueprint editor, select BP_Main as debug object instance.

If I run PIE, and keep “RefTheOther” node visible, you can see that editor become very slow.

Please remember I did not expand any item in Data Flow tab in Blueprint Debugger.

If you set breakpoint in FPropertyInstanceInfo::PopulateChildren , It will be hit.

Comments

This made me very scary at first. But after for a while, I thought that this might be necessary and might be needed and be intended if there is need to check properties of BP in real time.
If it is possible, it seems be needed the option that "Just show variables set as watching value, exclude instance “self” node of the Blueprint instances. Of course, “exclude” means exclude the process of iterating all properties of BP. In the Data Flow tab of the Blueprint Debugger UI, and Blueprint Editor, both.
To be honest, still I don’t know this is bug or intended behavior.
I’m developing turn based strategy game with hex grid. There are many BP classes with long length containers. And they referenced with each other very often. With this, I’m having a tough problem. These are options what I can think now:
- Just being careful when using Blueprint debugging functionality.
- Hide all those large containers against Blueprint. If I need to debug, use c++.
  - Wow. It hurts and painful. For some too large containers I did this, already. But in sometimes, I want to handle those on Blueprint for fast iteration.
- Time has come. Modify source code and make my own build of engine. (It hurts too)
- And else? Please advise me.
Note: To avoid infinity loop by circular reference, there is checking process in FKismetDebugUtilities::GetDebugInfoInternal.
- This process is based on local variable TMap<FPropertyInstanceInfo::FPropertyInstance, TSharedPtr> VisitedNodes in FKismetDebugUtilities::GetDebugInfoInternal.
- If property is in this TMap, the engine continue to next property. if there is not in TMap, Property is added to TMap.
- So this seems not about circular reference. Anyway, this means allocation in the TMap once for each properties. I’m suspecting this process makes the other memory overhead.
This issue seems based on same thing, but It could had more thing. This issue could be based on that recursively calling function through all Blueprint visible properties - specially items in container - what I said above. This could be also, about creating and destroying widget. In addition, I want to comment the steps to reproduce in that issue, must be corrected. “7. Add a Variable of type Color (Structure>Color) named ‘Colors’” In this line, ‘Colors’ must be TArray. Just single FColor variable can not have the length.

Thank you.

Added: meta keyword “DebugTreeLeaf”

FPropertyInstanceInfo::PopulateChildren has the codes :

if (UObject* ResolvedObject = Object.Get())
{
	if (ResolvedObject->GetClass()->HasMetaDataHierarchical("DebugTreeLeaf"))
	{
		return;
	}
}

So, by making a class like below, and changing the parent of Blueprint class, You can prevent to spreading of these recursive function calling chain between classes.

UCLASS(BlueprintType, Blueprintable, meta = (DebugTreeLeaf))
class REPRODUCEBASE_API ADebugTreeLeafActor : public AActor
{
	GENERATED_BODY()
	
public:	
...

But this is some kind of first aid. Not complete solution. It need c++, so this could not the solution of Blueprint, in principle. Furthermore, This way can not stop spreading of recursive function calling chains between structs and containers. If you debug directly some blueprint that has many properties, or blueprint that have container with many items, problem still happens.