Mutable loads all referencers of CustomizableObjects including blueprints, external actors in other levels, and sequences

We recently deployed Mutable for a project and encountered a massive slowdown in editor performance which we traced back to excessive loading originating from GetNodeGroupObjectNodeMappingImmersive. If we understand the logic here correctly, the system uses GetReferencers to gather all connected CustomizableObjects to ensure a root and its tree of COs are pulled in together. However, it uses an open FARFilter, limited only to the paths from the asset registry, rather than narrowing the filter to just UCustomizableObject. This can lead to any assets which contain a reference to a CO to be loaded, which we ran into after replacing actors in sequences with actors using COIs for appearances. We saw it loading sequences and even __ExternalActors__ assets which would spin up a WorldPartition level in the background. This led to a cascade of COs loading sequences loading other COs loading other sequences, until most of the game’s cinematics were loaded into memory, because we encountered one customizable object.

To fix this, we have inserted

if (!Filter.PackageNames.IsEmpty())
{
    Filter.ClassPaths.Add(UCustomizableObject::StaticClass()->GetClassPathName());
}

after the loop over ArrayReferenceNames, which filters out TempAutosave referencers. This leads to the ArrayAssetData only being filled with objects which can be cast to UCustomizableObject and filters out all unrelated assets, saving us from loading sequences and external actors in other levels, as well as any blueprints which may hold a reference. Note that it’s important we only add this class filter if the PackageNames has elements, else the filter will flip to gathering all COs in the project, which leads to a far worse problem as the function then tries to load all COs everywhere, potentially heading for stackoverflow as it recurses through every CO n-squared times. Can save you some time making that mistake.

[Attachment Removed]

Steps to Reproduce
Using MutableSample with 5.6 checked out from GitHub:

  • In Project Setting, set Load Level at Startup to None
  • Restart the editor to unload all COs

Add a breakpoint to the GetReferencers call in GetNodeGroupObjectNodeMappingImmersive (GraphTraversal.cpp)

  • In the Untitled empty start-up level, drag a CO_Character derived COI (e.g. COI_Cyborg) into the world, triggering the breakpoint
  • Step over GetReferences and inspect the newly filled ArrayReferenceNames array.

Note the presence of /Game/Infinite/Instance. This unrelated blueprint is then loaded and Cast to UCustomizableObject, before being discard because it fails to cast, being an unrelated type.

[Attachment Removed]

Hi Mark,

Thanks for reporting this issue. We did submit a similar fix to 5.8. I linked the GitHub commits, although I wouldn’t bother porting our fix if the one you have works as expected. We refactored the code to address multiple bugs altogether.

UE5 Main commits:

https://github.com/EpicGames/UnrealEngine/commit/f4dfca47f412fa86e9fe7f58ef7af2130606d80f (Base fix)

https://github.com/EpicGames/UnrealEngine/commit/6569e5fdb2c59a4146082b8aa28de05278ee94d0 (Fixes a hang we detected within a specific combination of nodes on 5.8).

Regards,

Pere

[Attachment Removed]