Mass crowds using SkinnedMeshInstance leak GPU memory — every wave of spawned-and-killed entities permanently grows the animation track pool (5.8)

[Mike_McQueen]

Summary

Mass crowds rendered with SkinnedMeshInstance never release animation tracks when
entities are destroyed. The track pool grows with every wave of spawned-and-killed
entities for the lifetime of the world, and persistent GPU transform memory grows
with it, since SkinningSceneExtension sizes that storage from the track count.
Spawning 20 entities, killing them, then spawning 20 more takes the track count to
39 — with never more than 20 alive at once. Expected ~20.
Only affects dynamic (non-stationary) Mass crowds. The release path exists but is
gated on a tag that only UMassStationaryVisualizationTrait adds, and the dynamic
skinned path explicitly requires that tag to be absent.

What Type of Bug are you experiencing?

Gameplay

Steps to Reproduce

1. Create a Mass entity config asset with UMassMovableVisualizationTrait
   (the dynamic trait — NOT UMassStationaryVisualizationTrait).
2. On that trait, fill SkinnedMeshInstanceDesc.Meshes[0]:
   • Asset: any USkeletalMesh with “Optimize for Instancing” enabled in its
     LOD0 Build Settings (required, or the transform provider silently fails
     to build and the crowd renders in reference pose)
   • TransformProvider: a UAnimSequenceTransformProviderData with that same
     skeletal mesh and one looping AnimSequence
   • AnimationMinScreenSize: -1
3. Set Params.LODRepresentation[EMassLOD::High] = SkinnedMeshInstance.
4. Add FMassRepresentationAnimationFragment to the entity, e.g. by listing it
   in a UMassAssortedFragmentsTrait on the same config.
5. Add Config/DefaultMass.ini entries so the skinned processors run in your
   net mode:
   [/Script/MassRepresentation.MassUpdateInstancedSkinnedMeshProcessor]
   ExecutionFlags=7
   [/Script/MassRepresentation.MassConsumeInstancedSkinnedMeshAnimationProcessor]
   ExecutionFlags=7
6. Play. Spawn 20 entities close enough to the viewer that they enter the
   High LOD tier. Read GetUniqueAnimationCount() on the instanced skinned
   mesh component’s transform provider — it rises to 20.
7. Destroy all 20 entities (Context.Defer().DestroyEntities). Wait until the
   entity count reaches 0. The track count stays at 20.
8. WITHOUT reloading the level, spawn 20 more entities in the same world and
   let them reach the High tier again.
9. Read the track count again.
   IMPORTANT: step 8 must happen in the same world. A level reload rebuilds the
   Mass visualizer and resets the pool, which masks the bug entirely.

Expected Result

After step 9, the track count is approximately 20 — the same as after the first
wave.
FAnimSequenceTrackPool::AllocateTrack already reuses freed slots: it calls
ActiveBits.FindAndSetFirstZeroBit() and only appends to the Datas array when
every existing slot is still active. So if the first wave’s tracks had been
released when those entities were destroyed, the second wave would reuse those
same 20 slots and the count would not grow.
The count should track how many entities are concurrently rendered at High LOD,
not how many have ever been.

Observed Result

The track count is 39 — roughly double. The first wave’s tracks were never
released, and the second wave allocated 19 fresh ones on top of them.
Measured on 5.8, logging the count every 2 seconds:
spawn 20 → tracks: 0 → 16 → 18 → 20
(all 20 killed; entity count reaches 0; tracks stay at 20)
spawn 20 → tracks: 20 → 39 (stable at 39)
Concurrent entity count never exceeded 20 at any point.
Because AllocateTrack reuses freed slots before appending, growth past the
concurrent count is itself proof that nothing is being released — a pool with
working release could not reach 39 while only ever holding 20 live entities.
This repeats per wave. A third wave of 20 takes it to ~59. There is no ceiling
short of a level reload, so a mission spawning 400-entity waves accumulates
~400 tracks each time, and SkinningSceneExtension.cpp:1311 sizes persistent GPU
transform storage as UniqueAnimationCount * MaxTransformCount * 2 — so VRAM
grows with the cumulative total rather than with what is alive.

Affects Versions

5.8

Platform(s)

Windows

Additional Notes

WHY THIS AFFECTS OTHER PEOPLE
This is the configuration Epic recommends for crowd rendering. Epic staff
recommend “Mass + ISKM and AnimSequenceTransformProvider” here:

500×500 2.84 KB

What is the progress of the AnimBank and Instanced Skinned Mesh Component for version 5.8? Any tutorials about this? If we want to simulate the NPC crowds like those in Hitman, which approach is recommanded? VAT with ISM? Massive Plugin or AnimBank + ISKM? Thanks [Attachment Removed]

Anyone building a Mass crowd that spawns waves over time will hit this.
ANALYSIS (our reading of shipped 5.8 source — worth verifying)
The only DeallocateTrack call in MassRepresentation is in
UMassVisualizationComponent::ProcessRemovesForComponent
(MassVisualizationComponent.cpp:1151-1166), gated on GetEntitiesRequiringRemoval()
being non-empty (:1128).
The sole writer of that array is
FMassLODInstancedSkinnedMeshSignificanceRange::RemoveInstance (:1544). Its only
callers are MassStationaryISMSwitcherProcessor.cpp:127 and
MassStationaryISMRepresentationFragmentDestructor.cpp:91, both gated on
AddTagRequirement(EMassFragmentPresence::All).
That tag is added in exactly one place: MassStationaryVisualizationTrait.cpp:23.
And MassUpdateSkinnedMeshProcessor.cpp:53 requires it to be ABSENT — the dynamic
path explicitly excludes it. So on the dynamic path ProcessRemovesForComponent is
unreachable and DeallocateTrack is effectively dead code.
Also verified directly:

• bRequiresExternalInstanceIDTracking is set true in exactly one place,
  MassStationaryVisualizationTrait.cpp:20, so the alternate branch reaching
  ProcessRemovesForComponent is stationary-only as well.
• MassRepresentation has exactly two EMassObservedOperationFlags::Remove
  observers: the stationary destructor above, and
  UMassRepresentationFragmentDestructor (MassRepresentationProcessor.cpp:650),
  which only calls ReleaseAnyActorOrCancelAnySpawning and never touches skinned
  instances or tracks.
  FMassEntityHandle hashes over {Index, SerialNumber}, so a respawn is a new key
  and gets a fresh track — hence growth per wave rather than a ceiling.
  PLEASE DON’T “FIX” THE LOD-EXIT RETENTION
  Tracks are also retained when an entity merely leaves the High LOD tier — the
  sweep at MassVisualizationComponent.cpp:812-828 removes the instance and the
  SharedIdMap entry but not the track, and re-entry reuses it via the
  bAlreadyPresent path at :761-770 / :852-861.
  That looks like part of the same leak but appears deliberate and worth keeping:
  FAnimSequenceTrackAutoPlayData::Position is only re-pushed when the sequence
  index changes, so the retained track is what preserves an entity’s animation
  phase across LOD oscillation. Releasing on LOD exit would make any entity that
  dips out of High for one frame snap back to its authored phase on re-entry —
  visible popping across the crowd. The gap is the DESTRUCTION path specifically.
  SUGGESTED DIRECTION
  A non-stationary sibling of UMassStationaryISMRepresentationFragmentDestructor:
  an observer on FMassRepresentationFragment with
  EMassObservedOperationFlags::Remove and no static-tag requirement, routing dying
  entities into EntitiesRequiringRemoval via
  FMassInstancedSkinnedMeshInfo::RemoveInstance
  (MassSkinnedMeshRepresentationTypes.h:700-706). ProcessRemovesForComponent then
  already does the rest — deallocate, remove the SharedTrackMap entry, clean up
  ResolvedAnimStates.
  NO PROJECT-SIDE WORKAROUND
  DeallocateTrack is ENGINE_API and BlueprintCallable, but EntityHandleToTrackIdMap
  is protected with UMassVisualizationComponent as friend
  (MassSkinnedMeshRepresentationTypes.h:332-333, 370). Freeing a slot without
  clearing the map entry desynchronises them: the stale entry keeps returning the
  old id while FindAndSetFirstZeroBit hands that same slot to another entity — two
  entities sharing one animation track.
  NOT PLATFORM-SPECIFIC
  The defect is a Mass observer that never runs, in platform-independent gameplay
  code. No RHI or driver involvement. Tested on Windows/D3D12 only, but it should
  reproduce anywhere the dynamic skinned Mass path runs.
  POSSIBLY RELATED
  Has anyone tried the Instanced Skinned Mesh Component in Unreal Engine 5.6 ?
  reports a performance penalty when spawning larger numbers of skinned instanced
  meshes, attributed to “Post Tick Component Update”. If that cost scales with
  entities spawned over time rather than with live instances, it may be this bug
  seen from the outside.
  Mass Replicated Entities are not getting destroyed when we tried to upgrade to UE5.8
  is a different entity-teardown defect in 5.8 (replication processor), but
  suggests entity destruction in 5.8 Mass has more than one gap.