HISM being super tricky to remove. How do i remove them properly with a delay.

[SophiaWolfie]

Made my class HISM_Cubes that inherit from HierarchicalInstancedStaticMeshComponent.

Im trying to make it stop when it arrives closer to the Destination, and then 4 seconds later make remove the instance (and its struct). The struct is stored in an array, thats where i store the data related to the cube, like Speed, and other things.

Though so far its super buggy and i cant understand exactly why.
Im pretty sure im doing it all correctly as i worked in similar stuff in the past.
But here, it has been a full day of work stuck on this.
It doesnt help that the HISM is super tricky to remove, because when it removes it does so by swapping the indexes. Which is a PITA sometimes.

So if i remove instance 5 of 10 instances, it will send instance 5 to 10 and instance 10 to 5, and then remove the last index instance.
Could you tell me what im doing wrong here or if there is a better simpler way of doing this?

On the DestroyCube():

The way i remove it is by passing the Struct. Then get the instance to remove. Make sure the Last struct in the array is updated with that instance now, because im using RemoveSingleSwap.

So far it seems to be able to sometimes remove the instances, but at times it “forgets/skips” to remove an index of the array cubeStruct_Arr.

Do the same for the instance (im swapping the transforms, and then remove the last).

Full code here:

GitHub - LastIberianLynx/HISMRemoveDelay: Temporary - Test HISM Destroy issue.

#include "HISM_Cubes.h"

UHISM_Cubes::UHISM_Cubes() {

	PrimaryComponentTick.bCanEverTick = true;


}
void UHISM_Cubes::MoveCubes(float DeltaTime) {

	for (int32 Index = cubeStruct_Arr.Num() - 1; Index >= 0; --Index)
	{
		Fcube2& cubeStruct = cubeStruct_Arr[Index];
		if (cubeStruct.bMarkedForRemoval)
			continue;
		int32 InstanceIndex = cubeStruct.instance;
		FTransform TCur;
		GetInstanceTransform(InstanceIndex, TCur, true);

		FVector NewLocation = FMath::VInterpConstantTo(TCur.GetLocation(), Destination, DeltaTime, cubeStruct.speed);
		TCur.SetTranslation(NewLocation);
		UpdateInstanceTransform(InstanceIndex, TCur, true);

		if (TCur.GetLocation().Equals(Destination, 100.f)) {
			cubeStruct.bMarkedForRemoval = true;
			FTimerHandle TimerDestroy;
			GetWorld()->GetTimerManager().SetTimer(TimerDestroy, [this, &cubeStruct]() {
				DestroyCube(cubeStruct);
			}, 4.f, false);

			return;
		}


	}
}

void UHISM_Cubes::TickComponent(float DeltaTime, ELevelTick TickType, FActorComponentTickFunction* ThisTickFunction)
{
	Super::TickComponent(DeltaTime, TickType, ThisTickFunction);
	MoveCubes(DeltaTime);
	MarkRenderStateDirty();

}

void UHISM_Cubes::DestroyCube(UPARAM(ref) Fcube2& cubeStruct) {

	if (cubeStruct_Arr.Num() < 2) {
		cubeStruct_Arr.Empty();
		ClearInstances();
		return;
	}

	int32 LastcubeIndex = cubeStruct_Arr.Num() - 1;
	int32 LastInstanceIndex = GetInstanceCount() - 1;

	Fcube2* Lastcube = &cubeStruct_Arr[LastcubeIndex];
	Lastcube->instance = cubeStruct.instance;

	FTransform T;
	GetInstanceTransform(LastInstanceIndex, T, true);
	UpdateInstanceTransform(cubeStruct.instance, T, true);

	RemoveInstance(LastInstanceIndex);
	cubeStruct_Arr.RemoveSingleSwap(cubeStruct);

}

[Everynone]

Instead of removing the instance(s), move them where you cannot see / interacted with them - sweep them under the carpet - the index will not change since we merely update a transform.

Once you’re really ready to remove them, batch remove them - there is a function that takes an array. This should minimise the index jumbling - but it will, of course, not eliminate it. If you have dependent data structures, that container will need rebuilding.

[SophiaWolfie]

Thanks, i wish i could do it that way but i cant.
One thing i tried in another project was to Scale the instances down to 0,0,0.
And then remove all the instances at once at the end of the level.
But this doesnt work well if you are in a battle game with many units firing arrows for example.
They will accumulate in the hundreds of thousand.
So the best solution is to remove them as you go.
But it gets even more tricky, when im moving the arrows using a separate struct, where i store speed, color, and other things for each instance.
So when you remove an instance you must also remove the struct that is stored in a separate array.
And this is even more tricky.
Because when you remove an instance, all the other structs instance reference suddenly becomes invalid, because it changes the index of all instances.
I put a lot of time in this, and so far im not being able to make a working solution for it.
It always bugs no matter what i do.

[Everynone]

@SophiaWolfie

So when you remove an instance you must also remove the struct that is stored in a separate array.

That’s what I meant when I mentioned rebuilding a container, yup.

@SophiaWolfie

So the best solution is to remove them as you go.

Agreed, but that’s where HISM suck, big time, too.

@SophiaWolfie

They will accumulate in the hundreds of thousand.

Could you pool them?

• do the scale trick, as you do
• update the data struct, flag it as inactive
• next time you need an arrow, grab one from the inactive pool
• every now and then you’ll need to expand / trim the pool if memory footprint is of concern.

Could that work?

[3dRaven]

@SophiaWolfie

But this doesnt work well if you are in a battle game with many units firing arrows for example.
They will accumulate in the hundreds of thousand.

Wouldn’t the arrows be mesh particles like the old project I sent you?

[sergio.gardeazabal]

Hi,

As you found when you remove an instance in a ISMC (same for the HISM) the instance index will change, therefore you can’t cache the Instance index to refer to a specific instance later on.

There is a static global delegate you can bind to, that when any ISMC is about to update an instance index, you get the old index and the new one. With that you should be able to update any cached index you have.

FInstancedStaticMeshDelegates::OnInstanceIndexUpdated

There are probably better examples in the engine for runtime applications, but that is what I use for the Chaos Visual Debugger tool.

In CVD (for short) although it is an editor tool, depending on the map being debugged we have to render tens of thousands of meshes to represent collision shapes, and in the live debugging mode we have to do it at the same frame time the game is running. As most of these shapes are the same but with different transforms we use ISMC heavily.

Similar to what you do, we have a struct instance that the rest of the system uses to be able to access the specific mesh instance in the ISMC, but we keep the cached instance index up to date by binding to FInstancedStaticMeshDelegates::OnInstanceIndexUpdated.

Here is a place where that is handled in CVD.

https://github.com/EpicGames/UnrealEngine/blob/ue5-main/Engine/Plugins/ChaosVD/Source/ChaosVD/Private/Components/ChaosVDInstancedStaticMeshComponent.cpp#L261

Something worth mentioning: You should bind to that event in a manager object (or something that is instantiated only once), not the ISMC . The callback you get gives you the correct component where the update is happening.

The events you care about are

FInstancedStaticMeshDelegates::EInstanceIndexUpdateType::Relocated
FInstancedStaticMeshDelegates::EInstanceIndexUpdateType::Removed

In Relocated, you will get the old index (the one you have cached in your struct) and the new one (the one you need to update to).

That said, I think the trick of setting the instance transform scale to 0 is good and something we use in CVD as well, as that is usually faster than doing the removal.

You will need to find the balance between hiding them and fully remove them performance wise.

Implementing some kind of pool like @Everynone suggest to achieve that is a good idea.

[SophiaWolfie]

Thanks. I didnt know about that function, will look into it because i came accross this issue several times.
I used to just scale it to 0,0,0. But in certain situations, then can just accumulate instances infinitely.
In my case, i was making a rts with arrows being shot by squads. Like each squad would shoot 300 arrows. In a minute you would have thousands of arrows scaled to 0.

Makes you think that ideally we should at some point be able to remove these 0,0,0 arrows, right?
Meanwhile i came up with the solution that was to create a map and tie the index of the struct in the array to an FGuid.
This was super confusing but it works. Took me like 2 days of trial and error:

1-
As you create a new instance, you create the struct for the instance where you store stuff about the instance like Destination or in my case checkpoints.

		FGuid UniqueID = FGuid::NewGuid();
		ArrowCheckpointsStruct.ArrayIndex = ArrowCheckpointsStruct_Arr.Num();
		ArrowCheckpointsStruct.UniqueID = UniqueID;

		ID_Index_Map.Add(UniqueID, ArrowCheckpointsStruct.ArrayIndex);
		ArrowCheckpointsStruct_Arr.Add(ArrowCheckpointsStruct);

So when you are about to remove it, you send in the UniqueID of the Index you want to remove.

			FGuid UniqueID = ArrowCheckpointsStruct.UniqueID; //UniqueID of the instance to remove
			FTimerHandle TimerDestroy;
			GetWorld()->GetTimerManager().SetTimer(TimerDestroy, [this, UniqueID]() {

				DestroyArrowByID(UniqueID);

			}, 4.f, false);

Since you are destroying this arrow in 4 seconds, then it means that other arrows that are being destroyed or will be destroyed meanwhile, can change their uniqueID before this one is even destroyed. And that can cause arrows to swap wrongly. Super confusing.

So then when removing, you update the UniqueID to its corresponding Index of the struct in the array, after you swap it, and remove last:

void UHISM_Arrows::DestroyArrowByID(FGuid ID) {

	if (ArrowCheckpointsStruct_Arr.Num() < 2) {
//if its 1 instance left, then just remove them all.
		ArrowCheckpointsStruct_Arr.Empty();
		ID_Index_Map.Empty();
		ClearInstances();
		return;
	}

	int32 RemoveIndex = *ID_Index_Map.Find(ID);
	FArrowCheckpoints2& arrowStructBeingRemoved = ArrowCheckpointsStruct_Arr[RemoveIndex];

	int32 LastArrowIndex = ArrowCheckpointsStruct_Arr.Num() - 1;
	int32 LastInstanceIndex = GetInstanceCount() - 1;

	if (RemoveIndex == LastArrowIndex) {

		RemoveInstance(LastInstanceIndex);
		ArrowCheckpointsStruct_Arr.RemoveAt(arrowStructBeingRemoved.instance);
		ID_Index_Map.Remove(arrowStructBeingRemoved.UniqueID);
		return;
	}

	FArrowCheckpoints2* LastArrow = &ArrowCheckpointsStruct_Arr[LastArrowIndex];
	int32 ArrowInstanceToRemove = arrowStructBeingRemoved.instance;

	// Instead of reassigning the reference, update the array directly
	ID_Index_Map.Remove(arrowStructBeingRemoved.UniqueID);
	ID_Index_Map.Add(LastArrow->UniqueID, arrowStructBeingRemoved.ArrayIndex);

	ArrowCheckpointsStruct_Arr[LastArrowIndex].ArrayIndex = arrowStructBeingRemoved.ArrayIndex;
	ArrowCheckpointsStruct_Arr[LastArrowIndex].instance = arrowStructBeingRemoved.instance;
	ArrowCheckpointsStruct_Arr[arrowStructBeingRemoved.ArrayIndex] = ArrowCheckpointsStruct_Arr[LastArrowIndex];

	FTransform T;
	GetInstanceTransform(LastInstanceIndex, T, true);


	UpdateInstanceTransform(ArrowInstanceToRemove, T, true);


	RemoveInstance(LastInstanceIndex);
	ArrowCheckpointsStruct_Arr.RemoveAt(LastArrowIndex);

Its quite confusing what im doing in this last step, but it works. Im basically getting the Arrow instance that is being removed, and the last index in the struct, swap them, remove the last, update the struct to the new real corresponding instance and ArrayIndex, and updating the TMap that ties ID to Index.
Notice im swaping the transform, and removing last.

[SophiaWolfie]

Also:
I couldn’t find the function you mentioned:

Im in UE4.27.2 . Is it a UE5 only function or something?
Thanks.

[3dRaven]

It’s in InstancedStaticMeshDelegates.h which is not present in UE4

[sergio.gardeazabal]

Yes, sorry, the delegate I mentioned was introduced in UE5 as @3dRaven mentioned