We’re hitting the instance limit in world partition and would like to do some debugging and find out what our worst offenders are. Is there a way to dump all of our instance counts out so we can find outliers and correct?
Thanks,
Brenden
Hi [mention removed],
You’re hitting the GPU Scene instance-ID limit that Unreal sets for all instanced primitives. This limit is defined in SceneDefinitions.h:
#define INSTANCE_ID_NUM_BITS (24u) // Max of 16,777,216 instances in the bufferThe warning is triggered inside the FGPUScene constructor, where Unreal compares the current number of allocated GPU Scene instance entries against that hard limit.
When the engine detects that the allocator (InstanceSceneDataAllocator) is approaching or exceeding the 16M cap, it logs the “instance overflow” warning.
if (MaxInstancesDuringPrevUpdate >= MAX_INSTANCE_ID)
{
OutMessages.Add(FCoreDelegates::EOnScreenMessageSeverity::Warning, FText::FromString(FString::Printf(TEXT("GPU-Scene Instance data overflow detected, reduce instance count to avoid rendering artifacts"))));
OutMessages.Add(FCoreDelegates::EOnScreenMessageSeverity::Warning, FText::FromString(FString::Printf(TEXT(" Max allocated ID %d, max instance buffer size: %dM"), MaxInstancesDuringPrevUpdate, MAX_INSTANCE_ID >> 20)));
if (!bLoggedInstanceOverflow )
{
UE_LOG(LogRenderer, Warning, TEXT("GPU-Scene Instance data overflow detected, reduce instance count to avoid rendering artifacts.\n")
TEXT(" Max allocated ID %d (%0.3fM), instance buffer size: %dM"), MaxInstancesDuringPrevUpdate, double(MaxInstancesDuringPrevUpdate) / (1024.0 * 1024.0), MAX_INSTANCE_ID >> 20);
bLoggedInstanceOverflow = true;
}
}
Extracting information like dumping all of the instances counts is because all the system that feed GPU Scene (ISM/HISM, foliage, PCG, Niagara, Nanite, etc.) funnel into private code, so access to it is not straight forward. It is not that accessible from game modules or Blueprints directly.
So extracting a full per-instance breakdown would require editing the engine, or writing instrumentation directly inside the renderer module.
GPUScene also has a some CVars we can actualy use but non of them seems like can help us with this. You can see these CVars inside GPUScene.cpp. At the most CSVProfile Start/Stop tells you the amount of instances that the GPU is working with, but not the actual instance clases. This tells you the total number of GPU Scene instance entries, which matches the allocator size, but it does not reveal which components or meshes contributed to it.
So CSV gives you the global instance usage, not the source of that usage.
The most straightforward methos available from public API is to interate over all actors/components, and find UInstancedStaticMeshComponent, UhierarchicalInstancedStaticMeshComponent and any other instanced types we want to keep track off. This approeach is useful but we are still lacking from some other instances the Engine manages.
struct FEntry
{
FString Name;
int32 Count = 0;
};
TArray<FEntry> Entries;
for (TObjectIterator<UInstancedStaticMeshComponent> It; It; ++It)
{
UInstancedStaticMeshComponent* HISM = *It;
if (!IsValid(HISM) || HISM->GetWorld() != World)
{
continue;
}
const int32 Count = HISM->GetInstanceCount();
if (Count <= 0)
{
continue;
}
FEntry& Entry = Entries.AddDefaulted_GetRef();
Entry.Name = HISM->GetPathName();
Entry.Count = Count;
}
I’ll continue exploring the renderer side to see whether there’s a clean hook we can use without modifying the engine. I’ll update as soon as possible.
Best,
Joan
Thanks,
We’re completely fine with engine changes, but in the interim your suggestion of iterating every ism and counting instances with additional info is a good one. We’ll add a console command that lets us do that the next time the scene blows up.
Brenden
Hi [mention removed],
With the following code you should get the Instances you have in your scene. This scripts prepares a CVar variable to call. You call it by writing r.GPUScene.DumpInstances to the console. If you write r.GPUScene.DumpInstances file, with the file word as an argument, it will get written to your Saved/Logs folder. You can change the path or the file name in the function if you prefer.
Overall, this function walks the renderer’s FScene, retrieves all FPrimitiveSceneInfo objects, looks up their instance scene data buffers, extracts the number of instances each primitive contributes to GPUScene, sorts them by descending instance count, and prints the results or dumps them to a text file. The name are extracted from the primitive itself. This is what ends up being filles in the GPUScene allocator warning you are getting. With this script you should be able to debug your project easily 
Create the following file, in my case I called it GPUSceneInstanceDump.cpp. You don’t need a .h in this case. So that it works straight ahead, add it inside the Engine/Source/Runtime/Renderer/Private/ folder. Adding it in other routes might need extra setup work.
#include "CoreMinimal.h"
#include "Engine/World.h"
#include "Engine/Engine.h"
#include "RendererPrivate.h"
#include "ScenePrivate.h"
#include "InstanceDataSceneProxy.h"
#include "Misc/FileHelper.h"
#include "Misc/Paths.h"
#if !UE_BUILD_SHIPPING
struct FGPUSceneInstanceDumpEntry
{
int32 NumInstances = 0;
const FPrimitiveSceneInfo* Primitive = nullptr;
};
static void DumpGPUSceneInstances_RenderThread(FScene* Scene, bool bWriteToFile)
{
if (!Scene)
{
UE_LOG(LogRenderer, Warning, TEXT("DumpGPUSceneInstances: Scene is null"));
return;
}
TArray<FGPUSceneInstanceDumpEntry, SceneRenderingAllocator> Entries;
Entries.Reserve(Scene->Primitives.Num());
int64 TotalInstances = 0;
const int32 NumPrims = Scene->Primitives.Num();
for (int32 Index = 0; Index < NumPrims; ++Index)
{
const FPrimitiveSceneInfo* PrimitiveSceneInfo = Scene->Primitives[Index];
if (!PrimitiveSceneInfo)
{
continue;
}
int32 NumInstances = 0;
//Get number of instances
if (const FInstanceSceneDataBuffers* Buffers = PrimitiveSceneInfo->GetInstanceSceneDataBuffers())
{
NumInstances = Buffers->GetNumInstances();
}
else
{
NumInstances = 1;
}
if (NumInstances <= 0)
{
continue;
}
FGPUSceneInstanceDumpEntry& Entry = Entries.AddDefaulted_GetRef();
Entry.NumInstances = NumInstances;
Entry.Primitive = PrimitiveSceneInfo;
TotalInstances += NumInstances;
}
// Sort descending by instance count
Entries.Sort([](const FGPUSceneInstanceDumpEntry& A, const FGPUSceneInstanceDumpEntry& B)
{
return A.NumInstances > B.NumInstances;
});
UE_LOG(LogRenderer, Warning, TEXT("=== GPUScene Instance Dump ==="));
UE_LOG(LogRenderer, Warning, TEXT(" Scene primitives: %d"), NumPrims);
UE_LOG(LogRenderer, Warning, TEXT(" Primitives with instances: %d"), Entries.Num());
UE_LOG(LogRenderer, Warning, TEXT(" Sum of primitive instance counts: %lld"), TotalInstances);
for (int32 i = 0; i < Entries.Num(); ++i)
{
const FGPUSceneInstanceDumpEntry& E = Entries[i];
FString Name = E.Primitive ? E.Primitive->GetFullnameForDebuggingOnly() : TEXT("No name set for this primitive");
UE_LOG(LogRenderer, Warning, TEXT("%4d: %8d instances - %s"), i, E.NumInstances, *Name);
}
UE_LOG(LogRenderer, Warning, TEXT("End GPUScene Instance Dump"));
if (bWriteToFile)
{
FString Output;
Output += TEXT("GPUScene Instance Dump\n");
Output += FString::Printf(TEXT("Scene primitives: %d\n"), NumPrims);
Output += FString::Printf(TEXT("Primitives with instances: %d\n"), Entries.Num());
Output += FString::Printf(TEXT("Sum of primitive instance counts: %lld\n\n"), TotalInstances);
for (int32 i = 0; i < Entries.Num(); ++i)
{
const FGPUSceneInstanceDumpEntry& E = Entries[i];
FString Name = E.Primitive ? E.Primitive->GetFullnameForDebuggingOnly() : TEXT("No name set for this primitive");
Output += FString::Printf(TEXT("%4d: %8d instances - %s\n"),
i, E.NumInstances, *Name);
}
const FString FileName = TEXT("GPUSceneInstanceDump.txt");
const FString FilePath = FPaths::Combine(FPaths::ProjectSavedDir(), TEXT("Logs"), FileName);
if (FFileHelper::SaveStringToFile(Output, *FilePath))
{
UE_LOG(LogRenderer, Warning, TEXT("GPUScene instance dump written to: %s"), *FilePath);
}
else
{
UE_LOG(LogRenderer, Warning, TEXT("Failed to write GPUScene instance dump to: %s"), *FilePath);
}
}
}
// Console command
static void DumpGPUSceneInstancesCmd(const TArray<FString>& Args, UWorld* World, FOutputDevice& Ar)
{
if (!World || !World->Scene)
{
UE_LOG(LogRenderer, Warning, TEXT("DumpGPUSceneInstances: No valid world or scene"));
return;
}
bool bWriteToFile = false;
if (Args.Num() > 0 && Args[0].Equals(TEXT("file"), ESearchCase::IgnoreCase))
{
bWriteToFile = true;
}
FSceneInterface* SceneInterface = World->Scene;
FScene* RenderScene = SceneInterface->GetRenderScene();
ENQUEUE_RENDER_COMMAND(DumpGPUSceneInstancesCmd)(
[RenderScene, bWriteToFile](FRHICommandListImmediate& RHICmdList)
{
DumpGPUSceneInstances_RenderThread(RenderScene, bWriteToFile);
}
);
}
//CVar
static FAutoConsoleCommand GPUSceneDumpCmd(
TEXT("r.GPUScene.DumpInstances"),
TEXT("Dump per-primitive GPUScene instance counts to the log. Adding arg 'file' will also write to a file in the Saved/Logs folder."),
FConsoleCommandWithWorldArgsAndOutputDeviceDelegate::CreateStatic(&DumpGPUSceneInstancesCmd)
);
#endif
Best,
Joan
Thanks, we managed to track it down to some level instances with painted grass. We painted the grass as foliage because there was about 3000 of these level instances and we didn’t want to have 3000 landscapes, however the expectation was that if the instances were distance culled that they wouldn’t be occupying this buffer. It seems we need to write our own level of culling here to ensure they don’t enter the buffer unless the player is near, our own sort of “distance culler”. Is it safe to assume that SetVisibility(false) on the prim would remove the instances counted against us here?
I don’t think hiding them will remove them from the GPUScene instance buffer. That buffer represents how many instance slots are currently allocated, not how many are actually visible. Hiding a primitive will stop it from rendering, but the instance slots remain allocated and still count toward the GPUScene limit.
If you truly want to keep those foliage instances out of the GPUScene buffer, they must not be created until the player/camera is close enough. Another option is to remove the instances dynamically from the component as the player moves, so the instance count goes down as they stream out.
With the script I posted earlier, you can quickly inspect the actual GPUScene instance count and verify whether your changes are having an impact with the instance count.
Thanks, I ran your script and it does indeed seem that setting visibility false clears the instances from the buffer.
Perfect! Nice to know it does
Will close the case for now Brenden, feel free to open the case at any moment if you see it appropriate.