How to properly update the PrimitiveLocalToWorld of FInstanceSceneDataBuffers and FInstancedStaticMeshSceneProxy?

Development Rendering
, ,
1

I’m trying to maintain a FInstancedStaticMeshSceneProxy without any actor component similar to FFastGeoInstancedStaticMeshComponent.

But I need to update the primitive transform of the InstancedStaticMeshSceneProxy instead of just static like the fast geo component.

However I couldn’t find a simple way to update the primitive transform of the proxy (All the instances’ datas like local transform are not changed).

​

I build the InstanceSceneDataBuffers just like how FFastGeoInstancedStaticMeshComponent::BuildInstanceData does

FInstanceSceneDataBuffers InstanceSceneDataBuffers{};

FInstanceSceneDataBuffers::FAccessTag AccessTag(PointerHash(this));
FInstanceSceneDataBuffers::FWriteView View = InstanceSceneDataBuffers.BeginWriteAccess(AccessTag);

// PrimitiveLocalToWorld
InstanceSceneDataBuffers.SetPrimitiveLocalToWorld(SourceComponent->GetComponentTransform().ToMatrixWithScale(), AccessTag);

// InstanceLocalBounds...
// LocalToPrimitiveRelativeWorld...
// InstanceCustomData...
// InstanceRandomIDs...

InstanceSceneDataBuffers.EndWriteAccess(AccessTag);
InstanceSceneDataBuffers.ValidateData();

NewDesc.SceneProxyDesc.InstanceDataSceneProxy = MakeShared<FInstanceDataSceneProxy, ESPMode::ThreadSafe>(MoveTemp(InstanceSceneDataBuffers));

Then I create the scene proxy and add primitive like this

auto& NewProxy = PLAProxy->Proxies[PLAProxy->Proxies.Add()];

Nanite::FMaterialAudit NaniteMaterials{};
const bool bUseNanite = SceneProxyDesc.ShouldCreateNaniteProxy(&NaniteMaterials);
if (bUseNanite)
{
    TRACE_CPUPROFILER_EVENT_SCOPE(CreateNaniteSceneProxy);
    NewProxy.PrimitiveSceneData.SceneProxy = ::new Nanite::FSceneProxy(NaniteMaterials, SceneProxyDesc);
}
else
{
    TRACE_CPUPROFILER_EVENT_SCOPE(CreateSceneProxy);
    if (SceneProxyDesc.bIsInstancedStaticMesh)
    {
       NewProxy.PrimitiveSceneData.SceneProxy = ::new FInstancedStaticMeshSceneProxy(SceneProxyDesc, SceneProxyDesc.FeatureLevel);
    }
    else
    {
       NewProxy.PrimitiveSceneData.SceneProxy = ::new FStaticMeshSceneProxy(SceneProxyDesc, false);
    }
}

FPrimitiveSceneDesc SceneDesc;
SceneDesc.SceneProxy = NewProxy.PrimitiveSceneData.SceneProxy;
SceneDesc.ProxyDesc = &SceneProxyDesc;
SceneDesc.PrimitiveSceneData = &NewProxy.PrimitiveSceneData;
SceneDesc.RenderMatrix = (InstanceData.RelativeTransform * PLATransform).ToMatrixWithScale();
SceneDesc.AttachmentRootPosition = PLATransform.GetTranslation();
SceneDesc.LocalBounds = SceneProxyDesc.bIsInstancedStaticMesh ? Desc.InstancesLocalBounds : SceneProxyDesc.StaticMesh->GetBounds();
SceneDesc.Bounds = SceneDesc.LocalBounds.TransformBy(SceneDesc.RenderMatrix);
SceneDesc.Mobility = EComponentMobility::Movable;

SceneProxyDesc.Scene->AddPrimitive(&SceneDesc);

It works great until I tried to update the primitive transform like this

    Scene->StartUpdatePrimitiveTransform(NumProxies);
...

    auto& InstanceData = PLAProxy->Components[Index];
    auto& Desc = PLAProxy->Descs[InstanceData.DescIndex];
    auto& SceneProxyDesc = Desc.SceneProxyDesc;

    FPrimitiveSceneDesc SceneDesc;
    SceneDesc.SceneProxy = Proxy.PrimitiveSceneData.SceneProxy;
    SceneDesc.ProxyDesc = &SceneProxyDesc;
    SceneDesc.PrimitiveSceneData = &Proxy.PrimitiveSceneData;
    SceneDesc.RenderMatrix = (InstanceData.RelativeTransform * PLATransform).ToMatrixWithScale();
    SceneDesc.AttachmentRootPosition = PLATransform.GetTranslation();
    SceneDesc.LocalBounds = SceneProxyDesc.StaticMesh->GetBounds();
    SceneDesc.Bounds = Desc.SceneProxyDesc.bIsInstancedStaticMesh ? Desc.InstancesLocalBounds : SceneDesc.LocalBounds.TransformBy(SceneDesc.RenderMatrix);
    SceneDesc.Mobility = EComponentMobility::Movable;

    SceneProxyDesc.Scene->UpdatePrimitiveTransform(&SceneDesc);
    SceneProxyDesc.Scene->UpdatePrimitiveInstances(&SceneDesc);
...
    Scene->FinishUpdatePrimitiveTransform();

It moves all the instances somehow, but I got an ensure of “Mismatched Primitive transform!”

And I can’t figure out a simple way to update the primitive transform inside the InstanceSceneDataBuffers.

[Image Removed]

[Attachment Removed]

2

I might figured it out, but I’m not sure is it the correct way.

I made a new class that inherit from the FInstanceDataSceneProxy and exposes a method to update the PrimitiveLocalToWorld

[Image Removed]

Then before send the command to update primitive transform, send a command to update the PrimitiveLocalToWorld first. [Image Removed]The ensures are gone, and the ISM is moving as expected.

[Attachment Removed]

3

The short answer is that nothing in your snippet for updating actually updates the instances. Since all the instance data is primitive relative it needs to be changed at that point. You can look at how FInstanceDataManager works or maybe use it directly to manage the instance data updates, it also contains machinery for doing the updates in an async task.

[Attachment Removed]

4

The key thing here is that the instance representation used in the renderer is relative to the primitive _location_, not local transforms as such. This allows a much faster path (on the GPU) that just adds the primitive offset (as opposed to needing to do full matrix concatenations), while maintaining precision. So when you update, this must be performed for every instance.

[Attachment Removed]

5

Also, I’d highly recommend using the FSceneInterface API for sending the updates rather than side-banding the data using an RT command as this ensures you are using the path that makes sure to trigger appropriate updates in various renderer systems.

[Attachment Removed]

6

Hi [Content removed] thanks for the replying!

Actually we don’t want to change any of the instance data, they just keep their local transform. Instead, we want to change the PrimitiveLocalToWorld which I just found a way to update it in the render thread but I’m not sure is it safe or not.

For the FInstanceDataManger, It is too complicated if we don’t need to track or change any instance data, also I see the comments said “Still somewhat tied to the UComponent”, but we want to eliminate the use of UComponent.

[Attachment Removed]