Implementing mesh paint textures on instanced static mesh component

Hi I was following the thread here: [Content removed]

Since the question is already closed, I feel in need to re open the conversation,

I am trying to implement that support for our custom engine, all the relevant code is vanilla 5.6,

So far I did the easy part, I created an array of virtual textures like the post suggests as well as the array of descriptors, I am reading what would it be the best way to pass that data as per instance data.

On the GPUScene I found out there is a big structured buffer with float4s that is used to pass payload data:

Inside the cpp there is this function void FGPUScene::UploadGeneral(

Which I see is the one in charge of populate the structured buffer that ultimately lands on the struct FInstanceSceneData of the SceneData.ush

Inside that function there is the the function FORCEINLINE_GPUSCENE void GetInstanceInfo(int32 ItemIndex, FInstanceUploadInfo& InstanceUploadInfo) const from where we retrieve all the information that we are planning to use, there I can see I can access the FPrimitiveSceneProxy,

For now I am just reading and exploring the code, I think I have 3 options here:

  • Modify the layout of the struct and implement a new entry that seems terribly difficult
  • Use the custom data, this is an option but we are already using it for other purposes
  • I see there is a payload extension, where we could potentially have an easy access on the UInstancedStaticMeshComponent::GetOrCreateInstanceDataSceneProxy to modify that data and pass it to the shader.

I am just looking for some guidance for the implementation, I will keep reading for now.

[Attachment Removed]

Steps to Reproduce

[Attachment Removed]

Hi Jorge,

I think you have summed up the options very well.

Using custom instance data is possibly the easiest thing to do on the code side if you have a constrained use case. But it does place restrictions on the content side if you are already using custom instance data for other things in your materials.

If I were implementing this, my first take would be to use the InstancePayloadExtension mechanism. Unfortunately there is not a lot of use cases to learn from. But the implementation in the TSplineMeshSceneProxyCommon scene proxy which uses FSplineMeshSceneInstanceDataBuffers looks like a good thing to start from. That is used for a specific component/proxy type which you might not want (you may want this all to be generally available in all ISMs). But you could look at how it is implemented and move the logic to the base FInstanceSceneDataBuffers.

Best regards,

Jeremy

[Attachment Removed]

hey Jeremy thanks for the info.

I am a bit stuck with this, see if you can help, I went of the rute of using the InstancePayloadExtension, I think I got almost everything correctly but I can’t manage to make it work, pretty sure I am missing something, lets go through my changes:

First I added the following fields into the InstancedStaticmeshComponent.h:

UPROPERTY(VisibleAnywhere, Category = "Mesh Painting")
TArray<TObjectPtr<UTexture>> InstancedMeshPaintTextures;
 
UPROPERTY(SkipSerialization)
TArray<FVector4f> PerInstanceSMPayloadData;
 
UFUNCTION(BlueprintCallable, Category="Mesh Painting")
ENGINE_API void SetInstanceVirtualTextures(const TArray<UTexture*>& InInstanceTextures);
 
ENGINE_API virtual void GetUsedTextures(TArray<UTexture*>& OutTextures, EMaterialQualityLevel::Type QualityLevel) override;

On the cpp I added my own function to the ChangeSet, right after setting the CustomData, inside the BuildInstanceDataDelatChangeSetCommon function like this:

ChangeSet.SetPayloadExtensionData(MakeArrayView(PerInstanceSMPayloadData));

Then on the BuildComponentInstanceDaata I am enabling the bHasPerInstancePayloadExtension flag is we have a texture into our array:

OutData.Flags.bHasPerInstancePayloadExtension = !InstancedMeshPaintTextures.IsEmpty();

My function to fill up the descriptors is very basic:

void UInstancedStaticMeshComponent::SetInstanceVirtualTextures(const TArray<UTexture*>& InInstanceTextures)
{
	if (InInstanceTextures.IsEmpty())
	{
		return;
	}
 
	InstancedMeshPaintTextures = InInstanceTextures;
	TArray<FTextureResource*> InstancedTextureResources;
	InstancedTextureResources.Reserve(GetNumInstances());
	for (int32 i = 0; i < GetNumInstances(); i++)
	{
		if (InstancedMeshPaintTextures.IsValidIndex(i) &&
			InstancedMeshPaintTextures[i] &&
			InstancedMeshPaintTextures[i]->GetResource() &&
			InstancedMeshPaintTextures[i]->IsVirtualTexturingEnabled())
		{
			InstancedTextureResources.Add(InstancedMeshPaintTextures[i]->GetResource());
			
			UE_LOG(LogStaticMesh, Verbose, TEXT("Assigning virtual texture: %s to instance: %i"), *InstancedMeshPaintTextures[i]->GetFName().ToString(), i);
			continue;
		}
		
		InstancedTextureResources.Add(nullptr);
	}
 
	int32 MeshPaintIndex = GetMeshPaintTextureCoordinateIndex();
	ENQUEUE_RENDER_COMMAND(FInstancedStaticMeshComponentUpdatePayloadData)(
		[WeakThis = MakeWeakObjectPtr(this), InstancedTextureResources, MeshPaintIndex](FRHICommandListImmediate&)
		{
			TArray<FVector4f> DescriptorsAsPayLoad;
			DescriptorsAsPayLoad.Reserve(InstancedTextureResources.Num());
 
			for (FTextureResource* Resource : InstancedTextureResources)
			{
				FUintVector2 VirtualTextureDescriptor = MeshPaintVirtualTexture::GetTextureDescriptor(Resource, MeshPaintIndex);
				DescriptorsAsPayLoad.Add(FVector4f(
					static_cast<float>(VirtualTextureDescriptor.X),
					static_cast<float>(VirtualTextureDescriptor.Y),
					0.0f,
					0.0));
				UE_LOG(LogTemp, Log, TEXT("Size: %f , %f"), static_cast<float>(VirtualTextureDescriptor.X), static_cast<float>(VirtualTextureDescriptor.Y));
			}
 
			AsyncTask(ENamedThreads::GameThread, [WeakThis, DescriptorsAsPayLoad]()
			{
				WeakThis->PerInstanceSMPayloadData = DescriptorsAsPayLoad;
				WeakThis->MarkRenderStateDirty();
			});
		});
}
 
void UInstancedStaticMeshComponent::GetUsedTextures(TArray<UTexture*>& OutTextures, EMaterialQualityLevel::Type QualityLevel)
{
	Super::GetUsedTextures(OutTextures, QualityLevel);
	for(UTexture *InstancedMeshPaintTexture : InstancedMeshPaintTextures)
	{
		if(InstancedMeshPaintTexture && InstancedMeshPaintTexture->IsVirtualTexturingEnabled())
		{
			OutTextures.AddUnique(InstancedMeshPaintTexture);
		}
	}
}

Then on the FISMInstanceUpdateChangeSet class I added my own function to pass the data holded on the component:

void FISMInstanceUpdateChangeSet::SetPayloadExtensionData(const TArrayView<const FVector4f>& InPerInstancePayloadData)
{
	GetPayloadExtensionDataWriter().Gather(InPerInstancePayloadData);
}
 
 

On the ISMInstanceDataManager.cpp I added the extra float necessary to the payload data stride:

int32 NumPayLoadExtensionFloat4S = 0;
 
if (ChangeSet.Flags.bHasPerInstancePayloadExtension)
{
NumPayLoadExtensionFloat4S++;
}
 
InstanceDataBufferHeader.PayloadDataStride = FInstanceSceneDataBuffers::CalcPayloadDataStride(ChangeSet.Flags, NumCustomDataFloats, NumPayLoadExtensionFloat4S);

Once the data I created a custom hlsl node inside the material editor just to verify the data is correctly hooked and I am using the debug floatx node, the info seems to be right, here is the custom hlsl code:

#if IS_NANITE_PASS
 
FInstanceSceneData InstanceData = GetInstanceSceneData(Parameters.InstanceId);
float4 PayloadExtensionData = LoadInstancePayloadExtensionElement(InstanceData, 0);
return PayloadExtensionData.xyz;
#else
return 0;
#endif

The last step does not seem to be working, I did modify the HLSLMaterialTranslator.cpp to use my custom hlsl functions on the MaterialTemplate.ush:

uint2 ResolveMeshPaintTextureDescriptor(FMaterialPixelParameters Parameters)
{
#if USE_INSTANCING && IS_NANITE_PASS
return GetInstancedMeshPaintTextureDescriptor(GetInstanceSceneData(Parameters.InstanceId));
#else
return GetMeshPaintTextureDescriptor(GetPrimitiveData(Parameters));
#endif
}
 
uint2 GetInstancedMeshPaintTextureDescriptor(FInstanceSceneData InstanceData)
{
 float4 PayloadExtensionData = LoadInstancePayloadExtensionElement(InstanceData, 0);
 return uint2(asuint(PayloadExtensionData.x), asuint(PayloadExtensionData.y));
}

Not sure what I am doing wrong at this point, a little orientation will be great

[Attachment Removed]

Hi Jorge,

The general flow of things seems reasonable. One thing that stands out as slightly odd is the double enqueue to render and then game thread in SetInstanceVirtualTextures(). I think I would expect that the conversion from texture resource to descriptor would happen inline when filling the ChangeSet.

Without direct access to the code it’s difficult to know exactly what the problem is. You’re probably already doing this, but I would try looking at a RenderDoc capture to validate if the expected data is in the GPUSceneInstancePayloadData buffer, and work backwards from there.

Best regards,

Jeremy

[Attachment Removed]

Hey Jeremy, thanks for the hit, I checked the data inside RenderDoc, it turns out I was packing wrong the descriptor, so I modify my function to pack the first int into x and y and the second int to be inside z and w:

FUintVector2 VirtualTextureDescriptor = MeshPaintVirtualTexture::GetTextureDescriptor(Resource, MeshPaintIndex);
check(VirtualTextureDescriptor.X + VirtualTextureDescriptor.Y != 0u);
float x = VirtualTextureDescriptor.X & 0xffff;
float y = (VirtualTextureDescriptor.X >> 16) & 0xffff;
 
float z = VirtualTextureDescriptor.Y & 0xffff;
float w = (VirtualTextureDescriptor.Y >> 16) & 0xffff;
						
DescriptorsAsPayLoad.Add(FVector4f(x, y, z, w));

Also a unpack version on HLSL:

uint2 UnpackPayloadExtensionData(float4 ExtensionData)
{
	const uint u1 = (uint(ExtensionData.y) << 16) | uint(ExtensionData.x);
	const uint u2 = (uint(ExtensionData.w) << 16) | uint(ExtensionData.z);
	return uint2(u1, u2);
}

[Image Removed]

I have 2 issues still, 1 is I dont know why some index is messed up, in the picture i just index 0, but it is actually random, sometimes is 2 or 3, and secondly, when setting the values the material does not refresh and I have absolutely no clue why, I have to open the material in the material editor and recompile haha,

Once the data is set, I am doing this:

AsyncTask(ENamedThreads::GameThread, [WeakThis, DescriptorsAsPayLoad]()
			{
				if (!WeakThis.IsValid())
				{
					return;
				}
				WeakThis->PerInstanceSMPayloadData = DescriptorsAsPayLoad;
				WeakThis->PrimitiveInstanceDataManager.PayloadExtensionDataChanged();
				WeakThis->MarkRenderStateDirty();
			});

The function you see there follows the pattern of the custom data one, not sure if it is enough:

void FPrimitiveInstanceDataManager::PayloadExtensionDataChanged()
{
	if (GetState() == ETrackingState::Disabled)
	{
		return;
	}
 
	LOG_INST_DATA(TEXT("PayloadExtensionDataChanged(%s)"), TEXT(""));
	bPayloadExtensionDataChanged = true;
	MarkComponentRenderInstancesDirty();
}

I am reading how the custom data is set, but basically they just do the same, mark the render state dirty, any other clue you can give me will be appreciated, I feel like I am close

[Attachment Removed]

I changed to PIX, boy much better than RendorDoc, so I found out that after running my logic, the buffer is filled with new data, but the FInstanceSceneData does not, so when calling:

FInstanceSceneData InstanceData = GetInstanceSceneData(Parameters.InstanceId);

the PayloadExtensionOffset is still undefined since it is -1 INDEX_NONE, that is bad, tracking how it changes I did this custom function:

void FPrimitiveInstanceDataManager::PayloadExtensionDataChanged(int32 InstanceIndex)
{
	LOG_INST_DATA(TEXT("PayloadExtensionDataChanged(IDX: %d)"), InstanceIndex);
	MarkChangeHelper<EChangeFlag::PayloadExtensionDataChanged>(InstanceIndex);
}

Also added a new entry for the EChangeFlag:

enum class EFlag : ElementType
	{
		Added, /** Set when */
		TransformChanged,
		CustomDataChanged, 
		PayloadExtensionDataChanged, // New entry
		IndexChanged, /** Implicitly set when calling Remove or RemoveAt as this causes movement to fill holes. */
		Num,
	};

Does not seem enough though, not sure what I am missing still

[Attachment Removed]

Well after some more researchI found it has something to do with the material not been refreshed, I looked into the UMaterialEditingLibrary::RecompileMaterial

and just by adding FMaterialUpdateContext UpdateContext; into my code fixed my problem it seem the GetRendererModule().UpdateStaticDrawListsForMaterials(MaterialResourcesToUpdate); called at the destructor forces the refreshing. Cool. now my only remaining problem is why the physiclal texture does no load some textures some time, my array looks like this:

[Image Removed]

In this case it did not load the 3 element, (green noise)

[Image Removed]

Looking into PIX, yeah the Scene_MeshPaint_PhysicalTedxture does not contain any info from that texture at all, maybe a little tiny to see, but trust me the green noise texture is nowhere to be seen:

[Image Removed]How can I tell to UE to load the VT data required, I am looking at the VT feedback I have the feeling the first problem is the same as the second, the GPU feedback must be working with primitive instead of instances, but not sure where is that code, I will keep looking at it, thanks for the support

Edit: definitely something wrong on the feedback loop, I am getting into this line of code:

const FAllocatedVirtualTexture* RESTRICT AllocatedVT = Space->GetAllocator().Find(vAddress);
		if (!AllocatedVT)
		{
			if (CVarVTVerbose.GetValueOnAnyThread())
			{
				UE_LOG(LogConsoleResponse, Display, TEXT("Space %i, vAddr %i[Content removed] vLevel);
			}

[Attachment Removed]

Hi Jorge,

I just wanted to check in here. Did you get this working?

Best regards,

Jeremy

[Attachment Removed]

Hi Jeremy

Not yet, I got it almost working, I am missing the feedback loading, it seems my descriptors are fine, but the feedback does not load the pages necessary when rendering the material, I tried to PIX everything and understand the code, but I could not find any solution

Regards

Jorge CR

[Attachment Removed]

Hi Jorge,

The virtual texture feedback is generated by the material shader when sampling the virtual textures. It uses the data in the descriptors to evaluate where in the virtual texture page table to sample. It then does the sample itself, and writes the requested sampling location back to the feedback buffer.

If you are hitting "Space %i, vAddr %i[Content removed] and _probably_ the associated virtual texture sample will be bad too.

The most likely cause for this is that one of your descriptors is bad, or that you are somehow reading data that isn’t in your descriptor array. It is worth double checking that.

At the location of that log in FVirtualTextureSystem::GatherRequestsTask() the values of ID, vLevel, vPageX and vPageY (which are all derived from PageEncoded) may give some hint as to what is going on. The value of ID indicates the virtual texture space for the feedback item. It should be the same for all feedback items corresponding to a mesh paint virtual texture. If you are seeing variation in that value it indicates some bad descriptor.

Best regards,

Jeremy

[Attachment Removed]

Hey Jeremy thanks for the anwser, we are going back to the issue this week, I believe there must be something wrong then with the packing I am doing, the payload data structured buffer is a Vector4, and the descriptor of the VTs is a uint2, so I was trying to do this:

float x = VirtualTextureDescriptor.X & 0xffff;

float y = (VirtualTextureDescriptor.X >> 16) & 0xffff;

float z = VirtualTextureDescriptor.Y & 0xffff;

float w = (VirtualTextureDescriptor.Y >> 16) & 0xffff;

To pack the pair uint into the 4 components of the vector4, does it seem to work due some data last, we are investigating still, but this a good hit, thanks

Jorge CR

[Attachment Removed]

I’m looking at the issue along with Jorge. I changed the packing code Jorge mentioned above to use a BitCast and a similar `asuint` on the shader side to readback the descriptor.

float x = BitCast<float>(VirtualTextureDescriptor.X);
float y = BitCast<float>(VirtualTextureDescriptor.Y);
DescriptorsAsPayLoad.Add(FVector4f(x, y, 0, 0));

Now, our descriptors are exactly the same on the CPU and GPU side. I verified using breakpoints on CPU and a PIX capture.

[Image Removed]

However, we’re still getting these errors `Space 0, vAddr 4117@0 is not allocated to any AllocatedVT but was still requested.`

I want to confirm that the value for `Descriptor.X` inside `MeshPaintVirtualTexture::GetTextureDescriptor` should be the same as what we should expect for `PageEncoded` inside `FVirtualTextureSystem::GatherRequestsTask`. Is that correct? The values I am seeing are very different.

[Attachment Removed]

Hi Sakib,

Descriptor.X and PageEncoded are expected to be different.

Descriptor.X contains part of the information required to describe where the all page table entries for an allocated virtual texture can be found within the page table texture.

PageEncoded contains a description of which individual page table entry was requested for sampling at a pixel.

However, one part should match. That is the top 4 bits which encode the virtual texture space ID in both the Descriptor.X and the PageEncoded. Knowing that might help validate the correctness of your implementation.

Best regards,

[mention removed]​

[Attachment Removed]

Hi Sakib,

Can you see why FinalizeVirtualTextureFeedback is compiled out? Usually it would be compiled in for any material that samples virtual textures. One thing to note is that FinalizeVirtualTextureFeedback will only actually write feedback for one pixel in each screen tile of size r.vt.FeedbackFactor. Sometimes setting that cvar to 1 can help make PIX style shader debugging easier.

For the “not allocated to any AllocatedVT” errors. Each mesh paint virtual texture should allocate some area in the VirtualTextureAllocator and the XY Offsets and Sizes are passed up in the texture descriptors. The GPU feedback should then only include pages within the allocated area. You would get this error if your descriptors are somehow incorrect (maybe not updated after a virtual texture reallocation?).

It could be worth dumping out each time you hit FVirtualTextureSystem::AllocateVirtualTexture() and FVirtualTextureSystem::DestroyVirtualTexture() along with the relevant parts of FAllocatedVirtualTexture such as VirtualPageX, VirtualPageY, WidthInBlocks, HeightInBlocks. Then you can see if your feedback items are pointing at stale/released areas of the virtual texture space.

Best regards,

Jeremy

[Attachment Removed]

Thanks for clarifying that. They do seem to match.

The following is at the point of calling MeshPaintVirtualTexture::GetTextureDescriptor

Descriptor Created : Space 0, vAddr 4@0
Descriptor Created : Space 0, vAddr 8@0
Descriptor Created : Space 0, vAddr 262144@4

All the errors from GatherRequestTask are for Space 0

Space 0, vAddr 4420@0 is not allocated to any AllocatedVT but was still requested.
Space 0, vAddr 4165@0 is not allocated to any AllocatedVT but was still requested.
Space 0, vAddr 4373@0 is not allocated to any AllocatedVT but was still requested.
Space 0, vAddr 4357@0 is not allocated to any AllocatedVT but was still requested.
Space 0, vAddr 4369@0 is not allocated to any AllocatedVT but was still requested.
Space 0, vAddr 4180@0 is not allocated to any AllocatedVT but was still requested.

It is an empty map with a single ISM with 3 instances each using 3 different VTs. So, there is not much else going on.

I’ll continue investigating. Thanks for helping us sanity check the implementation.

[Attachment Removed]

Additional info below ..

I noticed that most of the `AddressBlocks` inside the VirtualTextureAllocator do not have an AllocatedVirualTexture. The state says FreeList or PartiallyFreeList. Only 7 out of 49 address blocks had a valid AllocatedVirtualTexture.

[Image Removed]

The blocks seem to be allocated in `FVirtualTextureAllocator::Alloc`, which is called when we fetch the texture descriptor.

[Image Removed]So, I’m at a bit of a loss. If the allocation is correct and the descriptor is passed correctly, could we be getting bad feedback requests from the shader?

[Attachment Removed]

After some more debugging, I can see that `FinalizeVirtualTextureFeedback**`** is never getting called after `StoreVirtualTextureFeedback`. The finalize code path is compiled out. This probably explains why the feedback isn’t working as expected.

[Attachment Removed]