Modifying normals in CS

Development Rendering
1

Recently I’ve managed to deform a mesh through a compute shader. Although there are many approaches, I used the VertexBuffer of a StaticMesh to “directly” deform the mesh.
This is how I get the vertex buffer with the positions:


UStaticMeshComponent* mesh_comp = (UStaticMeshComponent*)RootComponent->GetChildComponent(0);
FStaticMeshRenderData* render_data = mesh_comp->StaticMesh->RenderData;
FVertexBufferRHIRef vertex_buffer = render_data->LODResources[0].PositionVertexBuffer.VertexBufferRHI;

I can then use the RHI vertex buffer to do whatever. I successfully managed to deform a mesh.

My next challenge would be to update the deformed mesh normals.

I’ve read that it’s possible to recalculate the mesh normals with some engine functions, but it seemed by looking at the ource that it only worked within the editor, which would not do for me.

Since that approach didn’t work, I was confident that I could also place the buffer with the mesh normals in the compute shader and process it there. The process I used is basically the same I had used for the vertices’ positions, only in this case the buffer I’d have to use is simply called VertexBuffer.

Like so:


FVertexBufferRHIRef vertex_buffer_nrml = render_data->LODResources[0].VertexBuffer.VertexBufferRHI;

Problem is that, somehow, the normals are weirdly codified into its buffer, because the same buffer supposedly has both TangentZ (normals) and TangentX. The values that shown up (if I check them directly) are nonsense (stuff like huuuge numbers or #QNANO).

Is there a way to only get the normals from the static mesh, assuming I’m not getting this all wrong? I was thinking that it would have something to do with the struct offset in the buffer. but I fiddled with it and got to no conclusion (talking about using STRUCT_OFFSET macro in RHILockVertexBuffer, for example).

I can be a bit more specific and post a little more code if necessary.

In any case if there’s a really simple way to recalculate a mesh’s normals dynamically that I haven’t heard of, please do let me know!

Many thanks in advance!

2

Ah well, I’ve figured it out. The VertexBuffer is using PackedNormals to transport both the tangentX and Z. Thing is that each of these PackedNormals (which are 32 bit sized) represent an entire vector, i.e., the engine stores a FVector (or FVector4) into an uint32.

If one knows how to decode the uint32 in the compute shader, one should be able to modify the normals “at will”.

Check PackedNormal.h to see what I mean.

Hope this helps someone. :wink:

3

What Module-includes did you use in your .target to be able to access the VertexBuffer without any linker errors?

4

To my .target? I don’t remember doing anything to it. I did add a few modules to the .Build.cs file if that’s any help: ShaderCore, RenderCore and RHI.

5

Whilst I don’t have the answer I am in a similar boat needing the vertices, UVs, Normals & Triangle Mesh data for a component I am writing. I believe the code I(we) are looking for is in the Engine source. Version 4.5 FbxMainExport.cpp line 1741 exports a static mesh to FBX. Specifically for normals on line 1826 it talks about the following:


	
        TArray<FbxVector4> FbxNormals;
	FbxNormals.AddUninitialized(VertexCount);
	for (int32 NormalIndex = 0; NormalIndex < VertexCount; ++NormalIndex)
	{
		FVector Normal = (FVector) (RenderMesh.VertexBuffer.VertexTangentZ(NormalIndex));
		FbxVector4& FbxNormal = FbxNormals[NormalIndex];
		FbxNormal = FbxVector4(Normal.X, -Normal.Y, Normal.Z);
		FbxNormal.Normalize();
	}

Which I think is probably what you are looking for?

6

I already found what I needed actually.

The static mesh has a FStaticMeshLODResources object with all the mesh’s information you’ll likely need. If you check the source code you’ll find it has a VertexBuffer (with normals’ information, albeit oddly codified), PositionVertexBuffer (with positions), IndexBuffer (for triangles), etc., etc…

Accessing and modifying those buffers worked for me, at least. Thanks anyway! :wink:

EDIT: forgot, the FStaticMeshLODResources object I mentioned is LODResources[0], such as is in the example in the first post.

7

How did you unpack the normal in the end? I have found the structure. I find if I call this code:


FVector Normal = (FVector) (RenderMesh.VertexBuffer.VertexTangentZ(NormalIndex));

I then get an unresolved external:


1>MyClass.cpp.obj : error LNK2001: unresolved external symbol "__declspec(dllimport) union __m128 const GVectorPackingConstants" (__imp_?GVectorPackingConstants@@3T__m128@@B)
8

Well, the VertexBuffer has a packed normal, as you might have already seen. In case you haven’t, I’ll briefly explain my understanding: each packed normal represents a FVector4 condensed into a 32bit value, presumably to save memory. So, each 8 bits of this value will be a normal component, such as X, Y or Z.

Since I modified it in the compute shader I actually used binary operations for the effect. I’d say you can do the same in C++, but I haven’t tried it. Here’s a sample code of how I codified a normal in my compute shader:


int u;

u = w;
u = (u << 8) + z;
u = (u << 8) + y;
u = (u << 8) + x;

return asfloat(u);

Do note that this is HLSL code!

9

About unresolved externals I might suggest checking if you’ve got ShaderCore, RenderCore and RHI in your .Build.cs file.

10

I can’t see a .Build.cs file I do have a .Target.cs though. I presume I add the modules in there?


OutExtraModuleNames.AddRange(new string] { "game", "RHI", "ShaderCore", "RenderCore" });

Still get the same unresolved symbols. That’s how you add modules to UDK? Neat.

11

You can’t find the Build.cs file? Odd… It should be in your project’s Source/“game name”/ folder. If it really isn’t, then I guess you could try re-generating your project files (by right-clicking on the .uproject file). Not sure if that works…