Material Local Space and Instanced Static Meshes

Hi people. I’m working with materials on Instanced Static Meshes and I’m noticing that when transforming positions to Local Space in the Material Editor (I’m specifically using the ObjectPivotPoint node), I always get the pivot point for the Instanced Static Mesh component (or the actor that owns it, not sure), but not each instance’s pivot point.
I’m doing cylindrical billboarding around the local X axis using World Position Offset, which works wonders on separate meshes. But on lots of instanced mesh sheets, which as well are segments of a lightning bolt, I’m always getting the same location for the pivot point. And since I need a reference of the camera location respect of the object space, this leads to incorrect behaviour. And the local X axis i’m getting is wrong also.

Help is appreciated.

I’ve taken a look at MaterialTemplate.USF, and it appears that TransformLocalVectorToWorld should return you per instance transformed position, but only if used in vertex frequency shader. and only if used with instanced meshes or particles. So to get what you need, you would have to use either (0,0,0) hooked up to TransformPosition(Local to Absolute World) or ObjectPosition node. They would have to be connected to two custom UVs slots. Then you can retrive the data in pixel shader by TextureCoordinate node. There might be a better solution, but this is what comes as obvious answer.

I’m having difficulty following your steps to get the per instance transformed position. Do you think you could provide a screenshot of the nodes (or a more detailed step by step procedure)? Thanks!

1 Like

Thank you! It works perfectly. (In case someone can’t find the customized UV connections on the main material node, they can be added by clicking on the main material node and setting the number of customized UVs in the details panel)

Do you have any good resources that explain conceptually what the material editor is doing? For example, like how you knew that nodes connected to a customized UV slot would be executed in a vertex frequency shader. Did you just figure it out by looking at the MaterialTemplate.USF? The UE4 wiki is great for beginners, but it really doesn’t go into depth.

Material editor is just transferring nodes into HLSL code and combining that with MaterialTemplate.USF to get actual final shader code.

I also faced the task of getting per-instance position once, and my journey through docs went nowhere, and answerhub was suggesting that it was not possible. Not being able to get per-instance position seemed more than strange for me and I went on with checking compiled and MaterialTemplate.usf in particular. Inside the file on line 523 I found what I was looking for. I don’t actually think that this is documented anywhere.

If that was of any help for you, please, toss me a vote or something. It would slightly facilitate my road to a forum badge :slight_smile:

3 Likes

Can you also set this up if you want the local position of each vertex of each instance?
Not sure if I understand Customized UV, but I tried to put the World Position Offset calculations into UV0 and UV1 (masked out RG into UV0 and GB into UV1). Then I used 2 TexCoords and plugged them into World Position Offset, sadly no luck :3
It seems like Absolute World Position won’t execute on the Vertex Shader.

I ran into the same problem.I am using instanced meshes and I needed to do local to world transform in the material pixelshader(not vertex shader) but the transform node is not supported in the PS when using instancing. However I found out you can fairly easily modify the USF files to include that functionality without having to change anything in the engine’s c++ code (you will also be able to access instance position this way in the PS). I simply added the needed matrix stuff to LocalVertexFactory.usf, MaterialTemplate.usf and LocalVertexFactory.usf and made it possible to access this stuff from the pixel/fragment shader in the material editor.

The following steps work for UE 4.14:
In these files (LocalVertexFactoryCommon.usf, MaterialTemplate.usf and LocalVertexFactory.usf) you just need to look for the “USE_INSTANCING” defines. I did the following changes
(please backup the files before you make any changes to them!):

1.Change following in “struct FVertexFactoryInterpolantsVSToPS” in LocalVertexFactoryCommon.usf :

#if USE_INSTANCING
	// x = per-instance random, y = per-instance fade out amount, z = hide/show flag, w dither fade cutoff
	float4  PerInstanceParams : COLOR1;

	//our new code:
	float4 InstanceOrigin : COLOR2;  // per-instance random in w 
	half4 InstanceTransform1 : COLOR3;  // hitproxy.r + 256 * selected in .w
	half4 InstanceTransform2 : COLOR4; // hitproxy.g in .w
	half4 InstanceTransform3 : COLOR5; // hitproxy.b in .w

#endif

2.Now in LocalVertexFactors.usf go to function “FVertexFactoryInterpolantsVSToPS VertexFactoryGetInterpolantsVSToPS(FVertexFactoryInput Input,”
Look for #if USE_INSTANCING. Here we copy the Instance Transform data to the Interpolants(so that they are copied to the pixelshader):

#if USE_INSTANCING
	
	Interpolants.PerInstanceParams = Intermediates.PerInstanceParams;
	
	//our new code:
	Interpolants.InstanceOrigin = Input.InstanceOrigin ;
	Interpolants.InstanceTransform1= Input.InstanceTransform1;
	Interpolants.InstanceTransform2= Input.InstanceTransform2;
	Interpolants.InstanceTransform3= Input.InstanceTransform3;
#endif

3.After that go to “struct FMaterialPixelParameters” in MaterialTemplate.usf and also look for #if USE_INSTANCING and change it too:

#if USE_INSTANCING
	half4 PerInstanceParams;

	//our new code:
	float3 InstanceLocalPosition;
	float4x4 InstanceLocalToWorld;
#endif

4.go to LocalVertexFactory.usf to function “FMaterialPixelParameters GetMaterialPixelParameters(FVertexFactoryInterpolantsVSToPS Interpolants,”
Again look for #if USE_INSTANCING and change it to the following code:

#if USE_INSTANCING
	Result.PerInstanceParams = Interpolants.PerInstanceParams;
	//our new code:
	Result.InstanceLocalToWorld = mul(GetInstanceTransform(Interpolants), Primitive.LocalToWorld);
#endif

5.Now we must add an entire new function “GetInstanceTransform”. So in LocalVertexFactory.usf look for the other GetInstanceTransform functions
and just paste the following function somewhere there:

float4x4 GetInstanceTransform(FVertexFactoryInterpolantsVSToPS Input)
{	
	#if !USE_INSTANCING_EMULATED
	return float4x4(
		float4(Input.InstanceTransform1.xyz, 0.0f), 
		float4(Input.InstanceTransform2.xyz, 0.0f), 
		float4(Input.InstanceTransform3.xyz, 0.0f), 
		float4(Input.InstanceOrigin.xyz, 1.0f));
	#else
		return float4x4(
			float4(1,0,0,0),
			float4(0,1,0,0),
			float4(0,0,1,0),
			float4(0,0,0,1));
	#endif	
}

6.Finally in MaterialTemplate.usf we need to change the function “MaterialFloat3 TransformLocalVectorToWorld(FMaterialPixelParameters”
which will allow us to use this transform node in the pixelfragment part of the material editor:

/** Transforms a vector from local space to world space (PS version) */
MaterialFloat3 TransformLocalVectorToWorld(FMaterialPixelParameters Parameters,MaterialFloat3 InLocalVector)
{
	#if USE_INSTANCING
		return mul(InLocalVector, (MaterialFloat3x3)Parameters.InstanceLocalToWorld);
	#else
		return mul(InLocalVector, GetLocalToWorld3x3());
	#endif
}

7.Now if you want to get instance object position in the PS too you can do that by modifying “float3 GetObjectWorldPosition(FMaterialPixelParameters Parameters)” in MaterialTemplate.usf

/** Return the object's position in world space */
float3 GetObjectWorldPosition(FMaterialPixelParameters Parameters)
{
	#if USE_INSTANCING
		return Parameters.InstanceLocalToWorld[3].xyz;
	#else
		return Primitive.ObjectWorldPositionAndRadius.xyz;
	#endif
}

That’s all that is needed. If you need access to other transform functions in the PS you might have to change one of the functions in MaterialTemplate.usf too
but change only those that have (FMaterialPixelParameters Parameters).

2 Likes

I’d be curious to know about this too. I am attempting to do vertex animation on instance meshes and I would like to apply a “yaw” offset to the vertices of each instance in local instance space. “TransformVector” appears to simply transform to component space, which is incorrect. I am working strictly in the vertex shader btw.

Hi,
Is it possible to access the ‘ObjectPositionWS’ node from PostProcess material?

The same above just in one diff. Init diff in Shader folder.

diff --git a/Private/LocalVertexFactory.ush b/Private/LocalVertexFactory.ush
index 604eb17..656c26d 100644
--- a/Private/LocalVertexFactory.ush
+++ b/Private/LocalVertexFactory.ush
@@ -382,6 +382,23 @@ float4x4 GetInstanceTransform(FPositionAndNormalOnlyVertexFactoryInput Input)
 	#endif
 }
 
+float4x4 GetInstanceTransform(FVertexFactoryInterpolantsVSToPS Input)
+{    
+    #if !USE_INSTANCING_EMULATED
+    return float4x4(
+        float4(Input.InstanceTransform1.xyz, 0.0f), 
+        float4(Input.InstanceTransform2.xyz, 0.0f), 
+        float4(Input.InstanceTransform3.xyz, 0.0f), 
+        float4(Input.InstanceOrigin.xyz, 1.0f));
+    #else
+        return float4x4(
+            float4(1.0f, 0.0f, 0.0f, 0.0f),
+            float4(0.0f, 1.0f, 0.0f, 0.0f),
+            float4(0.0f, 0.0f, 1.0f, 0.0f),
+            float4(0.0f, 0.0f, 0.0f, 1.0f));
+    #endif    
+}
+
 half3x3 GetInstanceToLocal3x3(FVertexFactoryIntermediates Intermediates)
 {
 	return half3x3(Intermediates.InstanceTransform1.xyz, Intermediates.InstanceTransform2.xyz, Intermediates.InstanceTransform3.xyz);
@@ -442,8 +459,13 @@ FMaterialPixelParameters GetMaterialPixelParameters(FVertexFactoryInterpolantsVS
 
 	// Required for previewing materials that use ParticleColor
 	Result.Particle.Color = half4(1,1,1,1);
+
+	uint primitiveId = GetPrimitiveId(Interpolants);
 #if USE_INSTANCING
 	Result.PerInstanceParams = Interpolants.PerInstanceParams;
+	
+	Result.InstanceLocalToWorld = mul(GetInstanceTransform(Interpolants),
+										GetPrimitiveData(primitiveId).LocalToWorld);
 #endif
 
 	Result.TangentToWorld = AssembleTangentToWorld( TangentToWorld0, TangentToWorld2 );
@@ -463,7 +485,7 @@ FMaterialPixelParameters GetMaterialPixelParameters(FVertexFactoryInterpolantsVS
 #endif	// LIGHTMAP_UV_ACCESS
 
 	Result.TwoSidedSign = 1;
-	Result.PrimitiveId = GetPrimitiveId(Interpolants);
+	Result.PrimitiveId = primitiveId;
 
 #if NEEDS_PARTICLE_LOCAL_TO_WORLD
 	Result.Particle.ParticleToWorld = GetPrimitiveData(Result.PrimitiveId).LocalToWorld;
@@ -932,8 +954,14 @@ FVertexFactoryInterpolantsVSToPS VertexFactoryGetInterpolantsVSToPS(FVertexFacto
 
 	SetTangents(Interpolants, Intermediates.TangentToWorld[0], Intermediates.TangentToWorld[2], Intermediates.TangentToWorldSign);
 	SetColor(Interpolants, Intermediates.Color);
+
 #if USE_INSTANCING
 	Interpolants.PerInstanceParams = Intermediates.PerInstanceParams;
+
+    Interpolants.InstanceOrigin = Intermediates.InstanceOrigin;
+    Interpolants.InstanceTransform1 = Intermediates.InstanceTransform1;
+    Interpolants.InstanceTransform2 = Intermediates.InstanceTransform2;
+    Interpolants.InstanceTransform3 = Intermediates.InstanceTransform3;
 #endif
 
 #if INSTANCED_STEREO
diff --git a/Private/LocalVertexFactoryCommon.ush b/Private/LocalVertexFactoryCommon.ush
index d9c08ec..8198b19 100644
--- a/Private/LocalVertexFactoryCommon.ush
+++ b/Private/LocalVertexFactoryCommon.ush
@@ -15,6 +15,11 @@ struct FVertexFactoryInterpolantsVSToPS
 #if USE_INSTANCING
 	// x = per-instance random, y = per-instance fade out amount, z = hide/show flag, w dither fade cutoff
 	float4  PerInstanceParams : COLOR1;
+
+    float4 InstanceOrigin : COLOR2;  // per-instance random in w 
+    half4 InstanceTransform1 : COLOR3;  // hitproxy.r + 256 * selected in .w
+    half4 InstanceTransform2 : COLOR4; // hitproxy.g in .w
+    half4 InstanceTransform3 : COLOR5; // hitproxy.b in .w
 #endif
 
 #if NUM_TEX_COORD_INTERPOLATORS
diff --git a/Private/MaterialTemplate.ush b/Private/MaterialTemplate.ush
index 509e457..e1e343d 100644
--- a/Private/MaterialTemplate.ush
+++ b/Private/MaterialTemplate.ush
@@ -301,6 +301,9 @@ struct FMaterialPixelParameters
 
 #if USE_INSTANCING
 	half4 PerInstanceParams;
+
+    //float3 InstanceLocalPosition;
+    float4x4 InstanceLocalToWorld;
 #endif
 
 	// Index into View.PrimitiveSceneData
@@ -868,9 +871,13 @@ MaterialFloat3 TransformLocalVectorToWorld(FMaterialVertexParameters Parameters,
 }
 
 /** Transforms a vector from local space to world space (PS version) */
-MaterialFloat3 TransformLocalVectorToWorld(FMaterialPixelParameters Parameters,MaterialFloat3 InLocalVector)
+MaterialFloat3 TransformLocalVectorToWorld(FMaterialPixelParameters Parameters, MaterialFloat3 InLocalVector)
 {
-	return mul(InLocalVector, GetLocalToWorld3x3(Parameters.PrimitiveId));
+	#if USE_INSTANCING
+        return mul(InLocalVector, (MaterialFloat3x3)Parameters.InstanceLocalToWorld);
+    #else
+		return mul(InLocalVector, GetLocalToWorld3x3(Parameters.PrimitiveId));
+    #endif
 }
 
 /** Transforms a vector from local space to previous frame world space (VS version) */
@@ -884,7 +891,11 @@ MaterialFloat3 TransformLocalVectorToPrevWorld(FMaterialVertexParameters Paramet
 /** Transforms a position from local space to absolute world space */
 float3 TransformLocalPositionToWorld(FMaterialPixelParameters Parameters,float3 InLocalPosition)
 {
-	return mul(float4(InLocalPosition, 1), GetPrimitiveData(Parameters.PrimitiveId).LocalToWorld).xyz;
+	#if USE_INSTANCING
+		return mul(float4(InLocalPosition, 1), Parameters.InstanceLocalToWorld).xyz;
+	#else
+		return mul(float4(InLocalPosition, 1), GetPrimitiveData(Parameters.PrimitiveId).LocalToWorld).xyz;
+	#endif
 }
 
 /** Transforms a position from local space to absolute world space */
@@ -910,7 +921,13 @@ float3 TransformLocalPositionToPrevWorld(FMaterialVertexParameters Parameters,fl
 /** Return the object's position in world space */
 float3 GetObjectWorldPosition(FMaterialPixelParameters Parameters)
 {
-	return GetPrimitiveData(Parameters.PrimitiveId).ObjectWorldPositionAndRadius.xyz;
+	#if USE_INSTANCING
+        return Parameters.InstanceLocalToWorld[3].xyz;
+    #else
+        //return Primitive.ObjectWorldPositionAndRadius.xyz;
+		return GetPrimitiveData(Parameters.PrimitiveId).ObjectWorldPositionAndRadius.xyz;
+    #endif
+	
 }
 
 float3 GetObjectWorldPosition(FMaterialTessellationParameters Parameters)