I have a task to add an editing features to display the world with Player Collision, but with each face being colored based on how similar its normal is to vertical. I am currently accomplishing this by enabling the default CollisionPawn viewmode and changing GEngine->ShadedLevelColorationUnlitMaterial to my own custom material. This works correctly, and I can swap back to the default engine material when I am finished.
Is there a better way to accomplish this besides changing GEngine->ShadedLevelColorationUnlitMaterial?
Beyond that, I am interested in using other materials that can visualize things about the actor they are attached to. For example, if the actor has a Faction component on it I’d like the color to change based on faction.
I am trying to avoid needing to edit the existing Static Mesh Scene Proxy class, but this also needs to work on arbitrary existing meshes.
Your approach of swapping ShadedLevelColorationUnlitMaterial is probably the best option at the moment, as we’ve hardcoded the view mode to use that material. For the other visualizations (e.g. faction), you should be able to use UMeshComponent::SetOverlayMaterial to apply a material that reads the data you want to visualize as custom primitive data:
FLinearColor FactionColor = GetFactionColor(Actor);
Mesh->SetOverlayMaterial(Overlay);
const int32 Index = Mesh->GetCustomPrimitiveDataIndexForVectorParameter(TEXT("FactionColor"));
if (Index != INDEX_NONE)
{
Mesh->SetCustomPrimitiveDataVector4(Index, FVector4(FactionColor));
}
I am running into a different issue now, where after swapping the material to my custom one it works well for static meshes, but not for Landscape actors in Player Collision mode. The material works fine in lit and unlit when assigned to the landscape’s Landscape Material property, but when I perform the GEngine variable swap and change the viewmode to Pawn/Player Collision, the Landscape actor attempts to render with my Slope Coloration material and fails in TryGetMaterials because ShaderMap->GetMeshShaderMap(InVertexFactoryType) is failing when the vertex factory type is FLandscapeVertexFactory. It reverts to using WorldGrid material as a backup.` How can I get my custom material to properly create shaders for FLandscapeVertexFactory? I am comparing to the built-in ShadedLevelColorationUnlitMateri [sic] material and I don’t see any differences in shader graph and details view, but the ShadedLevelColorationUnlitMateri reports 606 shaders and my custom material has 12. ShadedLevelColorationUnlitMateri also works correctly when in Pawn/Player Collision view mode.
I’ve managed to find the answer. There is a flag on all materials called bUsedAsSpecialEngineMaterial that forces the material to generate shaders for all possible uses. It is hidden from the editor by default, but using code to mark my material with this flag has it properly work on landscape.