I have been trying to implement a Thermal Render Plugin for CARLA (a autonomous driving simulation based on Unreal).
For that I wanted to allow artists to create ThermalMaterial node inside the material editor to assign textures to various thermal properties. These need to be customizable per object similar to visible RGB textures.
Additionally, my best approach here seems to be to create a custom FMeshMaterialShader to read these textures on a per-object basis.
The artists should then be able to place a Thermal Camera Actor, whose viewport renders my thermal view.
I have implemented a custom Vertex Shader, Pixel Shader (which don’t do anything but pass through vertex information and output red), and a custom MeshProcessor.
To render my scene with this setup I have created a View Extension, which hooks into PreRenderView_RenderThread and calls my ThermalRender function. This seems to be working in so far that the plugin compiles and regular engine playback starts.
But as soon as I open any material in the material editor, Unreal crashes with the error message:
LogMaterial: Error: Couldn't find Shader (ThermalVS, 0) for Material Resource NewMaterial!
RenderMeshShaderMap 1, RenderThreadShaderMap 1
GameMeshShaderMap 1, GameThreadShaderMap 1, bShaderWasFoundInGameShaderMap 0
With VF=FLocalVertexFactory, Platform=PCD3D_SM5
ShouldCache: Mat=0, VF=1, Shader=1
MaterialUsageDesc: LightingModel=MSM_DefaultLit, BlendMode=BLEND_Opaque, SpecialEngine=0, TwoSided=0, TSNormal=1, Masked=0, Distorted=0, WritesEveryPixel=1, ModifiesMeshPosition=0, Usage={}
[2025.02.08-14.54.30:863][280]LogMaterial: Error: ShaderType found in MaterialMap: TBasePassPSFNoLightMapPolicy
[2025.02.08-14.54.30:863][280]LogMaterial: Error: ShaderType found in MaterialMap: TBasePassPSFNoLightMapPolicySkylight
[2025.02.08-14.54.30:863][280]LogMaterial: Error: ShaderType found in MaterialMap: TBasePassVSFNoLightMapPolicy
[2025.02.08-14.54.30:863][280]LogWindows: Windows GetLastError: The operation completed successfully. (0)
[2025.02.08-14.54.46:268][280]LogWindows: Could not start crash report client using ../../../Engine/Binaries/Win64/CrashReportClientEditor-Win64-Debug.exe
[2025.02.08-14.54.46:268][280]LogMemory: Platform Memory Stats for Windows
[2025.02.08-14.54.46:268][280]LogMemory: Process Physical Memory: 15201.61 MB used, 15203.46 MB peak
[2025.02.08-14.54.46:268][280]LogMemory: Process Virtual Memory: 21488.71 MB used, 21490.55 MB peak
[2025.02.08-14.54.46:268][280]LogMemory: Physical Memory: 42510.70 MB used, 22793.27 MB free, 65303.98 MB total
[2025.02.08-14.54.46:268][280]LogMemory: Virtual Memory: 134199608.00 MB used, 18121.46 MB free, 134217728.00 MB total
[2025.02.08-14.54.46:268][280]Message dialog closed, result: Ok, title: The UE4-CarlaUE4 Editor has crashed and will close, text: Fatal error: [File:C:/UnrealEngine/Engine/Source/Runtime/Engine/Private/Materials/MaterialShared.cpp] [Line: 2215]
Fatal Error Material not found
I have debugged this issue to the point where I know that my ShaderPipeline is valid until the point where I open the material editor.
void GetThermalPassShaders(const FMaterial& Material, FVertexFactoryType* VertexFactoryType, ERHIFeatureLevel::Type FeatureLevel, TShaderRef<ThermalVS>& VertexShader, TShaderRef<ThermalPS>& PixelShader, FShaderPipelineRef& ShaderPipeline)
{
ShaderPipeline = UseShaderPipelines(FeatureLevel) ? Material.GetShaderPipeline(&ThermalOutputPipeline, VertexFactoryType, false) : FShaderPipelineRef();
if (ShaderPipeline.IsValid())
{
VertexShader = ShaderPipeline.GetShader<ThermalVS>();
PixelShader = ShaderPipeline.GetShader<ThermalPS>();
check(VertexShader.IsValid() && PixelShader.IsValid());
} else
{
ShaderPipeline = FShaderPipelineRef();
VertexShader = Material.GetShader<ThermalVS>(VertexFactoryType);
PixelShader = Material.GetShader<ThermalPS>(VertexFactoryType);
check(VertexShader.IsValid() && PixelShader.IsValid());
}
}
Here Material.GetShader(VertexFactoryType) runs into the error mentioned above.
My assumption is that my FMeshMaterialShader is not compiled for PreviewMaterials.
Now I am stuck and I cant really progress as I cant seem to find much documentation on MeshMaterialShaders online.
My goal is to implement these changes from within my plugin instead of having to adapt engine code.
Also, unfortunately, I am stuck with using UE4.26 as this is the version required by the CARLA version I need to develop this for.