Creating a Custom Shading Model in Unreal Engine 5

I think I can mostly answer my own question:

There is a new file called ShaderGenerationUtil.cpp that handles packing the GBuffer and some other stuff - that file also requires some changes to work with a new shading model. I made two changes to it - if you look at where the EYE shading model is used in the file you need to add similar stuff for your new shading model. DetermineUsedMaterialSlots and void FShaderCompileUtilities::ApplyFetchEnvironment(FShaderMaterialPropertyDefines& SrcDefines, FShaderCompilerEnvironment& OutEnvironment) need changes.

I also changed SetSlotsForShadingModelType but I’m not sure if that code is ever called (??) or what it does exactly…My new shading model uses CustomData0 and 1, which are normally merged into CustomData slot but can also exist as two separate 8-bit channels? I think I understand the intent of this code but I have no idea how it maps values to the slot definitions and some of the slots never seem to get used anywhere…

So:

  1. Change a couple places in ShaderGenerationUtil
  2. Maybe change a third place if you are adventurous

Other than that just doing what worked in UE4 seems to work. I also had to save some materials with my custom shading model as a different shading model then set it back to my custom one to get changes to take, but that might just be a shader caching issue.

9 Likes