Good practice to override engine ForwardLightingCommon.ush shader file?

LongNameOAO · July 12, 2022, 7:23am

I am trying to customize how lights are calculated into the scene similar to this shader file in Unity (basically trying to re-create the whole shader in UE4):
Line 26 at

github.com

ColinLeung-NiloCat/UnityURPToonLitShaderExample/blob/master/SimpleURPToonLitOutlineExample_LightingEquation.hlsl

// For more information, visit -> https://github.com/ColinLeung-NiloCat/UnityURPToonLitShaderExample

// This file is intented for you to edit and experiment with different lighting equation.
// Add or edit whatever code you want here

// #pragma once is a safe guard best practice in almost every .hlsl (need Unity2020 or up), 
// doing this can make sure your .hlsl's user can include this .hlsl anywhere anytime without producing any multi include conflict
#pragma once

half3 ShadeGI(ToonSurfaceData surfaceData, ToonLightingData lightingData)
{
    // hide 3D feeling by ignoring all detail SH (leaving only the constant SH term)
    // we just want some average envi indirect color only
    half3 averageSH = SampleSH(0);

    // can prevent result becomes completely black if lightprobe was not baked 
    averageSH = max(_IndirectLightMinColor,averageSH);

    // occlusion (maximum 50% darken for indirect to prevent result becomes completely black)
    half indirectOcclusion = lerp(1, surfaceData.occlusion, 0.5);

This file has been truncated. show original

It looks like that this cannot be achieved in PostProcess or Material since it is manipulating lighting calculation directly. I tracked all the way down and found ForwardLightingCommon.ush which seems to be the right place to make this kind of modification.

I tended not to directly change this file since changing directly on the Engine source code is always not the best option. I wonder if there is anyway to override this file. I found some articles saying about cutome shader files and hlsl files but they seem only work for Custom nodes for material editor.

Is there any option other than modifying this file directly?

ZkarmaKun · July 12, 2022, 11:43pm

There is no such thing as good practice trying to modify engine files, my suggestion is to create a fork from github and then modify the files as much as you please with your custom version.

Related to your topic, the place you are looking for to modify the light attenuation is inside the DeferredLightingCommon.ush, look at the function GetDynamicLightingSplit. I see you are trying to do some sort of toon effect, check this link, it might help, a bunch of dudes are supporting a toon shading model for UE.

LongNameOAO · July 13, 2022, 2:35am

First thanks a lot for that link reference!

It actually matches up with my new discovery. Creating a new shading model seems to be the best way to do this.