[UE5] Anime/Toon/Cel Shading Model (WORKS WITH LAUNCHER ENGINE VERSIONS)

Share some interesting things :grinning_face:

really like your shader, and I’m trying to integrate it with a regular scene. The goal is to make them visually consistent within the same lighting environment (previously I set up lighting channels to make them work together) and under the condition that Lumen is enabled.

Increase the direct light for other shading models in DeferredLightPixelShaders.usf.

#endif // USE_ADAPTIVE_VOLUMETRIC_SHADOW_MAP
float4 Radiance = GetDynamicLighting(DerivedParams.TranslatedWorldPosition, DerivedParams.CameraVector, ScreenSpaceData.GBuffer, ScreenSpaceData.AmbientOcclusion, LightData, LightAttenuation, Dither, uint2(InputParams.PixelPos), SurfaceShadow);
if (ScreenSpaceData.GBuffer.ShadingModelID != SHADINGMODELID_CLEAR_COAT)
{
    Radiance*=4;//Multiplying directly seems a bit reckless?
}
OutColor += Radiance;
#endif // SUBTRATE_GBUFFER_FORMAT==1

Exclude the Lumen ambient light received by SelShader in DiffuseIndirectComposite.usf.

    // Accumulate lighting into the final buffers
#if SUBTRATE_GBUFFER_FORMAT==0
    if (Material.GBufferData.ShadingModelID != SHADINGMODELID_CLEAR_COAT)
    {
       IndirectLighting.Specular *= GetSSSCheckerboadSpecularScale(PixelPos, Material.bNeedsSeparateLightAccumulation);
       FLightAccumulator LightAccumulator = (FLightAccumulator)0;
       LightAccumulator_Add(
          LightAccumulator,
          IndirectLighting.Diffuse + IndirectLighting.Specular,
          IndirectLighting.Diffuse,
          1.0f,
          Material.bNeedsSeparateLightAccumulation);
       OutAddColor = LightAccumulator_GetResult(LightAccumulator);
    }

#endif // SUBTRATE_GBUFFER_FORMAT==0

Under the same directional light source, they no longer appear visually separated:

Now, with exposure turned off and light intensity set to 1, they no longer appear to be from two different worlds.

This is what I got:


Ordinary objects can receive Lumen ambient light from the SelShader

Use FaceMap in unreal engine :upside_down_face:

Just look at the changes in facial shadows

Facemap looks like this:

// ==== Inputs ====
// Input(0): LightDir (float3)      - 世界空间下的太阳方向(指向光线方向)
// Input(2): UV (float2)            - 原始纹理UV
// Input(3): OffsetAngle (float)    - 角度偏移值(弧度)
// ==== Output ====
// float3(x: 新UV.x, y: 新UV.y, z: 输出值)

// 预定义常量
static const float EPSILON = 1e-6f;
static const float HALF_PI = 1.5708f;
static const float TWO_PI = 6.283185307f;

// --- 1. 物体旋转计算 ---
float3x3 WtoL = LWCToFloat(GetPrimitiveData(Parameters.PrimitiveId).WorldToLocal);

// 使用更快的长度计算并避免分支
float sy_sq = WtoL[0][0] * WtoL[0][0] + WtoL[1][0] * WtoL[1][0];
float sy = sqrt(sy_sq);

// 直接计算偏航角,避免完整欧拉角转换
float Yaw = (sy_sq < EPSILON * EPSILON) ? 0.0f : atan2(WtoL[1][0], WtoL[0][0]);

// 角度规范化
Yaw = fmod(fmod(Yaw, TWO_PI) + TWO_PI, TWO_PI);

// --- 2. 太阳方向计算 ---
// 假设LightDir已归一化,直接使用xy分量
float3 SunDir = float3(-LightDir.x, -LightDir.y, 0.0f);

// --- 3. 角度计算---
float cosYaw, sinYaw;
sincos(Yaw, sinYaw, cosYaw);

// 直接计算点积和叉积
float dotVal = cosYaw * SunDir.x + sinYaw * SunDir.y;
float crossVal = cosYaw * SunDir.y - sinYaw * SunDir.x;

// 计算角度并应用偏移
float angle = atan2(crossVal, dotVal) + OffsetAngle + HALF_PI;

// --- 4. 输出值计算 ---
float sinAngle, cosAngle;
sincos(angle, sinAngle, cosAngle);

float value = mad(-0.5f, cosAngle, 0.5f);

// --- 5. UV翻转 ---
float2 NewUV = float2(
    lerp(UV.x, 1.0f - UV.x, sinAngle > 0.0f),
    UV.y
);

// --- 6. 输出结果 ---
return float3(NewUV, value);

@Envieous Is it possible to show a transition area at the boundary between light and dark areas?

1 Like

No. Unreal does not produce a meaningful gradient between the two. This may be possible with raytraced lighting but I haven’t investigated that at all.

5.7 Version released.

1 Like

Is there a way to change the number of bands and the distance of them in this shader. And can it be done in the editor or is this done in the source?

Unreal does not produce a meaningful gradient with dynamic shadows. The lighting information to produce multiple bands does not exist. What you want may be possible with raytracing, but I haven’t explored that.


Hi, I’m having a problem with 5.6.1. I moved all the necessary files to the engine folder, but for some reason the engine keeps crashing. Either I did something wrong, or I just don’t understand it.

Either you downloaded the wrong branch for your version, or Epic made changes in an update that broke it. They usually don’t touch these files in updates, but if they did then you’ll have to compile it from source from my repo instead.

Very cool project OP!

However, I think something got changed, possibly broken between engine versions 5.4 and 5.5.

This is from 5.6, watch where the shadow line is on the spheres. It’s basically 80% shadow and 20% light, whereas on your gifs and screenshots the ratio would be about 50%.

And while the spheres at least look aesthetic enough, look at the floor and compare between the two screenshots, and watch the angle of the light source. I haven’t even reached 90 degree angle with the light source, but the ground is already entirely dark, it looks very jarring and means we can’t have long stretching shadow like you’d see in the golden hours, because everything flattens into darkness too early.

So I went ahead and checked 5.4 version, and yeah that confirmed my suspicion, something did change between 5.4 and 5.5 that changed behavior of the shadow lines. Ideally though, there would be some way to manually pick the ratio, and maybe there is, and I just missed it.

Unfortunately it also seems like there are no example project files for versions bellow 5.5 anymore, so figuring out how to do things here becomes quite a bit of a pain. If it’s not too troublesome, could example files for 5.4 be shared again?

One more thing though, earlier in the thread several people mentioned how when they open the example project, it appears darker than on the screenshots. You advised tinkering with the post processing. However what I found out, is that the light source itself, featured in the example project, has dark colors pre-applied to it, not just on the Shadow Colour, but also the Light Colour, both by default are set to two different shades of grey, whereas you’d think at least the Light Colour would be set to white (it’s a little deceptive looking, because it looks very close to white at a glance, but it isn’t). In other words, one would never get the real material colors unless they change the light source colors first. Ideally I’d say both of the light source colors should be set to white, so they don’t interfere with the materials out of the box.

The shadow offset value was changed because at the previous value it caused visual errors due to dynamic shadows not being correctly covered by the shadow terminator from NoL shadows from point/spot lights. You can adjust the shadow offset variable in shadingmodels.ush to the old value if you want that instead, but it was changed for a reason.

Also yes, the directional light should always be pure white.

1 Like

Is this the part that i should change? There’s no equivalent in 5.4 version, so I’m not sure exactly what i should put in.

I did try some things, like setting it to 0.0 (changes nothing), rising it, or making it negative. The results either make the shadow part bigger, or cause severe artifacts on the shadow line:

Thank you for the help, btw.

You want a value around -0.3, but you’re seeing why I don’t have it set to that already. 0.1 is the value where dynamic shadow artifacts are covered. Lower values will cover less of them.

1 Like

Hi! First of all, thank you so much for your amazing work on this shading model!
I’m trying to achieve heavy comic book style with multiple light sources. The problem I encountered, is that the directional light behaves completely different than the other light. For example, when the scene is only illuminated by a point light, the color is completely lost, both in light and in shadow. I’m using 5.7 launcher version, will this work as I intend in source compiled version?
EDIT: I managed to manage my desired stylisation with only a postprocess material, but I’m still curious if this could be solved, maybe that will help someone else

Can it work for baked GI on both forward and deferred rendering

(with all projects set to volumetric modes)

It’s deferred only, but yes, lightmaps are supported. I don’t know what you mean by project set to volumetric mode, but the only behaviour i modified is how shadows are rendered, anything outside that still behaves like it would it stock Unreal.

1 Like

I see. Is mobile deferred supported?

Not at the moment. When I’m done my game and port it to mobile I’ll probably support it but that’s a few years out.

2 Likes