[UE5] Anime/Toon/Cel Shading Model (WORKS WITH LAUNCHER ENGINE VERSIONS)

I haven’t looked at the code that handles that for a long time so I can’t say for sure, but I don’t think light intensity is actually passed on because it’s nothing more than a multiplier for the colour.

I haven’t actually tried to do anything with it, but the aforementioned setting to make my model output a true lighting buffer is the best way I can think of to accomplish this. You can write whatever rendering logic you want in a post process utilizing that data, to lerp between the basecolour and shadowcolour buffers.

I haven’t experimented with it, but if I was going to play with lumen and stylization that is what I’d try first.

Yeah, it would be just a more accurate solution to what I use currently - divide scene color and base color… Or have you noticed that it’s drastically better?

It’s the true lighting data instead of the crude approximation you get from scenecolour/basecolour. You still have to decide what to do with it, but yes, it’s better.

1 Like


Yay, modified the sources and it works just as I wanted!

Hopefully next engine updates (when they move their strata thing forward) won’t break too much stuff :expressionless:

Started with replicating your idea of directly outputting normal lighting, but in the end went with modifying lighting calculations and a custom shading model :v

1 Like

Do you happen to know of a way to remove self-shadowing (while receiving all other shadows) with engine code?

Conceptually, I think you’ll likely need to create an extra render pass with a custom shadow map of all casted shadows for the non-self-shadow meshes (i.e., shadows that these objects cast onto other objects) and create a mask to only render these shadows. For the non-self-shadow meshes themselves, you can turn “Cast Shadows” to False to receive all shadows from other objects. In the end, these meshes will be able to both cast shadows (due to the extra render pass) and receive all shadows that are not their own (the “Cast Shadows” = False setting).

Not sure if that makes any sense, or even works, but it seems pretty complicated for just removing self-shadowing while considering how shadow mapping works. Probably grandmaster-level shader programming proficiency in Unity, and likely impossible in UE? :rofl: :melting_face:

The reason I’m asking is that self-shadowing ruins shadow masking. Shadow masking is the superior way of anime face shading (as opposed to mesh normal editing or other shadow hacks to get around PBR shadows), and gives much cleaner and more controllable results.

I’ve only glanced at the shadow rendering files, so no I I don’t know how to do that, but even without self shadowing showing up (pointing my characters right at the light), I never found that other shadows looked very good casting on my character’s faces.

The Genshin Impact face shadow texture approach has been the one that I’ve found to look the best.

Have you uploaded this to github or something like that?
Your implementation looks amazing.

1 Like

This looks fantastic! Are you able to post anything on how you achieved this? /provide the fork where you got this result?

1 Like

Do you think this might do what GetPerPixelLightAttenuation(float2 UV) was supposed to do? I think this is really all I’m looking for, but it’s been broken since somewhere around 4.21.

GetPerPixelLightAttenuation(float2 UV) was an HLSL function. Would be interesting to see if that can be fixed – it’s far preferrable to do this in-shader rather than as a post process.

It outputs what the picture shows, which is more or less what you would see if you pressed alt+6 to get the lighting viewbuffer in the editor.

1 Like

5.2 Version added.

1 Like

Is it work on mobile ?? :smile:

Yes (probably, I havent tried it), but only through using the desktop renderer on mobile. After my game is finished I intend to port this setup to the proper mobile renderer, but that won’t be for over a year.

1 Like

Amazing work!

Any information under what license it is? :sweat_smile:

the same one as the regular unreal engine as far as i know

Amazing work!

Only problem I see with this solution is that you can’t mix toon shaded characters and a normally lit (PBR) environment. Any way to solve that? (Adding a new shading model seemed like a ton of work, so maybe use CustomData1 to switch models?)

It’s fairly simple (just tedious) to cordon off my changes in the usf/ush files and put them behind a bunch of if shading model = x then statements and restore the normal default lit behavior.

I don’t do that because there’s no situation where I’d want that, and it’s a big timesink to write the extra code to define a new shading model for certain behaviors, and then change the shading model dropdown in every single material to the one I want, but it’s not difficult to add if you wanted to.

You could use customdata1 to do the same thing but there’s not really a point because you’d still be going through my changes and placing a bunch of if thens, and just using that value instead of the shading model enum, so there’s not really a good reason to do it that way unless you wanted to lerp between the two shadings.

1 Like

5.3 Version released.

1 Like

Hi.
First, thanks for hardwork!
I have a question while using your modified engine, does it works well with UE 5.2 or 5.3 version?
cause when I build UE from your source code branch 4.25, it works well
but when I build from 5.2 or 5.3 branch, Editor crashes when loading projects.
(also it

Both versions work fine. I was just using 5.2 for my project last week and updated to 5.3 when I released that version. Try loading the engine without a project and creating a fresh one from that screen.