[UE5] Anime/Toon/Cel Shading Model (WORKS WITH LAUNCHER ENGINE VERSIONS)

[Envieous]

@LoafingShiba

Changing engine code can be pretty nasty each time a new UE version releases. There’s also the potential of needing to re-implement your shading if the shading code changes too much.

This is why I keep the changes as simple as possible. I change literally 5 lines of C++ code, and just hijack the existing shading model. It’s trivially easy to just copy paste them over between engine versions as long as Epic doesn’t throw in any curveballs, and the only one that was troublesome to figure out was them changing where the CustomData buffer is enabled between 4.27 and 5, otherwise they just seem to pointlessly shuffle shader code around with each engine version without making any actual changes which is usually not a big deal to figure out since the functionality didn’t actually change.

At this point the most obnoxious thing about this is that it requires wasting everyone’s time recompiling the engine for HOURS just to make basic C++ changes to open a material pin, and as of 5, enable the CustomData buffer, which you could do before in the USF/USH files.

The worst part is that a little before 5 came out I had made a version that worked without touching the engine source and worked on launcher builds (by using the compression method I use in the metallic pin), and then Epic moved the ability to enable the CustomData gbuffer into the C++ code.

@LoafingShiba

Have you considered using a post process cel shaded material in combination with some sort of object masking, like a Custom Stencil Buffer? This cel shader from the Unreal Marketplace pretty much combines all of those concepts.

I have, and they’re so limited that they’re borderline useless. Anyone selling them in the marketplace pushing them as a good solution is, at best just marginally less clueless than the people buying them or at worst taking advantage of the ignorance of newbie developers to make a quick buck. Their sole benefit is that people who don’t know what they’re doing can implement them in their projects easily.

That said I get why they’re so prevalent, there’s a big leap in knowledge required between throwing a post process into unreal and modifying unreal’s shader code with the only middleground inbetween being like 2 people’s shading models that may or may not fit your needs, and on top of that the only information on how to tackle this stuff is a few extremely out of date community tutorials, and studying the changes existing custom shading models have made.

@LoafingShiba

I haven’t investigated the performance of using a masked post process shader vs. modifying the engine code, but if you’re using deferred shading…that’s pretty much just adding one extra layer to the whole stack of G-buffer layers anyway? So it wouldn’t be too much of a performance hit? The gains in maintainability would be tremendous if the masked post process shader works out.

Up until recently I was using an RX580 which was basically a mid end card at best when it came out in 2017, and I’ve used both approaches without every running into any noticeable difference in performance. As I understand it using a post process does come with an added cost, but I’ve never found it to be significant.

@LoafingShiba

I thought Genshin/Honkai Impact’s subsurface scattering is just color/shadow banding? Pretty bleh. Your dynamic subsurface scattering looks substantially better , where it “turns on” when the light vector points more towards the camera.

I took a quick glance and couldn’t find the article again and I’m not really sure how genshin does it, but if all you want is for it to fade based on whether the camera is looking at the light source that’s very easy to accomplish in the material editor.

This is the main reason I made shadow colour a literal value instead of just multiplying it in the shader code. A lot of the shader code is is just the exact same logic you would make in the material graph, it just has access to additional information that isn’t available in the material graph, most importantly for our needs, the true lighting data (light colour, dynamic shadows, cubemaps, etc). For effects that don’t need this information (though with my shading model you do get to indirectly define behavior in shadow/lit) you can make your own on the material level and not be reliant on whatever hardcoded formulas are in the shadercode.