need help with a cel shading post process

i’m not exactly good at creating content for ue community but all the way around, as a person who just ask for content or tutorials, even tho i’m myself a software engineer, but anyways, i’m here for 2 things, i need help with a post process cel shading that i’ve been working around for a while, and collecting a bunch of information around the forums, i managed to get the cel shading for my game, but there are big problems here, first of all, this isn’t working on mobile, which is mandatory for me, and second, i made it to work on only rendered custom meshes, the reason why is because if i apply this to everything, my skyphere dissapears and i get an entirely black map, oh and a third one is that shadows aren’t applied on cel shaded objects

Are you using pixeldepth? Apparently it’s not supported in mobile post processes.

@Vasault I remember this [discussion](Need help with a sobel edge post process effect :( - Rendering - Unreal Engine Forums and i was trying to explain you, that the way you modified the sobel kernel is the reason you are getting odd results. Instead of fiddling with the kernel internals, you should try a different approach in order to get the sobel effect working on the surfaces where you are willing to project it. I actually experimented with it, and gave you a solution that you should investigate more, and maybe you can ask for help if you don’t find it sufficient.
@xnihil0zer0: You can perhaps modify the engine in order to get access to the depth map on mobie? I actually never looked into this in details, but if you were missing this texture in the postprocess then it’s possible you can change a few things to overcome this issue.

no no dude, i mean i’m great with the outline effect, i’m currently working on cel shading effect now, two completely different post process effects

I have found if you apply a ceil() function in the color reduction the dark scenes will appear to be more bright, thus making the edges more prominent. I also mix the original color with the quantized version, that result in a nice inbetween imo.

In this implementation i use the depth channel to slowly fade out the edges at the distance, but it not really a neccessity. If you just use the world normal (instead of the postprocess colour input), you won’t get any sobel edges on the skysphere anyways. Also, the sky in a toon environment could just be simply a gradient background, you don’t really have to use images of clouds and whatnot, therefore the edge will not disturb the background, and will appear to be more prominent in the foreground objects.

a desaturation of the colors is exactly the method i was thinking but i wasn’t sure of the result, thanks for the blueprint and video showcasing this way to do it, i also found that uncharted 4 cel shading method is kinda the same process, is the reason why i didn’t want to use the 3 bars of colors to make it, because it turns out to be not friendly with any others lights like spotlight for example

and once again, thanks for your help, i’m gonna credit you on my game

edit: oh well, it didn’t work :confused: blackout, i checked everything and it seems to be fine

You can monitor the individual nodes by simply pulling the end pin to the postprocess output, so you don’t have to guess what is happening at a particular point in the sequence. Anyways, it should work, you pry forgot to set something up correctly. I am using a depth map in the version above, but you don’t have to do that, so just simply disconnect the multiply and divide nodes around the ceil node, and just use 16.0 as a constant value for both operators. I also am mixing back the original content at the Add and Div/2 nodes, but you can pull the end node of (x*norm) mul result instead and that’s it.

This only is a cheap way to set the luminance, but it’s far from accurate. It’s better actually to split the channels according to the YCoCg color space, and you should be able to modify only the luminance Y channel. The engine already have this colorspace conversion thing implemented, it pry just not exposed to materials. However, this simple math code can be easily fitted into a custom node, only the math stuff is enough to Float3 as RGB in, Float3 as YCoCg is out. And then convert it back to RGB before outputing it to the postprocess node. I implemented both conversions in custom nodes and i can freely modify the luma this way.

I’m using a different gamma curve here, but the colorspace conversion really helps to use the full range of the luminance.https://youtube.com/watch?v=90qPpMaVNps
The banding gets too distracting to me. The clouds and the distant city at the outside scene are simply just disaster. Maybe a different quantize algorithm that weights more to the actual color differences (rather than only the luminance) would be a better way to set this up, but in the end the textures could really be the key to make the inkish effect more apparent on the entire scene and you don’t really have to quantize the color informations at all, the sobel should be enough that way.

sorry to bother you, but i have checked like 5 times now, and i don’t see what did i missed :frowning:

Yes it looks all right, so it must be something wrong with the Custom node. I suspect that it contains no script but a 1.0f as a result, which makes this equation to produce just 0.0f for the multiplication, therefore the end result will be pitch black always. In the custom node you can write your own extraction of the sobel edges (using hlsl code syntax), but just fiddling with the SceneTexture:WorldNormal node you can get the same results as well (tho i’m fairly certain the custom node can perform faster in this computation).

i wish i could debug that part of the node so i can see what am i getting as result :confused: