(Tutorial) Simi CelShade PostProcess Material work with light color, point lights and skybox

I’ve been working on Cel Shading with PostProcess material I want to share my work with you guys.

If you want to** cel shade** in UE4, and also** get the light colors back**, also want to work with skybox, unlit materials and point lights and also don’t want to write your own custom shading model, I hope this thread will help you.

I don’t know if I’m too late to post this thread but I want to help people who try to cel shade with postProcess material after this thread.

Limitiations

Do not work well with specular, roughness, metallic and other reflection related things ( Don’t use those things in your material, set specularity to 0).
Do not work with near distance exponential fog. (Set its distance to large values like 5000 if you still want to use)
Do not work with god rays.
Depends on the brightness of the scene.
You better use fixed value of the auto exposure :min brightness = 10 and max brightness = 2

Overall Logic

We extract the desaturated lighting of the scene so we get the black and white color of the extracted lighting (0 to 1). According to that lighting, we can give the scene whatever shade we want depending on if the light is greater or less than 0.5. Then we add the original post process to the scene to get the light colors back. Then we out-mask the skybox with scene depth and give it original post process effect. You can use custom depth to mask the objects you want to cel shade. That’s all.

overall graph

Only the objects with “custom depth” property set to true will get the cell shade. If you want to apply the cel shade to your whole scene, remove the group “Objects with custom depth use it” completely and plug the output of “if” from the group “Get skybox color” into emissive.

First of all create a postprocess material, then chosse the Blendable Location to Before ToneMapping [SIZE=4](Super Important). If you don’t do this, tone mapping will disturb the process of extracting the lighting form the scene.[/SIZE]

d5d18a98184d9d136866269cb939b397075b07c2.jpeg

Light is multiply onto the diffuse color. So to extract the light from the scene, we have to **divide **the diffuse color from the scene. NOT subtract it. If you want to understand it well, plug it to the emissive.

So we get the black and white lighting to the scene. If the light is greater than 0.5, we give scene bright shade, if less than 0.5, we give the scene dark shade. You can replace the shades with whatever shade you want.

The parameter “darkness Factor” is very important. It decides whater to give dark shade or bright shade.
This method depends on the brightness of the light of your scene. If your scene is very low-lighted (below the value of darkness Factor), your scene will be covered with dark shadows. If your scene is very bright-lighted (above the darkness Factor), there will be no dark shadows.

So you have to adjust the darkness Factor value depending on the brightness of the lighting of your scene. If your scene is too bright-lighted, you increase the darkness Factor amount so that you can get the shadows back. If your scene is very low-lighted, you decrease the darkness Factor amount to very small value so that you get the bright shading back.

The group “light areas” is for bright shade and “dark areas” is for dark shade.

The parameters “Light_PP_blend_amount” and “Dark_PP_blend_amount” are the parameters that controls how much original post process will be blended onto your scene. If you want to get the light colors back, you have to blend the postprocess onto your scene. This method breaks the fully flat 2D anime look cel shading but it give you the light colors back and unlit materials. If you want to cell shade like fully flat 2D anime, than set those values to 0 and you will lose your light colors.

I think it is impossible to look like 2D anime, and you still want to get your light colors and you still want to do it with post process material.

The parameters “light area tint” and** “dark area tint”** are the ones that gives you bright shade tint and dark shade tint. Change “light area tint” to larger values so your light areas become brighter and “dark area tint” to smaller values so your dark areas become darker. As they are vector 3 parameters, you can tint bright shade and dark shade seperately with colors.

If you want to get the fully flat 2D anime cell shade, set the Light_PP_blend and dark_PP_blend to zero and you can fakethe lighting color by setting bright_area_tint and dark_area_tint with colors.

Create a material instance of this post process material, assign it to Post Process volume and play around with the parameters until you get the best result.

Out-mask the skybox with scene depth multiplied with 100000 because the skybox is the most far-away object in your scene and it’s scene depth is always 1.

The objects with “custom depth” property set to true will get the cell shade. If you want to apply the cel shade to your whole scene, remove the group “Objects with custom depth use it” completely and plug the output of “if” from the group “Get skybox color” into emissive.


**By only using this celShade post process material, you will not get the perfect beautiful cel shade scene like magic.
Stylized rendering needs artistic skill and the beauty of it depends on the colors you choose, your textures, objects and details you add into your textures.

To get the best flat look play around with Contrast, Saturation, Crush Highlights, Bloom and Tint Color in your postprocess volume.**

I hope this thread helps you.

7 Likes

Awesome stuff, I’ll have to give this method a try. Thanks for the knowledge. :slight_smile: Would be interesting to see if the fog and other effects can be worked back in too… The fog is really needed to push the depth of the scene, looks a little too flat in the Cel-shaded screenshots so far.

I think you have to create a fog yourself with post Process material. Local Fog (in Volume?) - Rendering - Epic Developer Community Forums

Thanks to share this with us Thura Oo <3

Thank you! This is awesome!!!

Unfortunately our result was problematic. Even if we use plain colors and textures we get some problems around the borders of the light. I have little knowledge about lighting, maybe you would know what causes this?

I’ll post a screenshot…

Thanks!!!

http://i.imgur.com/RmfOBxI.png

It is awesome man, wish I had found this couple months ago so I didn’t have to pull my hair out to create a cel shaded look in one of my projects at work, lol. Now I may go back and give them a visual boost :))

Sorry for my late reply. Pls make sure you have set “Blendable Location” in PP Material to “Before Tonemapping”.

Hi, thanks for answering!!!

Blendable Location was already set to Before Tonemapping. Here is a screenshot of our graph, maybe there is something wrong here??

Your graph is correct. Do you use noisy normal maps? Noisy normal maps, reflection related things (specular, roughness, metallic) can cause such problem. If you don’t use them, I can’t think of any other possible solution to that problem.
How about if you test with simple cubes from engine and if the problem still cause, post screenshots?

I’m not sure it’ll work on mobile graphics.

Thanks for sharing this, I was able to utilize it with some great results:

Cheers!

Does someone know how could I create a setup for more than one Custom Depth effect at the same time?
I have more than one interaction that uses the buffer but all of them mix together.

How could I separate the post processes in different idexes or something like that?

Quick question, This looks really amazing in my current scene and I’d love to iterate on it to make everything pop, the one thing that I can’t fix which is a big part of the scene currently is my water. I’ve been using a translucent with depth fade and subsurface scattering coming from the sun.

Will I have to compromise in this regard and only use a flat shaded material to make this work or do anyone else have any tips to make a stylized water in the vein of Rime, Breath of the wild or this revision of my translucent water I had a few weeks back http://imgur.com/a/pA4C8
Really ANY knowledge of this will be super helpful as my deadline is in less than a week and I really like the effect this PP has!

Edit: I remembered the TemporalAA node, just plugged in a random 0.7 vector 3 node and it gave it this extremely grainy effect but it works, I asssume I have to use TemporalAA since my project is set in VR and translucency is normally to heavy to have. How would you set something like this up to still “fake it” but achieve som sort ok visual fidelity without the lag that comes with it.

Cheers

Does this method work with VR Forward rendering?

Sadly no, But I am here to ask if anyone knows how to get it working with Forward Rendering? If anyone knows how it would be sooooo incredibly appreciated. I would looove to use this on the game I’m working on right now.

Sorry for the newbie question, but after I created the material like you did, I don’t understand where I should put it to see it work on the scene.

Hi Cyberglobe, I’ve spend some time figuring it out myself so if you’re still struggling here it is:
You just have to create a PostProcessVolume and search for settings->blendables. Add a new element and choose “asset reference”, now you can drop your material into it.
If you’re on 4.16 it’s a bit more obvious, the “blendables” column have been renamed to Post Process Materials or something like that, much easier to find :slight_smile:

Thanks LucienG!
After some research I found out it was there, but thanks anyway for the answer :slight_smile:

When using this method, my colors are getting really washed out. Any advice?



Hey all, I seem to be having a bit of an issue trying to get this to work. From what I can tell, my issue stems right at the beginning where we are extracting the light info. For some reason, when I check the output of that division node, everything turns white, meaning that no lighting is actually extracted. No idea on what is wrong or how to fix. Any advice?
06d5e90c99758c82cd52a13b0ef547b66aa602af.jpeg

Hey. I have a problem, like the Knobbynobbes579a7d98f19d68b6cec744d9ffe2af1fdd74eafe.jpeg. The black and white image is extracted, but the effect does not work. What could be the reason?