I’ve been thinking of a way to get cell shading working in unreal without expensive post process effects, code plugins or forward rendering so I can take advantage of unreal’s deferred pipeline (multiple lights, etc.)
After much experimentation I came up with a method that alters the vertex normals of any mesh to achieve a cell shaded effect. It’s a bit of a hack and not quite perfect yet but it’s a lot cheaper than doing full screen post processing for just a character. Right now there are two different shading styles i’m torn between:
This model is really low poly and messy with pretty bad normals (It was a free download from some obscure site) so I imagine it will look better on a cleaner mesh. I haven’t got around to the outline stuff yet but that should be fairly straightforward I think. The model’s texture is a very basic flat color map, the cel shading simply comes from the vertex normal tweaks in the shader.
Hi bud!
May I ask you to share some hints about the tweaking of the vertex normals of your great cel shader?
I’m trying every possible way to have cel shading, both postprocess and material… I personally prefer to have full control on my object instead of using a postprocess. I now have a shader that performs the shading but, as you surely imagine, with just one light vector replicating the main directional light of the scene.
Be able to keep all the ligths would be fantastic and your system sounds an interesting idea
Thanks in advance for any help or suggestion you would like to share
Hi guys, sorry I totally forgot to come back and post my findings here. So basically my solution is not ideal visually but comes close to emulating that cell-shaded look without being too involved or restrictive.
Below is what I do to the vertex normals of my shader to achieve the look:
Simple way:
This gives you one subdivision of cell shading and is the most basic stylized look
This graph gives you control over the number of shading layers and also allows you to blend between the smooth normals and stylized ones:
so it’s not really true cell shading but a close emulation that works with any number of dynamic lights and light propagation volumes. This can also be used on lightmapped objects too but is less noticeable.
Overall I’m pretty pleased with the results - again this works with any lighting, LPV’s or static lighting, the only downside to this is that you can’t tweak how many cell shade levels you have but I think it’s a good alternative for a more typical cell shaded style:
This is incredible. I will definitely be looking into using this for my next project, as using custom scripts in the engine (which is what I’m doing now) can make things very complicated.
Having a fully working cel shaded system in vanilla unreal would definitely make things much easier.
Hi!
Your result is really good… I should give it a try
Mi solution at the moment was to take the abs values of the VertexNormals and use each channel as UV for a band/ink textures, make a new float3 from the 3 TextureSamplesand then multiply it for (VertexNormal/abs(VertexNormals)) to retain the sign of the components.
This simply makes all the VertexNormals as discrete values so the surface can react to the light as if polygons share the same normals… the final look is very similar to a cel shading.
Pros: it’s very easy to adjust number,thickness and smoothness of the ink-bands… And of course it reacts to all lights in scene.
Cons:it gives me strange result somewhere because it tends to generate squared shaded areas.
I don’t have the engine at home but I’ll study your solution and try it on mondey as soon I reach the studio
Great job bud!
Hi!
I’ve tried your solutions (both of them) this morning and my opinion is that your last shader gives you nicer results but very different to the previous in terms of “what polygons” are shaded and how.
Your method with the ShadingLevels parameters gives the mesh almost the same look of mine.
It’s hard to tell wich one is the most correct regarding to the lights in scene…
BTW, i’ts a great idea!
While this is a comparison between my code and yours (please note that now I’m referring to your “second to last” solution): as you can see the result is almost the same. I slightly prefer my approach because a texture for the ink levels allows you to control better the amount of inks, their intensity, transitions… and it’s possible too to not shade equally (you can reduce the dark part to a small section and then keep the rest light).
The look is good with some tweaking but the surface reacts to lights very differently in comparison with the other two systems.
So I’m not really sure about wich one is the best
I have the feeling to not have full control on the look of the object with your last idea… but maybe it’s just a matter of understanding better how to set parameters
Looks great and thanks for sharing! that’s a really interesting approach, I especially like how you implemented the use of a detail normal map and ramp for shading control - I think i’m going to merge some of our ideas and see what the result is!
The idea comes from this great tutorial about a cel shading material (but with a single and explicit light vector to fake a directional light) https://www.youtube.com/watch?v=9YQEAvvzh0k&t=1153s
Basically he uses a grayscale gradient as UV for the ramp texture, so I did the same thing to round the normal values: each channel value of the normal vector becomes one UV coordinate for the ramp.
I’ll try a guide :
Painting different areas in the ramp textures allows you to choose how to round the normals. So, instead of dividing the normal into 2, 3 or more equal steps you can adjust the ratio between dark and light areas and have smoothed transitions.
So a value for the normal of (for example) 0,27 as UV corresponds to, let’s say, a black color from the ramp (or 0)… or it could be 0,25 if you paint more gray bands instead of simply half black and half white.
Hope I was clear… sometimes I find quite hard to explain what I’m doing
I’m also having problems recreating the shader…
Those red nodes (Input Normal Map, Input Normal Map Multiplier and Input Inkband Texture), I couldn’t find it. Could someone tell me what kind of node that is?
Those nodes are “Input nodes” used inside a material function. My graph is actually inside a function because I like to encapsulate useful parts of code/nodes, use it elsewhere in different ways and share it between projects and people of the team
If you’re doing a simple material, you can use a standard texture sample with a normal texture inside instead of my input Normal Map. The other one (multiplier) is a float1 number while the last is a TextureObject Input: again, if you’re doing a material simply delete it and directly put the ramp texture inside the 3 sampler.
