Trying to make Toon Shader Material on UE4

Development World Creation
21

So Allar came to the rescue and was able to walk me through Sobel Edge Filter, which applies the transparency differently than the one I was using. It works really well, check it out here!!

Also I was able to get mine working, I couldn’t turn it into an opacity, everything was black with white outlines. But Allar pointed out that if I multiply white by screencolor I’ll just get screen color. So if I use x-1 on the filter I was using, then multiply it by screen color, it’ll apply it without any need for a transparency. Woo! Go Allar!

22

donno if this was already linked, but perhaps you’ll find some interesting info in here to help you out. Engine Feature Examples for Unreal Engine | Unreal Engine 5.1 Documentation

23
  • The actual files the document references hasn’t been released yet and the blueprints / material editor screens aren’t complete.
24

Hi Jv,

How were you able to only affect certain objects with your post process effect? The characters in the foreground appear cell-shaded but the sky does not - its a neat effect! I tried to replicate the shader using the method you described, but ultimately my skybox is also cell-shaded (which also looks cool, but wasn’t desired)

Thanks,

25

Found a way to achieve the effect I desire - perhaps others will also find it useful. It’s a bit hacky, but the only way I can think of at present.

I enable custom depth buffer on objects I want to cell shade. I perform an if statement in the material, and if custom depth buffer < depth buffer I use a cell shaded approach, otherwise default to normal approach.

This allows me to selectively choose which objects will be cell shaded. The chair on the right in the image renders to custom depth, while the one on the left doesnt.

http://s29.postimg.org/46b8y4dab/chairs.jpg

(Click for larger)](http://postimg.org/image/46b8y4dab/)

Cheers

26

Hello all. Long Time no see.

So I actully have the Emisive working (kinda).

http://s3.postimg.org/b5bznuylb/UE4_Cel_Shader_Post_Processing_2.jpg

And remember I was going to let y’all figure it out how I did this? I’m going back on my word with it (Because its a bit hacky).

http://s3.postimg.org/6i5xm3b8f/UE4_Cel_Shader_Post_Processing_mat1.jpg
http://s26.postimg.org/j305a421h/UE4_Cel_Shader_Post_Processing_mat2.jpg

(note: If you having trouble finding something like SceneColor or world normals it is coming for the SceneTexture)
On the cel shading part I re-did the lighting. I doted The Scene Color and the world normal and subtracted with a desaturated diffuse color. What that does it it made sure that I am removing any color that is coming out of the Scene Color and Just give us the light that is bouncing off the mesh. If you were to Subtract the Scene Color and the Diffuse color you will only get what was the light is getting from that mesh.

Then I just added the bands to it by just putting a Ceil to it divide by a Constant Scaler (which is just a float number ) and then Clamp it with a min of 0.1 and a max of 1. When I ceiled the light, I’m setting the value to a whole number up so if you have a float that is 0.3 It will go to 1(We’re not rounding it. we’re ceiling it). Light only have a scale of 0.0 and infinity (0.0 meaning no light and infinity mean a lot of light) by ceiling it we are setting the light to only have whole number. When I divide that Ceil to a constant number of 1 I’m telling it to only have one band of light to show. And when I Clamp it to only have a min of 0.1 I’m telling it to brighten up the shadows.

Then I multiply it with the Diffuse Color (or the Color Base, They both work). This is just to bring back the colored material that was applied to it.

When placed it into the emissive color and click the apply button everything work except the emissive color and specular that was applied on the mesh.(For example The sky light dome and the tech hex title didn’t appear). That is because I also got rid of it while I was trying to get only the light coming for the light and the diffuse color and base color doe not apply any emissive that was in the material. So I tried to bring them back in to it.

I did it by, Multiplying the Scene color and lighting model and then multiplying it with the specular color. then Desaturated it and then multiply it by the scene color. This only bring back any specular and emissive that was gone. It not the best way because it also got rid of the sky light. However I have no other Idea on how to bring it back fully.

Then I brought the skylight back by just inversing (oneMinus) the Material AO and Multiply the Scene Color. It s a hack way of bringing it back but it also got rid of my ability to to have opacity on certain material (if I want to do some type of fading I’ll have to figure out another way.)

Finally, I added the Cel Shade with the Emissive/Specular and the Emissive Sky to get the result I have now. I’m still working on ways to make it better. I’m Looking at EillotB way to used the Custom Depth Debuffer on the material.

I wonder on How ElliotB was able to do the if statement. I tryed it by placing the Custom scene into it but it telling me I need a Float.

27

Great tutorial, but it the resulting PP doesn’t seem to work well with point lights at all. I’ll try work on a workaround and report back.

28

Hi Jv!

Here’s how I’m performing my if statement:

b19f3f989ed487f63a30a52a0d3ec84dd0d41025.jpeg

You should wire your cell-shader emissive into the A >= B node of the if. Click an object, and under Rendering tick ‘Render Custom Depth’ (you may need to expand the menu to see it!)

Hope this helps :wink:

Cheers,
E

edit: that attachment is terrible quality, here’s another link:

http://s29.postimg.org/4grtafsd3/ifstatement.png

29

ToonShadingTest.jpg

Just posting a fun comparison shot of me playing around with post processing :smiley: Figured someone would enjoy looking at this. Most things (smoke, sky, FX in general) broke because I was using the initial algorithm. Just looked at the 2-3 last post and will do some fine tuning later. Still fun to look at though =]

30

actually if you switched the the Scene color to Post Process Input) you should be able to bring them back in.

Althought I have no Idea why they are in the PostProcessInput0 or what does the PostProcessinput Really do to get the particle.

31

Here’s what mine looks like now using sobel edge and photoshop to cartoonify my textures.

809×675 98.3 KB

What I want to figure out next is how to make the shadows thicker. Like a cartoon. Less fading and quality, more like bland single shade shadows. Anyone know what I could do?

32

Hi everyone,
I’ve been following this thread quite religiously since it started. I currently have JW’s cel shader working, although obviously not ideal. For instance, the edges of the “bands” tend to be a bit grainy, among a couple other issues.

I’m wondering if anyone has had any revelations or improvements they would be willing to share?

33

Hello everyone,

I made a Mistake on the band part. It was just brightening the light but not adding more bands to it. I changed it to make the bands work better.

http://s4.postimg.org/sz9k4e91l/UE4_Cel_Shader_Post_Processing_3.jpg

Whats I changed is that I’m now Multiplying the number of bands first then ceiling it, then finally dividing to the number of band again. What that does is it intensify the light (Darkening in in darker places and lighting it in lighter places) before it gets Ceiled, and after that it get divided to tone down the brightness of the bands.

If your also using the Sobel edge material, You notice that the Cel-Shade gets wash off a bit to where you didn’t notice them at all. What I did to get around it is that I multiply the clamp with scale parameter just to intensify the band brightness. That way it should not disappear completely.

One more thing, if you were playing the game you notice some unwanted bloom or motion blur. if you have the poss process box and have all of them check off it will still leave the default active. You may want to check on some of them and bring them to 0 just to tell them I don’t want to see it.

This probebly Does not answer the grainy part, nut I’ll talk a look into it.

34

Thanks Jv!
Any luck with the point lights?

35

If you use a ceil this will only intensify the light; it wont make it darker in darker places. It will still result in the distinct bands though, which is the effect you are after.

It’s quite common to use a ramp texture instead of using ceil (see eg here: TF2 Shading in UT3 tutorial - Unreal Tournament 3 - ModDB look at the section labelled ‘ramp’)

36

Thank you for clearing up on ceil. I was thinking of it as moving it up to a whole number.

Looks like Using a ramp would be my next thing I want to implement.

37

Also make sure to set your Blendable to before Translucency, otherwise your cell shaded objected will be rendered infront of your particles :stuck_out_tongue_winking_eye:

38

It is actually completely possible to create a Light Vector in the UE4 editor, I’ve done it and it’s working.

The difficulty is if you want multiple lights, this would be possible as well, but a little difficult. It involves using the insanely powerful new Material Parameter Collections, and using a blueprint actor in the level to continuously (well, if the light is ‘moveable’ or ‘stationary’, otherwise you only need to fill the values at begin play) update the vector parameter with the vector of the directional light, or whichever dominant light you choose. On a per-actor basis, you could even calculate the light vector for multiple lights in the blueprint before outputting it to a parameter in the shader.

Post process is probably slightly cheaper computationally, but it’s good to know that the old methods CAN be made to work!

To see a working example of this, open the blueprint, and the material for the BP_SkySphere in engine assets, or start a default level and grab it from there.

39

Exactly - hence why it will make things brighter, because intermediate brightness levels always get rounded up and never down.

40

Hi Jamsh,

What are the advantages of using a light vector versus post processing? I believe the post processing is actually more visually pleasing because it can incorporate the particle lighting new to ue4 (eg character’s shadows flicker with a flame nearby), but maybe I’m overlooking some other use case? I’d like to know.