Announcement

Collapse
No announcement yet.

Toon Shading Models, Stylized Rendering Experiments

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

    Toon Shading Models, Stylized Rendering Experiments



    Github fork link: https://github.com/SaimonMa/UnrealEngine
    Example Project: https://drive.google.com/open?id=1fv...DYWZXLN4pHneS2


    Hello! I just wanted to share some custom shading models I've been working on. I'm aware of the shading models that have already been posted on the forum (1) (2), but I wanted more control over the shading on a per-material or per-pixel basis, so I decided to fork the engine, teach myself HLSL, and take a stab at it. The idea for these models is to have soft indirect lighting but controllable direct lighting so it's possible to mix the default materials with the custom stylized ones. Note that the code does include a version of stororokw's anisotropic shader ported to 4.20 since I wanted to build on top of the anisotropic shading to create a toon version.

    I'm going to post my upcoming experiments as I try making some manga-style stuff in it, but I'd appreciate any feedback.

    Since Offset is a parameter in every Toon material, you can drive it with some screen-oriented masks for halftone or crosshatch effects:


    Also, materials have separate controls for specular and diffuse, which I find useful for stylized stuff:


    If GBuffers weren't limited, I could have just made a ubershader with all the features in one shader. Since there's not many unused GBuffer channels, here's a couple shading models that (hopefully) cover a wide range of uses, and their controls:

    Toon:
    General-purpose toon shader. Most control over the shadow and highlight locations.
    • Offset: controls the location of the shadow terminator. For faces, a vertex color channel or a texture can be used to push shadows away from the front of the face, or to increase how quickly the shadow appears in crevices, etc.
    • Roughness: controls the range of the shadow terminator. Roughness has been decoupled from specular. Max hardness for the terminator is reached at 0.5 roughness. Reflections are still rendered and will be visible at values below 0.5, if you want them.
    • Shadow Lightener: a float that lightens the shadow towards the base color of the material.
    • Specular Offset: Same as Offset, but for the specular highlight.
    • Specular Range: Same as Offset, but for the specular highlight.

    Skin:
    Skin toon shader with an adjustable SSS color. SSS calculation is based on the Subsurface shading model.
    • Subdermal Color: Takes a color. Note that the value for this color drives the strength of the effect, not the Opacity slot like in the standard Subsurface shading mode. I recommend multiplying your color or color texture by a scalar for easy iteration. If you are using a texture to drive the color of the SSS, I recommend multiplying it by a scalar, especially if you want really low SSS values. You'll get more control this way without texture compression crunching your values too much at low value ranges in the texture.
    • Offset: Same as Toon.
    • Roughness: Same as Toon.
    • Specular: For skin, the specular intensity and sharpness is linked, higher specular values also tighten the specular lobe, and vice versa.
    • Specular Offset: Same as Toon.

    Hair:
    Based on the built in Hair shading model for Unreal, so it will be fairly similar- though it lacks a Backlit parameter (since we're out of GBuffers again...). Has dual anisotropic highlights.
    • Roughness: Same as Toon.
    • Specular: Same as Toon.
    • Specular Lobe2 Strength: The second lobe is broader and more saturated. This parameter drives the strength of it
    • Tighten Specular: Higher values will make the specular highlight narrower.
    • Scatter: Works the same way as the built-in Hair shading model. Increase this value to lighten the hair.
    • Tangent: Takes the first two values of a vector to assign the anisotropic direction of the hair in UV space, same as the built-in Hair model.

    Toon Anisotropic:
    Anisotropic shader based on this stororokw's code, but ported to 4.20.
    • Roughness: Same as Toon, but also drives the range of the specular highlight.
    • Offset: Same as Toon.
    • Specular: Same as Toon.
    • Anisotropy: Controls the amount of anisotropy of the material.
    • Anisotropic Rotation: Rotates the anisotropy of the material.
    • Anisotropic Roughness: Drives the roughness of the anisotropic specular independently from the material's lighting range.
    • Aniso Tangent: 2D vector that controls the Anisotropy using maps like these. This page shows what kind of data the shader expects.

    Some of the known issues include:
    • odd behavior with point lights with inverse square falloff and area lights with 0 width and height dimensions. I'm not sure what causes this, so if you have insight it would greatly help. Current workaround is to make sure point lights have inverse square falloff turned off.
    • for Toon shading model, having a metallic material makes the specular color unable to to be turned off. I'm sure I can figure out where it is multiplying metallicity by specular color somewhere, so I'll fix this in the future.
    • completely untested with the forward renderer, but it almost certainly going to be broken. If you're interested in this and want forward rendering support, I would appreciate some pointers in the right direction as to what files to poke around in.


    Also, I couldn't have done this without these guys' tutorials paving the way:
    Last edited by doomfest; 12-04-2018, 07:14 PM.
    https://twitter.com/Doomfest

    #2

    Although I haven't had time to clone and compile your code, I have simply browsed it.
    Your implementation is very comprehensive, the ideas are quite advanced, and the choice of parameters is very elegant. Your understanding of the problem is quite profound.

    Therefore, I am very willing to provide the following suggestions, I hope these suggestions can help you.I have made a list of the problems that I have encountered that have been resolved and have not yet been resolved
    • Character Face Shadows (resolved): Added a Custom Material Node to modify normal in the engine.
    • Skylight, indirect light, and reflection are not stylized (resolved): The effect of normal on skylight, indirect light, and reflection is removed in the source code.
    • Outline and inner line (resolved): Add Custom data control node, specifically use two Custom data to control the post-process material.
    • Self-shadowing accuracy (resolved): Use ray-traced capsule shadows instead of shadow maps, combined with normal edits.
    • Character receiving shadow accuracy (resolved): Use a decal instead of a shadow map (remove the fade-out of the deferred decal X-axis).
    • Character contact shadow (unresolved): Shadows on small areas of the character, such as the projection of hair on the forehead.
    • Stylized eyes (unresolved): Porting some properties of the default eye material to the Toon Shading Model, adding a number of controllable fake highlights.
    • Stylized Perspective (unresolved): Achieve a few common perspective errors in Anime.
    • Stylized metal (unresolved): Makes metal objects in the scene more stylized.
    • Stylized edge light (unresolved): Rim light around static objects in Anime.
    • (And more to discover)
    Click image for larger version

Name:	image_139282.jpg
Views:	2
Size:	99.1 KB
ID:	1537466

    For a long time, I think the main reason for making 2D look like 2D is its motion.No matter how stylized rendered static images are done, as long as they move, they will look like 3D, like this one.http://artineering.io/articles/Art-d...-in-real-time/

    This concept has never been broken until I saw this dumplinghttps://skfb.ly/6AHqV
    If I can understand why this dumpling can look like 2D even in the rotation, I feel that I have mastered a big secret in stylized rendering.

    There are really not many people who care about Toon Shading. Toon shading lacks standards and requires a lot of art experience to control. The need for a lot of art experience means a lot of control, and the problem of insufficient GBuffer is easy to happen.
    ​​​​
    All in all, your implementation is already leading the entire UE4 community, looking forward to your new progress.
    Attached Files

    Comment


      #3
      Interesting.

      Since it appears you're making large use of the gbuffer how much functionality is lost under forward rendering?
      Last edited by Selentic; 11-09-2018, 01:50 PM.

      Comment


        #4
        HEYYYYYYYYYYYYYYY are you team shimapan?

        Hi

        Comment


          #5
          Looks amazing. I'm always looking for new ways for toon rendering and there are many nice looking versions around in the marketplace, but they all lack the same functions: In no version you can't really put an object into a really dark room so that the object is basically hidden in the shadow (I guess due to the fact that every post process is using the albedo color only and thus the renderer has no chance on applying "realistic" shadow values on it ). Second problem is fog with outlines. So far, every plugin renders the outlines on top of the volumetric fog, although it would need to dissapear/fade out in the fog as well. If those two features would work, that'll be amazing

          Comment


            #6
            doomfest
            Thank you very much for sharing.

            I built the latest code (verified it had the latest changes submitted on Nov.7ish), but the editor fps is roughly 1fps.
            Is it possible there are some other changes you haven't committed to github? The latest seems unusable to me.

            Comment


              #7
              @kusogaki77
              I have not found any possible reason for the frame rate to be so low. I suggest you use GPU and CPU Profiling to see what it takes to be too long.

              CONSOLE:
              Code:
              ProfileGPU
              stat SceneRendering

              Comment


                #8
                Originally posted by IOchair View Post
                Although I haven't had time to clone and compile your code, I have simply browsed it.
                Your implementation is very comprehensive, the ideas are quite advanced, and the choice of parameters is very elegant. Your understanding of the problem is quite profound.

                Therefore, I am very willing to provide the following suggestions, I hope these suggestions can help you.I have made a list of the problems that I have encountered that have been resolved and have not yet been resolved
                • Character Face Shadows (resolved): Added a Custom Material Node to modify normal in the engine.
                • Skylight, indirect light, and reflection are not stylized (resolved): The effect of normal on skylight, indirect light, and reflection is removed in the source code.
                • Outline and inner line (resolved): Add Custom data control node, specifically use two Custom data to control the post-process material.
                • Self-shadowing accuracy (resolved): Use ray-traced capsule shadows instead of shadow maps, combined with normal edits.
                • Character receiving shadow accuracy (resolved): Use a decal instead of a shadow map (remove the fade-out of the deferred decal X-axis).
                • Character contact shadow (unresolved): Shadows on small areas of the character, such as the projection of hair on the forehead.
                • Stylized eyes (unresolved): Porting some properties of the default eye material to the Toon Shading Model, adding a number of controllable fake highlights.
                • Stylized Perspective (unresolved): Achieve a few common perspective errors in Anime.
                • Stylized metal (unresolved): Makes metal objects in the scene more stylized.
                • Stylized edge light (unresolved): Rim light around static objects in Anime.
                • (And more to discover)
                Click image for larger version  Name:	image_139282.jpg Views:	2 Size:	99.1 KB ID:	1537466

                For a long time, I think the main reason for making 2D look like 2D is its motion.No matter how stylized rendered static images are done, as long as they move, they will look like 3D, like this one.http://artineering.io/articles/Art-d...-in-real-time/

                This concept has never been broken until I saw this dumplinghttps://skfb.ly/6AHqV
                If I can understand why this dumpling can look like 2D even in the rotation, I feel that I have mastered a big secret in stylized rendering.

                There are really not many people who care about Toon Shading. Toon shading lacks standards and requires a lot of art experience to control. The need for a lot of art experience means a lot of control, and the problem of insufficient GBuffer is easy to happen.
                ​​​​
                All in all, your implementation is already leading the entire UE4 community, looking forward to your new progress.
                Thanks for the feedback!

                I'm working on a test case that I will show when it's ready, but here's some things to reply to some of the listed suggestions:

                Metallic Gbuffer is already used in some toon shading modes to drive toon parameters, just not the default toon shader to the left. True Anisotropy actually requires 3 channels to control - two for a anisotropy direction and one for the level of anisotropic. I guess if you do some encoding you can use the magnitude of the first 2 vectors to get an anisotropic direction though, but then i'd have to pull the extra channel from something else and lose precision in another control. For this reason I just used another shading model for anisotropic toon shading.

                IIRC custom outputs cannot be collapsed into MaterialAttributes, so you can't wrap entire materials into functions for certain modular material workflows.

                face shadows: see following post on vertex normals (guilty gear xrd method)

                character receiving shadow accuracy: don't really see this as a problem in this particular implementation or my test case

                Contact shadows: stylizing the AO by messing with levels in texturing or using Simplecontrast node seems to do the trick.

                I've done a bit of work on toon eyes, but i'm not convinced it needs a separate shading model to achieve. In my test case i feel that you'd want to stylize the primary highlights on the eyes instead of making them accurate to the light direction anyway.

                Stylized perspective: this is better handled through rigging and camera animation. (see the guilty gear xrd gdc talk)

                stylized metal: todo

                stylized rim light: high exponential fresnel masked with AO seems to do the trick

                For vertex normals for faces and other parts where less shadow complexity is desired, I have done some tests with custom vertex normals in maya. will post a video/pictures soon once I am able to work on an example. I'm not convinced modifying normals in the material is the best way to go since the data can be stored in the vertices. If you need to store them in the material you'd have to either store them in a texture or vertex color channel(s) for example. There is already an offset parameter for moving the terminator too, so custom vertex normals would mostly be to simplify the edge on the terminator.

                Keeping the skylight and indirect lighting in the rendering is a stylistic choice, so I'm going to keep this in. Might add a flag later though. I think using a separate lighting channel and solid color skylight for characters would also do the trick, but I'm not certain unreal lets you use multiple skylights for multiple lighting channels. Will test this in the future. EDIT: Just been told skylights don't support lighting channels. Have an idea on how to encode the switch into metallic or specular channel though.

                outlines: in my testing, post process outlines that operate on contrast (depth contrast, world normal contrast, etc.) don't really work for important objects like characters since you get completely inconsistent silhouette outlines based on the context, which hurts readability. It's also more expensive than just using mesh outlines. It's better to handle this in maya (or your preferred 3d program) since you can pick which meshes you'd like to use the inverted shell method for outlines, then drive the width in the engine with world position offset. I'm also using vertex colors to drive line weight and opacity. Internal outlines should just be handled in texture. Maybe filter it with fresnel if you feel the style needs it.

                I don't really understand the dumpling example but yes, I feel that sometimes using hold keyframes and forcing animation to something like 10-24 fps also helps. (guilty gear xrd does this)
                Last edited by doomfest; 11-15-2018, 07:40 PM.
                https://twitter.com/Doomfest

                Comment


                  #9
                  Originally posted by Selentic View Post
                  Interesting.

                  Since it appears you're making large use of the gbuffer how much functionality is lost under forward rendering?
                  Unfortunately, probably everything for now. I know you're Sel on discord but I"ll answer here anyway so other people can read the answer. Will probably port to forward in the future.
                  Originally posted by Selentic View Post
                  HEYYYYYYYYYYYYYYY are you team shimapan?

                  Hi
                  :monkaS:
                  Originally posted by Erasmus View Post
                  Looks amazing. I'm always looking for new ways for toon rendering and there are many nice looking versions around in the marketplace, but they all lack the same functions: In no version you can't really put an object into a really dark room so that the object is basically hidden in the shadow (I guess due to the fact that every post process is using the albedo color only and thus the renderer has no chance on applying "realistic" shadow values on it ). Second problem is fog with outlines. So far, every plugin renders the outlines on top of the volumetric fog, although it would need to dissapear/fade out in the fog as well. If those two features would work, that'll be amazing
                  This doesn't use a post process shader to achieve the lighting, it adds lighting models to the deferred renderer. So dark room should be possible. The base code for this also doesn't add outlines by itself, I recommend using a combination of mesh outlines and post process outlines like they do in Honkai Impact. (there's a chinese presentation somewhere) Not sure about fog. I'm sure I'll run into this problem in the future so I'll post my findings here if I discover a way around the issue.
                  Originally posted by kusogaki77 View Post
                  doomfest
                  Thank you very much for sharing.

                  I built the latest code (verified it had the latest changes submitted on Nov.7ish), but the editor fps is roughly 1fps.
                  Is it possible there are some other changes you haven't committed to github? The latest seems unusable to me.
                  Hey, please send me a PM here or DM on twitter (I think you're the same person right?) so I can try to help you troubleshoot this issue, since it seems like it might take a lot of steps to figure out.
                  https://twitter.com/Doomfest

                  Comment


                    #10
                    doomfest Thanks for all the help on Twitter. I found the prob, and it was what you surmised. It was a prob with the DDC.

                    Actually, there was NO DDC designated for the engine.

                    For those who don't know the full story, I ran out of space on my initial install drive, so I moved all the engines/launcher/directxredist I had installed/built, and linked them there from that initial location. It appears that when it's linked, the DDC folder won't get created because the relative path can't be used?

                    Solution:
                    I went into this ini, located here, <ToonshaderModelsEngine_name>\Engine\Config\BaseEngine.ini, and specified a custom DDC folder under the [DerivedDataBackendGraph] section like this: Local=(Type=FileSystem, ReadOnly=false, Clean=false, Flush=false, PurgeTransient=true, DeleteUnused=true, UnusedFileAge=34, FoldersToClean=-1, Path=I:\Unreal\DerivedDataCache, EnvPathOverride=UE-LocalDataCachePath, EditorOverrideSetting=LocalDerivedDataCache), originally it was set to this "Path=../../../Engine/DerivedDataCache", and that fixed it.

                    Long story short, if your launcher and project's fps is sheit (like 1) it's possible you don't have a valid derived data cache folder specified.

                    Comment


                      #11
                      If you're willing to go to the trouble, submitting this as a fork request to the master build would be a massively awesome thing to do, as Epic could potentially roll this into their release builds. I'm certain a ton of developers would be hugely grateful, myself included. I plan on experimenting with this either way, as the current solution I use for toon shading works, but has some limitations that are challenging to reconcile, and I don't have the HLSL chops to come up with something as robust as what you've made here.

                      Absolutely phenomenal work, thank you so very much for this amazing community contribution.

                      Comment


                        #12
                        Here's the text version of the Honkai doomfest gave me a link to the other day (I just ran it through google translate, and it was completely intelligible.)
                        Here's a video version of the Honkai presentation for efficient npr, anime-style effects. It's professionally dubbed in Japanese, but you can enable auto-generated English titles which seem to be sufficient.

                        There's lots of good info in it. They made a bunch of great design decisions to make it run nice and look good on PC and mobile.

                        doomfest
                        Is there currently a way to change the offset (from shaded to non-shaded)? I mucked with the offset and it seems you can only adjust the NoL from the default .5,
                        to anywhere between .2625 or .7375 (originally you had it at .25 to .75 limits I think).

                        I don't think it's possible, seeing as all the gbuffer custom channels are taken, but do you know if it's currently possible to somehow change the number of shaded steps?
                        It seems like it's only shaded or not, currently.

                        Also, though I doubt it's possible, on a related note I think it'd be really cool if one could specify a lut for customized shading band thickness. Arnage's shader had it so each shading band/step had the same thickness. Maybe we could use the normal's alpha channel as a lut? heh If one could specify a lut to determine shading step/band thickness, maybe we could mess with the UV mapping on the lut to stretch it to the far left (black, no bands), or far right 1 band in full light (effectively like unlit). They did fully black, fully lit/unlit in old school Dragonball, and it has the effect of accentuating emotion in some situations.

                        Comment


                          #13
                          Originally posted by hippowombat View Post
                          If you're willing to go to the trouble, submitting this as a fork request to the master build would be a massively awesome thing to do, as Epic could potentially roll this into their release builds. I'm certain a ton of developers would be hugely grateful, myself included. I plan on experimenting with this either way, as the current solution I use for toon shading works, but has some limitations that are challenging to reconcile, and I don't have the HLSL chops to come up with something as robust as what you've made here.

                          Absolutely phenomenal work, thank you so very much for this amazing community contribution.
                          Thanks! As far as I know epic hasn't approved a pull request for toon shading ever, but maybe if we make enough noise they'll consider it. I do need to make a couple adjustments to this before I think it's ready for a pull request though.

                          Originally posted by kusogaki77 View Post
                          Here's the text version of the Honkai doomfest gave me a link to the other day (I just ran it through google translate, and it was completely intelligible.)
                          Here's a video version of the Honkai presentation for efficient npr, anime-style effects. It's professionally dubbed in Japanese, but you can enable auto-generated English titles which seem to be sufficient.

                          There's lots of good info in it. They made a bunch of great design decisions to make it run nice and look good on PC and mobile.

                          doomfest
                          Is there currently a way to change the offset (from shaded to non-shaded)? I mucked with the offset and it seems you can only adjust the NoL from the default .5,
                          to anywhere between .2625 or .7375 (originally you had it at .25 to .75 limits I think).

                          I don't think it's possible, seeing as all the gbuffer custom channels are taken, but do you know if it's currently possible to somehow change the number of shaded steps?
                          It seems like it's only shaded or not, currently.

                          Also, though I doubt it's possible, on a related note I think it'd be really cool if one could specify a lut for customized shading band thickness. Arnage's shader had it so each shading band/step had the same thickness. Maybe we could use the normal's alpha channel as a lut? heh If one could specify a lut to determine shading step/band thickness, maybe we could mess with the UV mapping on the lut to stretch it to the far left (black, no bands), or far right 1 band in full light (effectively like unlit). They did fully black, fully lit/unlit in old school Dragonball, and it has the effect of accentuating emotion in some situations.
                          I clamped the offset because it looked a bit odd at the maximum values, but now that you mentioned it, I realized some artists would want to be able to remove the shade altogether at the maximum value. I'll fiddle with it so that the shadow can be completely removed if the input is maxed out (offset = 1). I'm working on some other stuff all weekend though, so I'll handle this on Monday.

                          Unfortunately there isn't a way to change the number of steps currently. Even if there's enough gbuffer channels left to dictate the number of steps, there's still the issue of being able to adjust the offset and thickness of each additional step. I noticed you can sort of cheat and add a third shade if you use an ambient occlusion texture with sharp edges- this obviously isn't dynamic but depending on your style, sometimes it's good enough. Ambient occlusion gets masked by light, so for areas receiving direct lighting, it doesn't show up.

                          Can you give me an example of what you mean by the effect used in Dragonball?

                          The issue with LUTs is that deferred lighting only has the GBuffers to work with. For LUTs to work (essentially a color ramp shader), the material has to pass a texture reference to the deferred lighter, which isn't possible since the only information materials can pass to the lighter is in the form of GBuffers. Ideally the perfect solution is to add an RGBA Gbuffer that is an atlas of all the ramps used in the scene and the alpha channel is a buffer for telling the lighter which pixel uses which tile in the atlas, but this is obviously a lot of work and outside of my capabilities at the moment. Another way it might work is if there's some kind of gbuffer for LUT reference and a big list of texture references somewhere... Someone did get a (grayscale) LUT to work in an older engine version (http://monsho.blog63.fc2.com/blog-entry-161.html), but the problem is that it just forced a texture to always be loaded and everything in the scene always used the same LUT, which isn't particularly useful, as a perfect implementation for this use case would require each material to be able to specify at least one LUT, IMO.
                          Last edited by doomfest; 11-16-2018, 10:21 AM.
                          https://twitter.com/Doomfest

                          Comment


                            #14
                            kusogaki77 brought up a couple use cases where being able to fully put a character in shadow or light using the material would be useful, so I extended the range of Offset to allow for this. (thanks!) This is the new range for Offset, note that if you've made materials you might need to update your scalar values for offset. if you used a texture to drive offset, I recommend using a Lerp to adjust the range of offset for that texture (I use that on my test cases):

                            https://twitter.com/Doomfest

                            Comment


                              #15
                              doomfest
                              Thanks a lot for all the info. You make it very easy to understand.
                              I'll try messing with the AO input to achieve a form of stepped shading in the meantime.

                              [as a perfect implementation for this use case would require each material to be able to specify at least one LUT, IMO.]
                              Same here.

                              I'll look for some dragonball references. I'm pretty sure other animes, like Bleach, do it too, it's just that dragonball gave me the most lasting impression.
                              (A couple months ago, I saw all of them from the beginning of DGB to the end of DGBZ.)

                              I'mma update to your latest on github. Thanks a ton for the help!

                              Update: That offset you implemented works great! No artifacts when lerping between the extremes.
                              Question: The toonhair model seems to be ultra sensitive to light sources, though. Are you seeing that? Maybe my setup is wrong, but whenever the toon hair material is "lit" by more than 1 source, I get a bloom effect unless I put the light intensity to something like .001 on the additional source.

                              Fully lit reference (2 lit characters are stressed/shocked, the normal shaded character is reporting the facts)

                              Fully shaded reference (First time shenron is shown is in silhouette, and then gets lit up)
                              Attached Files
                              Last edited by kusogaki77; 11-19-2018, 12:57 AM.

                              Comment

                              Working...
                              X