Announcement

Collapse
No announcement yet.

[TUTORIAL] Edge Detection Post Process Effect using World Normals

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

    [TUTORIAL] Edge Detection Post Process Effect using World Normals

    Hello fellow Unreal Engine users!

    This is not really a tutorial in that I guide you through every single step. Instead I'm going to throw my Material asset (including comments) at you for everyone to use, adjust and play around with.

    Note that the following images were captured using the Vehicle Game Sample/Template/Demo which is made by Epic Games. I've only created the post process material and applied it to their content/game.

    So before I'll give you the material setup, you can see the post process effect in action if you like.


    Or just enjoy these postcards, err... I mean screenshots.
    Click image for larger version

Name:	bridge.jpg
Views:	1
Size:	530.7 KB
ID:	1165856

    Click image for larger version

Name:	lake.jpg
Views:	1
Size:	496.9 KB
ID:	1165857

    Click image for larger version

Name:	pyramids.jpg
Views:	1
Size:	513.4 KB
ID:	1165859

    Click image for larger version

Name:	temple.jpg
Views:	1
Size:	501.3 KB
ID:	1165860

    Click image for larger version

Name:	walls.jpg
Views:	1
Size:	520.2 KB
ID:	1165861


    So here is the actual material I created:
    Click image for larger version

Name:	NormalEdgeDetectionMaterial.png
Views:	1
Size:	254.9 KB
ID:	1165858

    As you can see it does not need many nodes. This is because most stuff is hidden in the Custom Node.
    This node uses three inputs which you'll have to set up. "offset", "UV" and "scale". The Output Type is CMOT Float 1. The following is the code used for that node:
    Code:
    //kernel size
    int sizeX = 5;
    int sizeY = 5;
    
    //offset added to all uvs to get from the center to the corner
    MaterialFloat2 baseOffset = MaterialFloat2(-sizeX/2.0f, -sizeY/2.0f) * offset.rg;
    //sample the world normal from the GBuffer
    float4 baseNormal = SceneTextureLookup(UV, 8);
    //dot product for two normalized vectors ranges between -1.0f to 1.0f
    //we're going to search the smallest value so initialize to something larger
    float minDot = 10.0f;
    
    for (int i = 0; i < sizeX; ++i)
    {
    	for (int j = 0; j < sizeY; ++j)
    	{
    		//calucalte clamped uvs
    		float2 uvs = min(max(UV + baseOffset * scale + i * offset.r * scale.r * float2(1.0, 0.0f) + j * offset.g * scale.g * float2(0.0, 1.0f), float2(0.0f, 0.0f)), float2(1.0f, 1.0f));
    		//sample world normal from the GBuffer at the new position
    		float newDot = dot(baseNormal, SceneTextureLookup(uvs, 8));
    		//if smaller than our current value
    		if (newDot < minDot)
    		{
    			//use the new value
    			minDot = newDot;
    		}
    	}
    }
    
    //return smallest dot product in this region
    return minDot;
    There are a few more things you'll need to set up before you're ready to go. The first is to set the Blenable Location for the main material node to "Before Tonemapping". The second step is to create a Material Instance from this Material (you can right-click it and choose "Create Material Instance" in the Content Browser) which allows you to control the edge thickness as well as how fast an edge is detected. Finally you just have to make use of the Material Instance for example by adding it to the Blendables of a PostProcessVolume.


    I did not spend alot of time improving it, so it has a few issues. If used with TXAA some edges flicker. On the other hand some edges look worse without TXAA.

    It's been over a year since I initially created this with 4.2 so some things may be different and require some adjustments in other versions.
    Youtube - UnrealEverything

    Games:
    [Android] World of Bricks Released!

    [TUTORIAL] Edge Detection Post Process Effect using World Normals

    2B || !(2B)

    #2
    What exactly is that first element, the TexCoord? Sorry, I am new to all of this.

    Comment


      #3
      All it does is call your UVs as a float2. You can manipulate your UVs using that node.

      Comment


        #4
        Not working in 4.12: "Error [SM5] Material.usf(1257,21): error X3013: 'SceneTextureLookup': function does not take 2 parameters".

        Comment


          #5
          Originally posted by bakelite View Post
          Not working in 4.12: "Error [SM5] Material.usf(1257,21): error X3013: 'SceneTextureLookup': function does not take 2 parameters".
          you need to replace the "SceneTextureLookup(UV, 8);" in the custom node with "SceneTextureLookup(UV, 8,true);" (or use a false if you don't want filtering (got this from the HLSL output of another shader))
          It would be better to provide the scene lookups as input parameters, but the second one may not be easy.
          Does anybody know that the node version of the lookup is?

          I made a slight improvement, I added the unroll specifier to the loops:
          Code:
          //kernel size
          int sizeX = 5;
          int sizeY = 5;
          
          //offset added to all uvs to get from the center to the corner
          MaterialFloat2 baseOffset = MaterialFloat2(-sizeX/2.0f, -sizeY/2.0f) * offset.rg;
          //sample the world normal from the GBuffer
          float4 baseNormal = SceneTextureLookup(UV, 8,true);
          //dot product for two normalized vectors ranges between -1.0f to 1.0f
          //we're going to search the smallest value so initialize to something larger
          float minDot = 10.0f;
          
          [unroll]
          for (int i = 0; i < sizeX; ++i)
          {
          	[unroll]
          	for (int j = 0; j < sizeY; ++j)
          	{
          		//calucalte clamped uvs
          		float2 uvs = min(max(UV + baseOffset * scale + i * offset.r * scale.r * float2(1.0, 0.0f) + j * offset.g * scale.g * float2(0.0, 1.0f), float2(0.0f, 0.0f)), float2(1.0f, 1.0f));
          		//sample world normal from the GBuffer at the new position
          		float newDot = dot(baseNormal, SceneTextureLookup(uvs, 8,true));
          		//if smaller than our current value
          		if (newDot < minDot)
          		{
          			//use the new value
          			minDot = newDot;
          		}
          	}
          }
          
          //return smallest dot product in this region
          return minDot;
          Never say Never, Because Never comes too soon. - ryan20fun

          Frames Per Second is NOT a proper performance metric, You should use frame time. You can read this or this as to why.
          (Profiling) Tools: RenderDoc (UE4 Plugin), NVIDIA nSight, AMD GPU PerfStudio, CodeXL
          Good articles/Series: A trip through the Graphics Pipeline 2011

          Comment


            #6
            Hey there, this is great! Easy-to-follow tutorial and I got it working in a few mins
            I'm wondering how I would use this method with scene colour as well and not just world normal, but I don't have a single clue how HLSL code works..
            and scene depth, that would be great too. I'm assuming its something embedded in that custom node, but again no clue.

            Visit my portfolio: portfolio.jhgrace.com

            Comment


              #7
              Hey everyone, sorry it took me so long to reply.


              Originally posted by ryan20fun View Post
              you need to replace the "SceneTextureLookup(UV, 8);" in the custom node with "SceneTextureLookup(UV, 8,true);" (or use a false if you don't want filtering (got this from the HLSL output of another shader))
              It would be better to provide the scene lookups as input parameters, but the second one may not be easy.
              Does anybody know that the node version of the lookup is?
              Thank you for fixing the error and the [unroll] optimization.

              The node version should just be SceneTexture:WorldNormal like the one used in the screenshot of the material above (the color pin).

              You may be able to provide the scene lookups as input parameters yes, and yes it would probably also be better to do it that way.
              Since the intention of the function was to only be used with the world normal, I hardcoded the scene texture index that represents the world normal: 8. You could probably make another input parameter that replaces this and specifies which scene texture index you want to use. The problem with that is you won't know what that value means and where it comes from unless you dig it up or have it commented/documented somewhere. But using the dot product on those values probably has a different meaning than what was intended.



              Originally posted by Construc_ View Post
              Hey there, this is great! Easy-to-follow tutorial and I got it working in a few mins
              I'm wondering how I would use this method with scene colour as well and not just world normal, but I don't have a single clue how HLSL code works..
              and scene depth, that would be great too. I'm assuming its something embedded in that custom node, but again no clue.
              Thanks for the kind words!
              I currently do not have the time to write a follow-up about how to do edge detection on scene color or scene depth. But this https://udn.epicgames.com/Three/Deve...Detection.html may get you started with a scene depth edge detection material. Do note that the link is on the UDN and for the UDK but most of the nodes should be very similar (does not require any HLSL but you could condense it a lot with it). You could also just go ahead and replace the "8" in the HLSL code with some other number. But be warned, you may get really strange results since it really is only meant to be used with a value of "8"
              Youtube - UnrealEverything

              Games:
              [Android] World of Bricks Released!

              [TUTORIAL] Edge Detection Post Process Effect using World Normals

              2B || !(2B)

              Comment


                #8
                Hey

                Great tutorial on normal edge detection! Plus the postcards look fantastic.

                I'm currently trying to implement it myself and my edges aren't being rendered smoothly at all like in your video. I have attached my post process blueprint below. I am using PostProcessInput0 as I get the following error with SceneTexture:
                Error [SM5] (Node SceneTexture) SceneColor lookups are only available when MaterialDomain = Surface. PostProcessMaterials should use the SceneTexture PostProcessInput0.

                I know this post is dead but any and all help is appreciated!

                Click image for larger version

Name:	Vector Edge Detection.jpg
Views:	1
Size:	321.6 KB
ID:	1114301

                Comment


                  #9
                  Originally posted by INeedFerrets View Post
                  Hey

                  Great tutorial on normal edge detection! Plus the postcards look fantastic.

                  I'm currently trying to implement it myself and my edges aren't being rendered smoothly at all like in your video. I have attached my post process blueprint below. I am using PostProcessInput0 as I get the following error with SceneTexture:
                  Error [SM5] (Node SceneTexture) SceneColor lookups are only available when MaterialDomain = Surface. PostProcessMaterials should use the SceneTexture PostProcessInput0.

                  I know this post is dead but any and all help is appreciated!

                  [ATTACH=CONFIG]107799[/ATTACH]
                  all you need to do for that is change the scene texture option in the menu to PostProcessInput0 instead of SceneColor

                  How exactly do I implement the code for the custom node? I keep getting errors

                  Comment

                  Working...
                  X