How to create customisable rim erosion and offset outline for mesh

[xeryvelgarde (Xer)]

Thank you to anyone who responds.

Is there a way to create an effect that looks like this, where the edges are eroded and there’s an offset outline but without the issue that I’ve circled in red and the fact that the outline doesn’t work at certain angles for boxes? Preferably

• erosion degree can be controlled
• erosion appearance can be controlled via texture
• base material would be independent
• offset of the outline can be controlled
• outline color can be changed for each mesh

MostHost has suggested me to create a topic on this to see if someone has already found a simpler solution.

Can't figure out how to use Vertex Color instead of VertexNormalWS to create rim offset and offset outline based on Stylised paint shader breakdown

Really, there is no reason not to use a postprocess effect for at least half of this. The outer outline is trivial in a PP. Near impossible in a shader (the shader doesn’t know what is around the object). As far as the effect itself. Its not exactly Edge Detection. That would be figuring out where the hard edge is, more so than figuring out where the mesh ends. Fresnel is nothing but the dot of the camera vs the object’s single tris normal (vertexnormalws). Basically, you are looking to i…

So far, for the rim erosion I’ve done this:

1. Base material
   
   
2. MF_ShellCalculation
   
   
3. MF_ColorRamp
   
   

and the goal was to create an inner shell that’s solid with the base color and an outer shell on top of that with the gradiant that acts as the erosion.

However according to MostHost you can do this with stencils. I’m not entirely sure how you would implement that though? And also using fresnel don’t really work well with flat surfaces from my understanding.

The second part is that I do want an offset outline as well. Temporarily I’m using an overlay material M_OffsetOutline but I’m pretty sure you want to do this in post-processing.

But for the post processing, PPM_OffsetOutline, I based it on Sobel Edge Detection in Unreal Engine, but how would you modify it so that you can change the color of the outline and scale it bigger than the actual mesh it’s outlining around. My understanding of custom stencils is limited, I get that you can use a value for the custom stencil, compare in an if statement and if it meets a certain condition, you assign a certain color?

How to Make Toon Outlines in UE5

Actually I’m going to follow this tutorial since it’s really in depth and try to modify it for my purposes.

[xeryvelgarde (Xer)]

Made a PP outline based on the video above. Currently it looks like this. It’s detecting edges, not gonna be able to detect pixels if they’re invisible and the rim erosion is a fresnel currently.

I think there’s two ways to go about it.

A.

de Laubier ended up using several static mesh instances with different custom data values to do the trick. So I think I’d have to do something similar. First instance would be for the inner shell, the solid part, the second would be for handling the outer shell, which is the rim erosion.

Because I already have a PP outline, I think I’d have to add the first instance/inner shell with its own custom depth stencil value (1), have the outline add colors based on that, then somehow offset/inflate based on it. However I have no clue how you would do that with pixels.

B.
Somehow make the rim erosion itself a PP effect. From what I’ve researched, you can get edge wears using Sobel or whatever edge detection algorithm, can’t remember where I found it but saw something similar about edge detection in general here, and I already have an outline PP material.

Theoretically couldn’t I just grab the outline I made, and instead of having a color there, somehow apply the rim erosion gradiant to it, play around with the alpha and scale of it to give that effect?

Still use the PP offset outline material though based on the stencil value the inner shell has then scale it up from there.

Update:
Tried using custom data values for rim erosion, the shader complexity is pretty high though. PerInstanceCustomData with Value = 1 and Index = 0 controls the outer shell. Also ignore the comment, that should be for the outer shell inflating and deflating.

Stylised Paint Shader (Custom Data) posted by anonymous | blueprintUE |...

Ah, about the inner shell and outer shell, , I think it’s actually supposed to be more reminiscent of shell based fur texturing. Looking at GitHub - Aqu-Hub/ShellBasedStroke for reference. Although it looks like with this method draw calls is expensive, but since I’m only doing an inner shell and an outer shell, hopefully it’s not that expensive? Although I’d have to test it.

[xeryvelgarde (Xer)]

Update:

Can't figure out how to use Vertex Color instead of VertexNormalWS to create rim offset and offset outline based on Stylised paint shader breakdown

After thinking a bit. The effect you eant is easily achievable with 2 shells, IF, you set the normals for the outside shell to be inverted - inside only. Ill give you a quick blender made example if I can… PP does it’s thing. UVs are on the object. the 2 are unrealated. You can tell a mesh to belong to whatever stencil you want. Most common use for this - with completely separate meshes in this case - is to hide water from inside a boat. Anyway. let’s get to the example here the model’s …

Can't figure out how to use Vertex Color instead of VertexNormalWS to create rim offset and offset outline based on Stylised paint shader breakdown

Are we talking about the output as in what we plug in for the Base Color? Sorry that I’m not getting it right away. Or this, because I think the issue might have actually been that I didn’t flip the blacks and whites of my detail map and I think that was it. Also made it two sided to see what it would look like. [image] [image] Thank you again, for your patience.

I also wouldn’t be surprised if it’s possible to do this using Substrate where you have 2 Substrate Slab BSDF, and then a Substrate Vertical Layer or a Substrate Horizontal Blend, slap the gradient into the Top Thickness or the Mix.

The problem would be how do we inflate the outer shell. There isn’t a WPO input but theoretically we should be able to do that.

[xeryvelgarde (Xer)]

How to create customisable rim erosion and offset outline for mesh

Update: I also wouldn’t be surprised if it’s possible to do this using Substrate where you have 2 Substrate Slab BSDF, and then a Substrate Vertical Layer or a Substrate Horizontal Blend, slap the gradient into the Top Thickness or the Mix. [image] [image] The problem would be how do we inflate the outer shell. There isn’t a WPO input but theoretically we should be able to do that.

For the outline, I’ve accepted the limitation with using WPO otherwise I think it’s just impossible:

How to create customisable rim erosion and offset outline for mesh

Thank you to anyone who responds. [image] Is there a way to create an effect that looks like this, where the edges are eroded and there’s an offset outline but without the issue that I’ve circled in red and the fact that the outline doesn’t work at certain angles for boxes? Preferably erosion degree can be controlled erosion appearance can be controlled via texture base material would be independent offset of the outline can be controlled outline color can be changed for each mesh MostHost…