If anyone’s been following my previous posts, I managed to finally get things rolling with somewhat of a workaround in overcoming IMHO certain serious Decal shortcomings in UE4 by building my own.
However I seem to have now run into yet another problem and i’m looking for some help to find the culprit.
Anything above or below or in the Decal setup is displaying strange edge artifacts. The artifacts consist of jagged edges as well as border edge outline of whatever color the decal texture is.
Basically any object hovering over the custom decal box (in this example below the cube) also gets the same artifcating. Also for the record the cylinders inside the decal box are excluded via custom stencils.
I am for the record using Forward renderer with MSAA.
I had this problem with stencils 2 months ago. Try changing when the postprocess material to be blended in the pipeline - Before Tonemapper or Before Translucency. See screenshot for ref.
Thanks for trying to help! The thing is I am not using any post process materials here, I have a custom material setup which is unlit with modulate blending mode for the Decal. and in the mat I am simply masking out the stencils. Is there another step I need to be doing here?
Also its not just the stencils that are causing this, as you can see any object hovering over the Decal box is artifacting and those are not part of any stencils.
Ahh, I just misunderstood you saying Stencils that I interpreted as custom stencils which you can use in postprocess to mask out individual objects. So you are talking about decals mainly.
Ok, I dont know what kind of challenges you had before so I cannot help you specifically (do you have a link?). But anyway could you try to copy the project and change it to deferred rendering and use temporal aa just to see if you see this edge problem? Also why forward shading? Is this project for VR?
Ok will try out with TAA but its useless for us even if it works there since all our project is in Forward. it did occur to me that it maybe something with antialising, not sure if aliasing happening on one of the passes.
As to why forward well for multiple reasons:
the project may have a VR option but its not VR.
Our objects on screen are small and fast moving we noticed immediate loss of detail on all elements and blur when switched to test on Deffered with TAA.
MSAA seems to be doing a fine job for us and only forward supports this.
Loss of detail in particle effects and other areas with differed TAA, also noticed more texture blurring.
Ok, I just wanted to see if its really necessary for your project. And for the TAA thingy its just to see if the problem comes from the AA solution and not from something else.
Can you give me link to your previous thread? It could give me some clues about your problem because I can see that there is some kind of jaggy black “outline” that happens on the box which is very weird. So I am curious on what kind of workaround you have done. Could be that this is the source of the problem.
Oh man I was reaaally hoping you wouldn’t say that! Whats the solution or there is none with forward, any idea if Epic is going to have a fix for this, I mean isn’t this kind of an issue?
Edit: Suppose i’m going to ignore using the stencil passes through that custom scene depth node entirely, is the rest of the setup which i assume is using just the scene depth still going to be affected by this? Isn’t the scene depth already anti aliased by MSAA?
Edit2: I tried removing the custom scene depth node and all stencils, the issue is still there. not sure if i’m doing something wrong or it is currently a limitation within the engine.
I am not using a decal material it is a custom one, will this method still apply? Then again if its not ant-ialiased with MSAA as Deathrey suggests I don’t see how.
Below is my current material setup that works except for the artifacts mentioned here, which are for the most part sadly a deal breaker to me if there is no solution.
These “Decals” were meant to cover large portions of a map as you can imagine anything moving over them will artifact, just saying its not simply meant for small bullet holes.
Ok, I have some theories because I cannot replicate your material setup in a normal decal material. Why do you need to use decal? I mean if you will visualize only one explosion texture and not multiple, why not using a postprocess material that have a world aligned node for the explosion texture and use the custom stencil to mask the objects out. Then you can do that the postprocess is blended before tonemapper/translucency to fix the jaggy/jumpy edge that happens with custom stencil.
Ok, I’m not sure I understand what you meant by using post process material, wont that be projected through the camera only? Any examples out there that can help?
1 - I do need to have multiple such effects on screen, there are many units exploding and should leave these decals on the landscape behind.
2 - I am also using this method to project large scale shadows onto the terrain.
3 - Also using it for other smaller shadows.
In order to replicate and make the above material work, simply apply a black and white texture to the texture node and apply the material to a cube.
You can project textures with world cordinates inside PP, but you will use multiple explosions so the idea of PP material is out.
I meant I cannot replicate the material because in 4.22 its impossible to get modulate on decal material, only translucent. I have an idea tho, can you try to mask the stencil like the screenshot I uploaded instead using the IF node and connect it to a new lerp instead? Just for a test.
Thanks, will try what you suggested, but my mat is not exclusive to 4.22 and should work in earlier versions. I am not using “Decal” material setup, check the bottom right corner of the screen, it is an unlit surface material with modulate.
Edit: Also keep note that its not just the stencil masks causing this, I removed them entirely and the issue persists with any object hovering over the effect.
Since no other nodes related to Scenetexture is used in the mat, my main culprit could be the node “Worldpositionbehindtranslucency” I would like to know if there is an issue going on with this node and MSAA/forward if utilized this way.
Edit2: Sometimes I wonder if I end up using Quake 2 engine for my game I would’nt have half the problems i’m having with state of the art engine in 2019!
Hmm, I see, you just repeated a lot “decals” and I just automatically changed it to Decal surface mode haha. But yes, I have no idea what could the problem be. I am sorry that I cannot help you.
I can confirm this is the case, scene depth is also not very clean, I did temporarily switch ti to TAA and the result was cleaner, however I can’t use TAA its messing up my setup visually. So I suppose the only solution left would be to have upsampling. I have a feeling upsampling would solve this issue at whatever cost it will imply.
Its too bad though, I really o hope Epic doesn’t neglect forward in the future and will keep amending and adding new features to it, most importantly fix these shortcomings somehow.
Well yeah, the scene depth you are sampling is already resolved texture and operation is performed per pixel, while it should be done per sample. There are no straightforward ways of doing it gracefully.
