Problem with TAA + Contact Shadows + Masked Material

Development Rendering
1

I’m trying to solve a problem getting these three systems to play together nicely and I’m hoping someone might have some advice.

I have a UI element, placed in the world that moves with a character. I am using TAA and the UI element shows far too much ghosting/tearing along its edges if it uses a Translucent material. I’ve turned on Responsive AA but that has not made an appreciable difference. Fortunately I am able to significantly improve this by switching the material to use Masked blending.

Unfortunately I recently turned on Contact Shadows, which work great for the rest of the game, but interact with the UI material in an unintended fashion. Because I’m using Masked blending, it’s writing to the depth buffer, causing the raycasting for the Contact Shadows to pick up the UI and create a shadow from it (irregardless of shadow casting settings on the mesh and the fact that the material is set to Unlit). The shadow is very undesirable and if I can’t get rid of it I won’t be able to use Contact Shadows.

I tried improving the TAA’s effect on translucency but had little success with this. For example I tried a Translucent material that worked as a masked material, clamping the alpha to 0 or 1, and that did nothing.

I looked at the code in ShadowRayCast that does the raycasting for the Contact Shadows thinking that I could ignore unlit pixels but I couldn’t see a way to get the GBuffer data from there for arbitrary pixels and realized that this wouldn’t work anyway for a number of cases.

I looked for a way to simply set the Masked material to not write to depth (a feature I would generally expect) but there is nothing in the Editor to do this and looking through the code it was not obvious how I might add that.

Does anyone have any other ideas?

2

Just a note. The depth buffer is probably what fixes the TAA ghosting for you.

3

Hrm. Unfortunate but makes sense.

Any advice on making fast moving, thin, translucent objects look better? I’ve looked at some of the TAA improvements you’ve posted on here but those seem to focus more on maintaining detail on the interior of the object than preserving the edges IIRC.

4

I’v had a similar problem, the solution for our project was to modify engine slightly (just the PostProcessing.cpp file) and reorder separate translucency to be composed AFTER motion blur & TAA pass, this way we have layer that is unaffected by those effects and could display clear UI/HUD with alpha blending on it (all translucent materials marked as ‘after DOF’ are rendered into this layer).
This is not ideal (since particles not affected by DOF are also not anti-aliased), but works quite well, and antialiasing for alphablended objects can be done by hand anyway.

5

Thanks. This sounds like a very promising avenue, do you have any more context on the changes you made to PostProcessing.cpp? I took at stab at this but it was tough going figuring out what controlled the ordering of these passes.

For now I’ve moved to using custom depth/stencil and a post process pass after tonemapping. This is working out “ok” though now my UI is heavily aliased due to the stencil buffer not having a concept of transparency and bit of multisampling in my post process material is only somewhat mitigating the issue.