Render Screen-Space Widget Behind Character?

[FrozenHarlequin]

I’m currently trying to make Life is Strange-style UI elements that always face the camera, are placed at a specific location in the 3D space of the level and that are rendered on-top of every 3D element except for the player character.

My strong preference would be to use a screen-space widget for this since I’ve run into some isues with world-space widgets (see below) but I have yet to find a way to render the character mesh in front of a screen-space widget. I attempted to put my character mesh into a custom depth pass and use a custom stencil in the widget material to mask out the area he occludes but depth passes and scree-space widgets don’t seem to work together. Is there any way to do this using blueprints?

(Widget material blueprint. The Custom Stencil works fine when widget is set to world space but not when set to screen space.)

---

Alternatively, I have been able to sort of achieve the effect using a world-space widget by disabling depth test in the material and adding a custom depth pass for my character mesh and some simple blueprinting to keep the widget aligned with the camera. But there are some pretty big caveats:

1. The widget can intersect with my character since it exists on a 3D plane in 3D space. It doesn’t look too great. Now I could make the widget vanish whenever it gets too close to the character mesh but I’d rather not have to do that.

(Widget intersecting character - please ignore the placeholder character model.)

2. When using a world-space widget, TAA is applied to it just like it would be to a normal game world object. This leads to impaired legibility and ghosting whenever you move the camera. It’s not pretty and basically makes world-space widgets unusable with TAA. My attempt at a workaround was to put the widget into its own Custom Depth Pass und use a post-process material and a Custom Stencil to bypass the post-processing pipeline for the widget. That works but the problem now, of course, is that there’s pretty significant aliasing on the widget and I can’t seem to get rid of it.

This does not seem to be an issue with screen-space widgets which are neither affected by AA, nor show aliasing. (Which is probably obvious to anyone who has a better understanding of how screen-space stuff is rendered vs world-space stuff than I do.)

(Aliasing on widget after post-process material has been applied. It doesn’t look that bad in the screenshot but looks terrible in motion.)

(Post-process material.)

---

I have also attempted using a Text Render component but font materials are set to masked, meaning you cannot use the Disable Depth Test function on them. When I change the blend mode to translucent, I get pretty terrible motion artefacts on the text so that doesn’t seem to be an option, either.

---

So yeah, I guess I’m asking if anyone has any ideas on how I might be able to solve any of these issues? Or is there another way I might be able to get the desired result that I haven’t thought of yet? Either way, I’d be incredibly grateful :).

[FrozenHarlequin]

Okay, so after experimenting around with it for another day which nearly brought me to the edge of my sanity, I’ve realised that, essentially, there’s no way to have a screen-space object look like it’s behind my 3D character. I’ve reverted back to my world-space solution and tweaked it. It would be incredibly handy if Epic could implement some sort of FXAA node for use in post-process materials. As it is, I’ve had to resort to very low-radius Gaussian blur applied to my very noisy, heavily aliased custom stencil. The result is acceptable but certainly not perfect. If anyone else stumbles across this question with similar issues, I’ll try to sum up what I’ve done:

1.Create a widget with your text/graphic/whatever.

2.Create a blueprint actor and place the widget inside. Make sure it’s set to world space.

3.Duplicate the Widget3DPassThrough material that should’ve been automatically applied to your widget and check “Disable Depth Test” on your new copy.

4.In your character blueprint, select your character mesh and check “Render CustomDepth Pass”. Set “CustomDepth Stencil” value to a number of your choosing and remember that number.

5.In the copy of the Widget3DPassThrough material that we just created, use a SceneTexture node to access the custom stencil you just assigned your character mesh to. Multiply this stencil with the nodes that were plugged into the opacity input of the material and plug the result of this multiplication into the opacity input instead to mask out the screen area that will be occupied by your character.

6.Go back to the blueprint containing your widget, select the widget and apply the new copy of the widget material that we just created to it.

7.The widget still selected inside the blueprint, check “Render CustomDepth Pass” and, again, set a CustomDepth Stencil value, as we did with the character mesh. Just make sure it’s not the same value you used on your character.

8.Create a post-process material, make sure that “Blendable Location” is set to After Tonemapping. Inside the material, create a SceneTexture node and set it to PostProcessInput0. Connect it to the A input of a Lerp node. Create a constant 4 Vector and set it to the colour you want to use as your UI/text colour (alternatively, you can use textures, gradients, noise, etc. if you don’t want your UI to just be one flat colour).

9.For the Lerp alpha, I resorted to a custom Gaussian Blur shader node by Tommy Tran which can be found here: Unreal Engine 4 Custom Shaders Tutorial | raywenderlich.com . You need some sort of blur because the stencil on its own will have some very heavy aliasing on it. I modified the shader slightly to work with the custom stencil by changing SceneTextureId to 25 and replacing line 12 (starts with float3 PixelColor) inside the GaussianBlur node with this:

float3 PixelColorA = SceneTextureLookup(Offset, SceneTextureId, 0).rgb;
float PixelColor = 0;
if (PixelColorA.r == 2)
{
PixelColor = 1;
}

You will need to replace the 2 with whatever Stencil Value you used on the widget as this bit of code filters out the widget stencil (otherwise the colour fill and blur will also be applied to the character stencil which we don’t want).

10.Run the result of the Gaussian Blur node through a Component Mask with only the R channel enabled before passing it into the Lerp Alpha. Plug the result of the Lerp into the material’s Emissive. Adjust the Radius value until you get acceptable results between not too blurry and not too aliased.

11.Don’t forget to place your blueprint actor in your level in the spot where you want the widget to show up.

I also had to set “CustomDepth Stencil Write Mask” on my character mesh to “All bits” for things to be sorted properly but I haven’t played around too much with the other options to figure out if sth else might also work and if you’re using stencils for other tasks, using the “All bits” option might interfere with those. If you want it to look even more like a screen-space widget, you can do some additional blueprinting inside your blueprint actor to dynamically adjust the rotation and scale of the widget according to camera rotation and distance. So yeah, for now, without being able to write my own shader code, I think this is the best I’ll be able to do. Hope it’s helpful to someone.

[Shrooblord]

Hello past human!

Hope it’s helpful to someone.
Very very much so! Exactly what I was looking for. Thank you so much for the detailed write-up.

What did catch me off-guard and was a couple hours of debugging was this almost throw-away line in Tom Looman’s post:

Custom Stencil is disabled by default, to enable go to Window > Project Settings > Rendering > Post Process > Custom Depth-stencil Pass and set it to Enabled with Stencil.

Don’t forget that! Even if your Custom Depth is Enabled, you need to set it to Enabled with Stencil!! Crucial difference!!

One more thing I did differently: when I followed your tutorial to the letter, I noticed it was my character mesh that got coloured in the UI colour, indicating that the stencil was doing the exact opposite of what I wanted; I changed around which object I gave which stencil (i.e. your tut says to give the character #2 and the UI something else, eg. #3; but I did #2 for the UI and #3 for the character). Works a charm!

Might’ve misunderstood what you meant and just made a fool of myself, but if it helps a future human…!

Furthermore, in UE 4.25.3, I was able to use the ‘Default’ instead of the 255 bits option you mention at the end, but also not super sure what you meant with ‘sorting’ so maybe I’ll run into that problem later down the line and return here to correct myself. x)

P.S.

Some references for you friendly folks:

Get custom shaders (‘/Project/Shaders’ “folder”) in Material Editor to work again in 4.25.3:
https://forums.unrealengine.com/development-discussion/rendering/1562454-virtual-shader-source-path-link-custom-shaders-shadertoy-demo-download

Tom Looman’s post on Custom Depth:

Multi-color Outline Post Process in Unreal Engine

(Note that the ‘PP Volume > Settings > Blendables’ instruction he mentions is most likely now the PP Material part of the PP Details panel, i.e. where you assign your PP Materials; Blendables is old nomenclature I guess!)

[Shrooblord]

Hmm, nope! It seems I was fooling myself because of the Material I was using for my character mesh, which is translucent. When I applied an opaque Material, the character mesh behaved quite the same as the rest of the geometry: the 3D Widget overlaps it, like the custom depth buffer wasn’t even used.

Back to the drawing board! Will reply again if I figure out a solution.

Incidentally, here’s what I have currently: 3D widgets showing how they both occlude an object in the background (that’s to be expected), as well as an object in the foreground (behaving like a generic UI element). You see how my ‘character’ model (i.e. a large hand) passes through the UI element, but not in front of it. Instead, the custom depth buffer ‘reveals’ part of the ‘UI Colour’ node you show in your post, exactly where the Mesh intersects the UI element. I feel like I’m very close, because this would seem to be the exact opposite behaviour from what I want. Now all I have to do is figure out how to… flip it.

[Shrooblord]

Okay then. Not sure if this will help anyone else, because I’m not even sure yet if it helps me. But, I made a fluke discovery that seems to work in my particular use case.

If you take a look at the post-process Material that is being created for the Widget, where you mask out the CustomDepth stencil of your Mesh from the UI’s opacity, you notice the constant ‘1’ that is being passed into the bitmask? If you change that constant, and set the Custom Depth Stencil of your mesh to that number (i.e. constant set to 2, and CDS of Mesh set to 2 also), then your Mesh will draw over the UI, even when the UI is occluding other Meshes (and even if they are using Custom Depth).

Note that in this case, I have removed the Post Process Material from my PP Volume entirely! I’m not running a PP Material on my Mesh anymore; I’m only using the CustomDepth attribute on that Mesh, and only running a CD Mat on the UI element.

And here’s the result:

Let’s analyse this picture:

1. The UI renders over objects in the background
2. The UI renders over objects in the foreground; in this case, that object is even using a Custom Depth itself (Tom Looman’s, to be precise)
3. The Hand renders over top of the UI object, as is the intended effect.
4. [Not shown]: The Hand does not get shown rendered ‘behind’ occluding objects, unlike with my previously described approach.

What is interesting here, is that when I switch back to my translucent Material, it also masks the UI as if it was an opaque Material. It isn’t quite what I expected, but it does make sense when I think about exactly what I programmed in.

I’ll do one more pass of experimentation to see if I can combine the two effects: proper occlusion like with the opaque Hand, but using a transparent Material for my mesh.

[Shrooblord]

Aaalllright one more from me. Got it working!

Here’s the final result:

As you can see, the opaque Hand correctly occludes UI, while the UI still occludes normal objects as it should.

When the Material of the Mesh is translucent, the Custom Depth Buffer doesn’t care about it, and so the translucency works as expected: overlaying itself rather than ‘cutting out’ the UI below it.

I’ll detail the full process now for total clarity:

1. Created a Blueprint with a 3D Widget in it. Browsed to the Material it was using (a Material Instance), and browsed up the parent chain until I had the ‘Widget3DPassThrough’ Material they were derived from. Duplicated that, and am now using it as my base for my own Material.

2. Edited the Material like so:

Key here is to note the constant going into the bitmask – this is the Custom Depth Stencil that this Material is ‘listening’ for. I’ll probably parameterise it and turn it into a Material Instance so I can edit this on the fly in Blueprints.

My modification is to test if the Custom Depth is ever smaller than the Scene Depth. If it is, we’re dealing with a translucent Material, and so we skip the part where we cut out our UI in the places where it overlaps our Custom Stencil (i.e. choose 1 in the multiply node if A < B).

3. Note that ‘Allow Custom Depth’ is true for both the UI Material and the one on my translucent Hand (I’m using CD for backface culling on the Hand; unrelated to this).

4. In the Character Blueprint, turn on Render CustomDepth → Stencil value equal to the constant going into the bitmask (254 in my case).

That’s it! No need for PP Mats, although you can layer them on top, like I have done. Only one Mat for the UI element, and only turn on CD settings for the character.

[Shrooblord]

Posted my solution to these comments as a separate answer. See below!

[Shrooblord]

A slight curiosity is when there’s an opaque mesh occluding the UI which is being occluded by other geometry: the UI occludes the geometry as expected, but because the mesh is occluded by the geometry too, it doesn’t occlude the UI in the same places as before. So you get this strange cut-out / overlay effect.

Anyway, it works a charm for my purposes.

[syrcon (Kortiplex)]

The custom shaders for 4.25.3 link seems to not work anymore, but this seems to be the updated one: Virtual Shader Source Path - Link custom Shaders - Shadertoy Demo download