Tutorial Preview: Putting Live 3D Models in your HUD

[Kneebiter]

I thought I'd post a quick teaser of a tutorial I'm working on right. It will show show how to put a live 3D model into your HUD. I'm hoping to get it finished and out sometime this week.

In the lower-left, you can see a TF2-style live character preview. Periodically, in the video, I bring up a larger-size character overlay. This larger overlay is the type of thing that you might use for a Skyrim or Rust style inventory system where the appearance of your character changes as you change which items are equipped.

In the tutorial project, we simply pick up colored spheres that change the character's color and reflect that in the HUD display, but the same basic approach I'll go over can be used to put any live 3D content into a HUD or any other UMG screen.

Note - this approach does use scene captures, so it has the potential to impact performance and raise your minimum system requirements.

[Cinebeast]

Very cool. I hope to see more of this.

[Innos]

Can't wait for this! Thanks!

[Eqric]

Was actually just about to create something similar to this but ran into problems on how to rotate the mesh without actually spawning it into the world. I captured the image from the Actor Viewport for that actor that is displayed.

[Yggdrasil]

Sounds like a nifty little feature. Think I will keep some attention to this.

[ZioYuri78]

Subscribed

[Naveed]

There's also this tutorial by @Tolc, https://engineunreal.wordpress.com/2...mg-widget-hud/, which does a great job in getting something similar. It uses a 2DTextureRender node. The only problem is masking out the background reflections on your static mesh.

Interested to see the approach used here.

[Kneebiter]

There's also this tutorial by @Tolc, https://engineunreal.wordpress.com/2...mg-widget-hud/, which does a great job in getting something similar. It uses a 2DTextureRender node. The only problem is masking out the background reflections on your static mesh.

Interested to see the approach used here.

The approach I used isn't all that different than the one you linked, actually. In fact, the main difference is that that tutorial makes it more complex than is actually needed.

I'm actually on travel right now unexpectedly, so won't be able to finish the tutorial for a couple more days. I'll give away the secret now instead of making you wait for the full tutorial.

There's a much easier way to mask out the background than in that tutorial-- which is to simply not have a background in the first palce. Instead of using a "room" with an emissive material that gets masked out, I placed the SceneCapture2D outside of the sky sphere, pointed away from the sky sphere. Doing that gives me automatic alpha masking for any part of the captured image where the background is visible on camera (though I do have to one-minus the alpha channel in my material) and saves me the work of building a "room" around the object. Anywhere that nothing is drawn on camera, the alpha automatically comes through as transparent alpha.

[Naveed]

Cue the face palms and why didn't I think of that Very kind of you to divulge your masking approach. Looking forward to seeing your tutorial once you're back. Safe travels.

[Chosker]

There's a much easier way to mask out the background than in that tutorial-- which is to simply not have a background in the first palce. Instead of using a "room" with an emissive material that gets masked out, I placed the SceneCapture2D outside of the sky sphere, pointed away from the sky sphere. Doing that gives me automatic alpha masking for any part of the captured image where the background is visible on camera (though I do have to one-minus the alpha channel in my material) and saves me the work of building a "room" around the object. Anywhere that nothing is drawn on camera, the alpha automatically comes through as transparent alpha.

but if you do this, you cannot use translucent alpha. it won't generate depth so it will not render on top of the void, so it just won't appear at all :/

[DjGrave]

pretty cool

[Kneebiter]

but if you do this, you cannot use translucent alpha. it won't generate depth so it will not render on top of the void, so it just won't appear at all :/

Hmm... I'll have to check that out and come up with a workaround if it's true. Shouldn't be hard to test.

[Naveed]

I tried putting the capture outside of the sky sphere and added the 1 minus to the material. I had an atmosphere volume that was still being captured so had to disable that and my skeletal mesh is captured beautifully in the UMG Widget. Since my skeletal mesh has no transparency it worked fine for my purpose. But the blanket alpha mask won't be a good solution to everything. And as mentioned earlier using an emissive room leaves some terrible reflections on anything you're trying to capture unless you swap out all materials for materials which have roughness = 1 and metallic = 0.

[John Alcatraz]

The approach is really interesting, that's actually a really good way to avoid having to mask with a color! Thanks Jeff Problem is just, I need the atmospheric fog... Is there anything I could use instead of "nothing" to get the same result as with using "nothing"? With atmospheric fog you unfortunately never have "nothing" to get the real nice perfect mask.

[John Alcatraz]

Ok, you just need to disable the fog in the SceneCapture settings

One real problem I found is that you can't use any Post Processing, so no Anti Aliasing, which makes it looks quite aliased. While you get a not-clean edge if you mask against a color, there you can use AA so the edge will be much smoother.