Performant 3D UI

Development Programming & Scripting UI
, ,
1

World Space UI utilizing UMG is far from performant.

3rd Party Plugins resolving this space fall short, they fail to keep with UE versioning, fault in performance or provide dated forms of authoring.

Off the cuff our intuition was to render 3D Planes and reconstruct a portion of the UMG paradigm. We rapidly prototyped this utilizing plugins such as Geometry Scripts and Procedural Mesh. Creating dynamic and static elements for the broad set of use cases (Text, Layout, etc). The next step would be to roll an authoring tool to support this approach.

The divergence from out of the box UMG is a significant choke point, yielding more to the problem space. Taking a UMG authored widget and converting it results in a broader problem space; Material Domain Change, MVVM usage, Sequencer, Stylization, Blueprint logic, Interaction, the list extends beyond a reasonable scope.

We touched on the topic with Rendering Engineers at Epic recently, their suggestion was to use in world planes and render materials directly on those planes; the same intuition as ours.

They suggested making a post here to gather insights from individuals within Epic and from the broader developer Community that mind or may have hit the same constraints utilizing UMG in it’s 3D form.

How would you approach the problem space, does a native solve yet exist?

2

Hi,

The native solution here would be Widget Components, have you given that a try? Or is that what you’re referring to when you say world space UI isn’t performant? If you’re seeing performance issues with widget components then we should dig into the cause, they do support invalidation so you should be able to avoid redrawing the widget if the contents haven’t changed. The approach there is similar to what you describe; we use an FWidgetRenderer to draw the UMG widget to a render target instead of directly to the viewport, then apply that render target to a mesh that is created via FDynamicMeshBuilder. The big cost (compared to a screen-space widget) is the allocation of that render target, but that can be mitigated a bit by ensuring the widget size is only as big as it needs to be and avoiding any transparent space on the outskirts.

If there are specific performance issues you’re seeing with world-space widget components and you can grab an Insights trace then we can dig in and see if anything stands out. Having to make separate authoring tools for in-world UI does seem like a very heavy lift, and it sounds like there may be some crossover with the Motion Design plugin already included in the engine (though typically intended more for broadcast television than games).

Best,

Cody