Use of mesh painting on ISM or PLA

Development Pipeline & Plugins
, ,
1

What is the “Epic” way of doing modular buildings when it comes to breaking repetition? We are building big city-like environment separated into isolated tiles with a big number of buildings. Our requirements are:

1) rich variation of building shapes

2) being able to reuse same buildings in multiple places

3) being able to edit buildings in a centralized manner (==building templates)

4) being able to add unique artistic polish to each building instance

We invested into modular building composition with templating by using Packed Level Actors (PLA). Right now we have a big number of unique buildings built from ~6m modular nanite pieces and packed into PLAs. This workflow closes requirements (1)-(3), but we just cannot figure out what to do with (4). Our environments look “plain” and “repeatative” and Art direction expresses a dire need to have an ability to add a hand crafted building-instance-based variety to the environment the way we did prior to Nanite: blended materials + vertex painting to add dirt, stains, stucco/brick combinations etc. Given our current pipeline we can’t figure out a way to achieve this and need an advice.

Because we use Nanite, we obviously can’t use Vertex Painting. Our original plan was to migrate to UE5.5 and use Mesh Painting to enable blended materials on our Nanite environments. Recently we did some prototyping in UE5.5 and it seems Mesh Painting is not available for Instanced Static Meshes (ISM), And ISMs are PLAs’ core. So our initial plan is invalidated. How should we proceed in our situation? Is there a setting we’ve missed on Mesh Painting that would allow it’s usage on ISMs? What is “the epic” way of doing modular buildings?

Options we considered:

- Wide decal usage to add variation? We have an impression that it is advised against using decals with Nanite because of performance implications. Though how bad this really is?

- Usage of masked opacity meshes instead of decals? Same worries as above.

- Merging buildings with the “Actor merging” tool? We will lose centralized modification ability, increase memory consuption because of prolifiration of huge unique building meshes instead of modular piece ISMs.

- Breaking PLA into separate mesh actors to paint them as a final destructive touch on the environment? This would make the game heavy, right? Because no ISMs, huge number of actors.

- Use meshes to add details? It is just not “smooth” enough.

- Make more modular piece variations and produce even more buildings variations? Basically balancing building template reuse vs building uniquness. Even this way individual modular pieces would feel repeatative. And introducing each new set dressing to the same buildings would cause a combinatorial explosion of either modular pieces or buildings needed.

- Fancy procedural materials with CPD, world coords, actor coords, normals, etc?

- Some custom tool to produce low-res volumetric textures that would be somehow injected into a material of each individual building PLA instance to emulate instance MeshPainting. How can it be implemented?

- Any ideas for possible engine modifications?

1 Like
2

Hey there!

Up top I wanted to highlight that we added support for mesh painting on Nanite meshes by going the virtual texture route rather than per-instance vertex colors. Check out the documentation for more info. We haven’t talked about it much, but I’d recommend exploring that route as I think it can help you meet all your requirements in a way that can be both performant and memory-efficient. This is effectively the last option you outlined, we just took a different approach.

As you noted, and what I want to reiterate: I won’t recommend is the Merge Actors tool to merge multiple meshes into a single static mesh asset. Nanite rendering of modular pieces is really efficient so long as they’re all using the same material, so a combination of virtual texture mesh painting, custom primitive data, and per-instance custom data can all be combined to reduce the total number of Material Instance Constants in the scene.

Hope that helps!

3

Hey Matt!

Thank you for getting back to us.

We probably weren’t clear enough.

We intended to use Texture Painting as you advised, but it is not available on InstancedStaticMesh components, it seems!

We successfully tried it out on StaticMesh components - it works fine.

But on ISM components the options to enable texture painting are grayed out, texture painting mode is not available. We weren’t able to find a way to make it work. And it is a problem for us, because the only type of component in PackedLevelActors is InstancedStaticMesh component. So it seems we just cannot paint virtual textures on our PLA buildings.

So is Virtual Texture Mesh Painting really not working on ISMs?

Can you give us further advice how to proceed?

4

Hey there! I confirmed with the devs and unfortunately it’s expected that the new mesh painting doesn’t work with ISMs (I’m also learning this for myself just now, apologies for not independently verifying that sooner). I’m not sure what the timeline looks like for getting that implemented, so I’m not confident it’ll land with 5.7.

So that being said! The alternative approach you can explore is likely decal-based. You can use decals to blend between two layers, too! If you set the the base mesh’s Decal Response to “None”, you can then use the DBufferTexture node to sample one of the decal channels to drive your layering instead of having to come up with something per-instance. I’d caution that you’ll want to rigorously test the performance impacts of this for your project with a beautiful corner example that pushes the limits. In my experience, it’s very possible to cause some significant performance headaches with too many decals in your scene.

Thoughts?

5

Hey Matt, thank you for the idea.

We started prototyping and will discuss the Custom Decal Response approach internally.

But just to double check, are there other expected benefits of using Custom Decal Response materials in our case? Except driving the material layer blending, of course. Maybe there are some positive performance implication over a usage of plain decals for decorations?

6

The other benefit I can see over using plain decals is that you can keep the evaluation cost in the surface material rather than in the decal, but given modern hardware I’m not sure just how much of a benefit that would be. Another benefit is that you’d be able to go beyond the color/roughness/normal channels on your surfaces.