Interlude: Translucency Sort Order
One problem I noticed when stacking decals is that the sort order wouldn’t respect geometry. You can see it in the image below. Even though the screws are placed well above the paint strip, the paint gets rendered after the screws and as a result overwrites the screws’ color information. That’s not what I want.
It’s true that UE allows users to define the sort order for translucent geometry and decal actors, but only relative to other actors. The problem is that the above does not show two actors fighting for the Z, it’s two materials on the SAME actor. None of the settings available to us handle a situation like that. It gets even more annoying when it’s the same material that contains stacked decal geometry.
There are several possible solutions for this problem. The first one would be to split overlapping decals into separate actors. Looking at the image above, I could import the wall as the first mesh, the paint strip as a second and the screws as the third mesh. I’d place all of them in the level and could set the translucency sort order in the actor settings so that the screws would always render on top of the paint.
I’d also quickly start to hate the **** out of it. Having three separate actors for what should be only one, always taking care to move and rotate them in sync, and maintaining sort order values that don’t conflict with other translucent pieces of geometry that might enter the scene just sounds like trouble. I’m doing this to make my life easier.
So I took a look at what detemines the order in which different materials on the same mesh are rendered, and - to the surprise of probably no one - it’s the order of the material slots on the mesh. The top-most material is rendered before the one below it and so on. Which means all I have to do is make sure that the order of the material slots corresponds to the intended sort order of the decals.
Maintaining Material Slot Order
As far as I could find out, the order of material slots on an imported mesh is determined by the application that exports the fbx file. The Unreal editor preserves that order on import. So this part is not really related to UE but to Maya, Max, Blender or whatever application you use to build and export your 3D models.
I use Blender, which is free software and thus made it easy to spot where the order is determined. It’s not something the artist has any immediate control over, so I modified the code in the fbx exporter to allow for some user control. The modified exporter is on GitHub, but please be aware that I also changed the way how the exporter handles root bones on exported skeletons. This is a longstanding cause of confusion especially for people new to Blender and Unreal, but I don’t want to derail this thread, so I invite you to google it. Just keep in mind that if you use the modified exporter for skeletal meshes, an additional root bone is never created and UE imports it without modifications to the skeleton.
The way Blender’s fbx exporter sorts meshes now: If you’re using the binary exporter and “Selected Objects” is ticked, the meshes inside the fbx file will be sorted alphabetically, according to their names as shown in the outliner.
This allows you to easily control the order of material slots. You could call the wall mesh “AA”, the paint mesh “AB” and the screw mesh “ZZ” and it would guarantee that the screws are rendered last. The result is this:
Note that it doesn’t really matter what the other meshes are called, as long as the first instance of a paint mesh is sorted alphabetically before the first instance of the screw mesh, and if you don’t have overlapping decals on a mesh, you can ignore all of this completely. Also note that Blender sorts case-sensitive. I realize that giving meshes some random names does not fit everyone’s workflow. Mine is highly modular even inside Blender, using instanced meshes and linked duplicates, so I usually don’t care about the names of the meshes that are about to be exported.
I don’t know if Maya or Max users have similar problems with sorting material slots and unfortunately I lack the time to dig into it. I hope you are fine, but if not, you can always switch to Blender 