I’ve been googling and trying various combinations of settings here, and I can’t make it work, but it seems like something that should be so trivial to accomplish, I’m sure I’m missing something.
I’ve got an actor which casts a ‘fake shadow’ using a render texture target and decal on a boom arm projected to the floor. The reason for doing this is so the shadow can be used to gauge jump/landing position; it always appears directly below the player, and will show up even when standing in shade. This is a pretty old 3D platformer trick.
This works fine in 99% of cases. But, I’ve got some objects in my game world which are unlit (because they use a special cel shader model) and I cannot project the decal correctly on to these, and I need to.
Via experimentation, I’ve discovered that the reason for this seems to be that when dealing with emissives, decals (DBuffer or GBuffer) only work additively; i.e. they can only lighten the scene. Since the decal in question is a shadow, obviously, that’s not really ever going to be desired behavior. No matter what blend mode I try; translucent, stain, alphacomposite, DBuffer color, I always get something between “it’s not there” and “it’s a slightly whiter area” depending on the color beneath it. Is there no way to have a decal darken an unlit (or emissive, as I have tried and gotten the same results setting my unlit materials to be lit and simply fully emissive) material? Surely if the results of the decal are written to a separate buffer, it’s just a question of how that buffer is composited with the other rendering passes, right? There must be a way to tell it “use the dbuffer opacity channel to lerp between the scene texture and the decal texture, don’t do any other adding or multiplying”, right?
EDIT: a picture for clarity. On the left, you can see the stencil shadow rendering as intended on a standard material, but on the right it’s disappeared as the engine tries to render it over an unlit “cel” material.
I want to swing back by for posterity’s sake and say I did come up with a solution for this, and if anyone stumbles upon this post later with the same issue, I can explain
Basically, what I did was say to myself “a decal is just a transparent material with some view angle trickery. If I can get a transparent material to behave how I want, just not projected, then all I have to do is manually do the projection”. Though I started to do this by hand, mercifully I discovered there’s already a material function (no docs for it, I believe it’s user submitted? If so thanks to that user) called StaticMeshDecal_Function which does this work.
You create a 1x1x1 corner-pivot box (the engine has a prefab mesh that suits this), scale it to the desired size like a decal actor, and apply a translucent material which uses this function to generate UVs for your decal texture and a mask (well, a series of masks, I used the camera clip and extent bounds masks, the angle one I didn’t understand) you multiply together with your decal mask. It creates a material for the translucent box that behaves like a decal actor’s bounding box, and you just place it as normal (though the Z axis of the box is the decal depth plane, rather than the x-axis which is what decal actors use, so if you’re swapping out you may have to adjust some rotations)
I will maintain that having to go to these lengths was silly. The fact that this was so readily doable once the shader math was hooked up makes me wonder why exactly I couldn’t just use this ordinary translucent blending mode behavior in the decal space by default! This cost me a CustomStencil channel to mask out objects I didn’t want to receive this decal (namely, the character itself) since I couldn’t use the “Ignore Decals” flag and dbuffer to do it, but in the end it worked. I just feel like it’s reinventing the wheel a little bit, surely this is 99% of what decal materials are already doing on the back end, right?
Anyway. If you’re one of the users (and I know they’re out there, I’ve seen the unanswered forum posts in my searches to fix this) stuck trying to get decals to behave like ordinary translucent materials (especially against unlit materials) this will work, so don’t despair.
You have to use the 1x1x1_Box_Pivot_-XYZ static mesh for this, and then you freely scale it to set your bounds. You can find it in .../Engine/Content/Functions/Engine_MaterialFunctions02/SupportFiles/1x1x1_Box_Pivot_-XYZ.uasset
As far as I can tell, this is the only way to project decals onto Unlit materials currently (for projected drop shadows and stuff).
