The problem with flatten normal is the method DOES alter the blue channel on the normal map. That method is intended to flatten the normal so when the results are completely flat, you get a pure 0,0,1 normal. Adding strength will start skewing the z value of the normals in the opposite direction, resulting in the material requiring backlighting. And while the multiply add command works as intended, it uses 3 more instructions than necessary to get the same result using the proper method. The only suggested method in this thread that works is the one posted by RI3DVIZ: it’s actually 1 instruction cheaper than the flatten normal method, since you’re only affecting 2 channels instead of all 3, and it produces the right strengthening effect without altering any z-values. The only problem with that method is you can’t get a perfectly flat normal: the R and G channels will flatten to 0, but not B/Z. However, as long as there is some definition in your map, the method works great and looks fantastic. This is actually the method I use in my marketplace materials to strengthen normals, though I achieved it simply by masking the RG channels, multiplying that, then appending the blue channel.
If you’re not careful with your normals, you will have funky lighting appearing on the other side. At that point you’d be much better off using parallax or parallax occlusion methods with ambient occlusion to get the extreme level of depth you’re looking for.
Scaling either x and y with t or z with 1/t is exactly same operation after normalization.(this is always done for normal output) Only difference is that when you scale with z you can’t get fully flat normal because t = 0 is not defined. But in reality you can just use really small t value for same effect. Difference is only single ALU so this isn’t likely most crucial optimization.