Using red and green channels as normal map and using blue as mask?

[anonymous_user_fbe2d247]

@Chaoss

All depends on what you are trying to do…

Personally, where I’m limited in both samplers and memory in a game that requires a lot of textures I’d do the following…

I first pack my tangent space normal map into the ‘green’ and ‘alpha’ channels of the normal map as those have the ‘best’ quality in DTX1 compression and least chance of channel crosstalk, then use the red for my roughness channel and the blue for my metallic.

Then when importing this into UE4 you’ll need to disable SRGB and set the compression to DTX1 which i think in ue4 is just called “Default”. Now in your material multiply both the green and alpha channels by 2 and subtract 1, append this with a third value of 1 and plug that into your normal map channel.

There are some things to be aware of though, the first is you will see some artefacts, now if your designing quite a dirty or post apocalyptic looking game you’ll be all good to go. However if you are designing quite a clean game with lots of flat surfaces you’ll definately want to stick with the standard way of doing normal maps and just using an additional texture for masks.

Lastly if you really are strapped for video memory you can pack the normal data into the and green and blue channels while using the red for roughness, you’ll get some minor crosstalk but should still be acceptable in a majority of cases this way you’ll save half the video memory as you don’t need to use the alpha channel.

If you’re just doing this as a minor optimization step though you are best just using 3 RGB maps, the first being base color, second being the default normal map and the third being a 1/2 resolution RGB texture, If you need to use more masks then use the steps above (packing normals into green and blue or alpha channel) and this will free you up either 1 or 2 additional masks.

You should avoid using alpha channels where possible as this doubles the memory usage of the texture and is a bit of a waste except in the above scenarios where you are also limited by the number of samplers but still need to use 2 masks.

edit: Don’t forget about vertex colors, you can use those as masks as well.

Some problems in the post. DXT1 does not have alpha channel.(there is 1bit alpha channel mode but it’s not supported by engine.) When using rgba texture with default compression it’s using DXT5.
Your normal map encoding is dead wrong. Result is not unit length. Correct way is to use DeriveNormalZ node.(but there is actually bug in the 4.18 so I just use this custom node.

MaterialFloat NormalZ = sqrt(saturate(1.0f - dot( NormalXY, NormalXY) ) );
    return MaterialFloat3(NormalXY.xy, NormalZ);

This is what engine does underhood when using NormalMap compression combined with BC5 compression.(just two channel version of BC4 which is same as alpha channel of dxt5).

Problem with combining other channels with normal map is that you get different quality for x and y. Your method is giving almost perfect quality for y channel but x channel is garbage. Where y channel do get 8-bit endpoints and 8 interpolated values between those. X is getting only 6-bit endpoints and 4interpolated values between those. Also x is getting cross channel talk from roughness and metallic.

“…you are best just using 3 RGB maps, the first being base color, second being the default normal map and the third being a 1/2 resolution RGB texture…”

This is actually really good advice. When using separate textures for separate needs you can reason about texture size a lot more. When there is less artifacts from compression you also can use a lot smaller textures.
Usually I can use 1/4 texture size for metallic map(BC4/Alpha) and emissive.(DXT1). Ambient occlusion is 1/2 or 1/4.
So when you can trust that you get good quality compression you get bigger savings from choosing texture sizes per usage.