So I have been going through Tor Frick’s tutorial from Eat3D and trying to adapt his approach on modular textures from UDK to UE4.For vertex painting rust, he uses a normal map’s blue channel to store a mask then use it to create a blend mask in his material. When I try this however, it looks like ue4 is scrapping the blue channel and not displaying that mask as long as the compression is set as a normal map. If I change it to color it shows up.
The whole purpose is to have vertex painting conform to the pattern in the mask to give a more realistic look to the painted rust.
The point of picking NormalMap compression is that it throws away the blue channel to save space. So if you want to keep the blue channel to use for something else like this, use the default compression.
UE4 uses BC5 compression for Normal Maps Textures now. This means that both the Blue and the Alpha Channel will be unavailable for use. If you want to go back to DTX Normal Map Texture compression you can, you just need to follow the information that is in this blog.