Normal Maps - Is There Some Sort of Formula To Get Things Looking Right..?

You won’t get a 3D effect from normals in VR. It will only look good from a distance or for really small surface details. You will need to add the macro surface structure as geometry.