In Blender, make sure you are using flat shading, not smooth. The normal map should be plugged into the normal of the displacement node, not the normal of the BSDF. If you are using cycles, you need to change the displacement setting in the material to “displacement only.”
Edit: Ignore what I said about plugging in the normal. You should not actually need a normal map when you have a displacement map. As the renderer should have a good idea or where each part of the surface is pointing.