Virtual texture normal map issue

Normals off the RVT are in worldspace, so just keep that in mind whenever you sample off it. Hence the worldspace normals would likely work out for you.