I figured out that there is an issue with Nanite displacement, virtual shadow maps and vrpixeldensity. When I have vrpixeldensity on 1 this issue only appears when I put my head close to the surface. As bigger the vrpixeldensity gets the earlier you can see these issues. At 1.5 it starts to get really disturbing. It would be great if this could be fixed in the next versions or someone has an idea how to get arround this issue. My current UE version is is 5.4.4