Hello!
I’ve been having a very similar issue on both 5.3 and 5.4. If the camera gets too close to the displaced / tessellated surface, the GPU drivers crash.
(Latest NVIDIA drivers, RTX3090)
It can be prevented by pressing the Q key on the keyboard to move your camera up the moment the engine hangs, sometimes it moves the camera before the GPU drivers freak out.
From what I’ve been able to tell, moving your meshes closer to world origin (XYZ 0,0,0) fixes this problem. Personally been only running into it when my meshes were very far away from the world origin. Similar to how say, in games your FP arms would start glitching out if you fell thru the world and continued on falling for very long - float point value issues.
Also, as far as I know, RVT does not work on Nanite meshes. Personally, I went around this problem with duplicating all the meshes I need for RVT, disabling Nanite on them, decimating them, giving them a simple albedo-only material, and then I moved those meshes by -200 world units on the Z axis. Then I drew the VTs from these meshes.
Sadly something seems to be completely wrong with RVT in 5.3 AND 5.4, the shading and color is all over the place when shadowed.
Hope this helps!