I don’t have the best knowledge of VR, but you could add a camera smoothing to resolve that issue. This isn’t the best of references, however it may lead you in the right direction: Advanced Camera Smoothing in UE4 (Tutorial Part 1) - YouTube
If you are unsatisfied with the tutorial I linked, let’s hope someone else can chime in on the subject.