6k 360 video is stuttering


I am currently working on a VR project, ideally having it run via a Pimax, alternatively HTC Vive.

However, a 360° video I am trying to implement suffers from strange framerate drops and stuttering in
the medie player preview and ingame while running it outside of Unreal works as intended.

Video specifications are H.264 mp4, 60 fps, bitrate 1, 6k (6144 x 3072).
Is there anything I am overlooking to make it work flawlessly?

Here are the settings I have for a 360 video: