In case you or anyone else are still having this problem - I also had the same problem when I was contracted to fix 360 video performance in someone’s project, and delved into the engine code to work out why. Basically, there’s a fundamental problem with the way that unreal currently plays video via Java which means it sucks at playing video when you’re using Vulkan on Android. Unity has a similar problem, but there is a plugin you can buy to fix it (Renderheads AVPro).
Given there wasn’t anything to buy to fix it for me, I built a plugin to fix it and have put it for sale on Marketplace. It enables high performance fully hardware-accelerated playback of high resolution video. I can play 8k 360 video without any frame slowdown on a Quest 3 (or an older Pico Neo 3 for that matter), i.e. video is smooth, and the Quest continues to run at 72 fps. Right now it only works if you’re playing local video from assets, i.e. not for streaming video, I’ll add that if there’s much demand for it.
https://www.unrealengine.com/marketplace/en-US/product/direct-video-android-beta