Compositing in Stereoscopic 360 (UE4 graphics over Equirectangular stereo-video)

Hi all,
Learning UE4 here, with some rather ambitious plans in mind. I’m looking at creating a shortform ‘interactive’ version of Dirrogate - the VR experience that I’d done almost a year ago, which was received quite well with the VR community and was invited by Samung’s MilkVR to be hosted on the platform. On the web: http://www.vrideo.com/watch/Zndcocm

or on youtube:

…but getting back,
I’m interested in using video and still backgrounds from scenes, such as this image:


So, the idea behind the workflow would be:

  • Set up a stereoscopic “skybox” that has the left and right image/video wrapped on these skybox spheres (one for left eye / one for right)
  • Create elements such as rocks, bricks etc…in UE4 and place them at the floor/on the road
  • make sure that the appropriate skybox background is seen for the corresponding eye when viewing with a VR headset (Oculus or GearVR) - In Unity I had done this last year via layers and culling for the “Ovr camera”

Does anyone know of way to do this?
If any forum members would have the time to throw even a basic template that I can dissect and learn from, I’d be grateful.

Kind Regards.

It would all depend on the format of the images/video. I take it they would all be over/under?
The images by default would’nt be an issue performance wise but the performance of UE4’s media plugin might be a limiting factor for video.

Ok, understood. Thanks Opamp77.
Yes in most cases the video/images will be over under, but that’s the least of my bother (can do sidebyside or even separate left/right image sequences) what i’m trying to understand is how would I map left and right equirectangular images (skyboxes) to the insides of two spheres such that the left eye (sphere) will not show in the right eye Oculus/gearVR camera output and vice versa.

Any ideas on this part?
Regards.

After some reading, I came across this thread on reddit: Reddit - Dive into anything

Would anyone have some time to make an example scene of what is being spoken about here? I can’t quite understand how to do that half-n-half UV bit.
Best Regards.

This should get you started, It’s not perfect though there seems to be an artifact on the right hand side but it may just be the image you posted.

Thanks so much Opamp77!
I’ll get to replicating this and hope to learn along the way.
Much appreciated!

Your welcome, BTW disabling mips on the texture seemed to help a bit with the artifact.

I tried this out with a couple different movies and it works great wrapping the stereoscopic video but the video looks really pixelated no matter the resolution. Also, I’m seeing a drop in framerate to around the 30fps range regardless of the size of the movie file. Any idea how I can address this?