360 StereoPano Export Method

Hi folks. I’ve been hearing about KiteAndLightning’s plugin coming soon, but have heard no official word from Epic on the matter (It’s in their hands now). I got tired of waiting, so I built my own using captureCubes, BPs, Materials, and Matinee. No plugins required, but there is a pretty serious limitation. 2k x 2k per eye. This is a hard limit on how high a captureCubeTarget can be set. Even with this limitation, the resulting sideBySide 4k x 2k panos are still pretty nice, so I’m totally willing to share the solution until a better one comes along. Here’s an image to show what it can spit out.

There are some improvements to make, and a few bits I need to clean up, but the method is working. This image is a sideBySide stereoPano. There are some minor seam artifacts that can be improved upon, but this serves as a base method for correct stereo on the 4 main axis. Zenith and Nadir can be problematic, but I’ve been negating stereo on them in finals anyway.

Many thanks to RyanB for making this possible, the math of which can be found in this thread.

I understand completely that captureCubeTargets were never meant to be used this way, but I would ask that their resolution limit be unlocked, so that exporting higher resolutions could be done. It would be very useful! (OK, now that I’ve unlocked the higher res renderTargets, I see why you left them at 2k;) It’s not for the feint of heart. It’s very, very slow, and eats ram for breakfast. It can also crash your video drivers due, I suppose to hogging up video ram. Buuuut, with care, you can get it to work.

UPDATE: Got 8k animated stereoPano export working. Here’s a vid.
Note, YouTube has not at this time enabled stereo360 vids. So in an effort to showcase the working stereo aspect, I left it as a normal video but with red/blue stereo enabled. As soon as YouTube gets that switch flipped, I’ll update the video for cardboard and gearvr.

1 Like

Second Version Released! Have a go!

Tutorial video here.

General Usage Instructions:

  • Make a copy of the project you wish to record. You’ll be making some changes to its functionality, so keep the original in-tact
  • First open the panoCam project file and migrate all the assets in the panoCam folder to your desired project copy.
  • In the panoCam folder, you’ll see 3 blueprint actors. BP_panoCam, BP_panoCaptureSBS and BP_panoCaptureTopBot.
  • Drag BP_panoCam to your scene. It is your camera. Place it where you need, and you should be able to animate it (translation only for now) using Matinee if you like.
  • Drag either one of the BP_panoCaptures(TopBot or SBS) into a non-visible part of your game level. Somewhere outside the view of the panoCam. This is used to record and composite the output of panoCam
  • In Matinee (I’d create a new one and label it pano"something"), while you have the BP_panoCapture selected, right click in the bottomLeft zone in matinee to “create empty group” The captureCam should now be active as that track
  • In Matinee, create a new directorGroup
  • In the directorGroup, set a key, and choose the captureCam, hit OK.
  • Make sure to enable this new Matinee to play by editing your Level Blueprint. onBegin, play panoMatinee (take time here to disable other things in LevelBlueprint that might override normal playback of your scene. Remember, you should be using a copy of your original scene). If you have toggles for reducing quality for VR, or fullscreen, or other oddities, you should disable them.
  • Use Matinee movie export to record your image sequence. Set res to 4096x2048, enable cinematic mode, disable texture streaming.

Limitations:

  • 4k by 2k max res by default, but you can edit a small bit of source code to get 8k support on a graphics card with decent ram.
  • Rotational animation not yet supported
  • Only side-by-side stereo for now. Working on a toggle to allow single view output.

*** Note: Bringing the panoCam into your scene will slow it down. By default (to avoid bogging down systems too much) the 4 capture cubes (Front, Right, Back and Left) are set to 512. When it comes time to export your 360, set them each back up to 2048 for best quality.

Nice work, 3dlight!

Did you though about not 4, but more cameras? In theory you can get less terrible sews…

My assumption at the start of this, was that I would have to go with more than 4 angles. However, I decided to start with 4 and see what the issues were, since it was still theoretically very effective. I prefer as few stitching seams as possible. If I go to 8 or more, then there are stitches everywhere. While they become less apparent, they are still there, and there’s more of them. I’m also avoiding some stitching issues by rendering the full 360 distorted view for each of 4 angles. They tend to blend pretty nicely, but yeah, seams are still an issue. In my tests so far, the seams aren’t that bad though. I wouldn’t call them “terrible.” :wink:

Performance is also an issue. This isn’t a 30fps sort of thing. fps will slow down to something like 3fps with this in your scene, so rendering even more views could make it very unwieldy. I’ll probably create a quality control to improve performance while mucking about with your scene, then at rendertime, set it back up to full res. I’ll also experiment with 8 views, since there is some possible optimization headroom. In the meantime, I figured some folks might find this useful.

Andrew

BTW, that obvious “seam” down the middle is not actually a problem. That’s the division between the L and R view and not a stitching seam. When viewed in stereo, that simply doesn’t exist.

First off, great tutorial and thanks for sharing the pano cam! So I have a silly noobish question- how do I put these rendered 360 pano images to to proper use after following all the steps in the video tutorial? I tried loading them into a software like kolor eyes 3d, and even following youtube’s 360 video setup, but the panoramic image is distorted. Do these only work in stereoscopic mode or can I set them up to work like the 360 videos on youtube (but with a single frame for example).

This currewntly exports a stereo-only side-by-side image. Only apps that are set up to support that format will work with this currently. Apps that I know work are:
whirlygig (DK2) and the GearVR image viewing app. I haven’t tested KolorEyes, but this page:
kolor | GoProOculus_Rift#2D.26_3D
At the bottom, says you can set to top/bottom or sideBySide stereo. Select SideBySide and it should work. I can try to test it soon.

You could always crop one of the eyes (halves) of the image and view as mono… but the resolution wouldn’t be great.

I’m adding some features. One of them is Mono output (already in my beta) and another is previewing the stereo, with control over the zPlane depth, and IPD while viewing the stereo in a DK2. This feature is in progress.

I just had a glance at the desktop version of KolorEyes documentation a bit more, and it seems it relies on “Extended Mode” Which is basically history now with Oculus. While it is still supported for a few more weeks until 7 is out, it’s a hack that I won’t spend time testing. Extended Mode must die;)

Thanks for the response! I’ll check out whirlygig and also try it on the GearVR when I get a chance.

I agree with the extended mode. Can’t wait for the .7 update!

Updated the first post to show 8k stereoPano export.

Can someone explain to me how to turn the left and right eye HDRs into a stereoscopic video file using Adobe After Effects, please? And what to do to put it on Youtube as 360 stereoscopic video?

Monoscopic is working for me, but I’m struggling with the stereoscopic workflow :slight_smile:

Hi 3dlight! Thank you for sharing the Pano cam. The only think is that I have a problem :frowning: when I export/render (even with a very small resolution: 640x480), at the beginning the first frame, how do you said, is incorrect but after a couple of minutes its very slow and it stops (like crushing) and begins again to render wrong. What I have to do? I’ve done everything step by step following your video tutorial. Can the issue regard the version? I have the 4.9. Thank your in advance!!

If you have a long sequence, you may have to split it up in frame chunks. To fix the frame becoming incorrect after a certain duration, go to the matinee and increase the timeline in the director track. I’m guessing that once you get to the end of the timeline in the director track, the camera is reverting to the default cam/character.

Thank you! Now the render works fine :slight_smile:

Hi 3d light,
Great Blueprint, I’m still struggling with the upload to youtube as a 360 video. Have you experimented with that and got some pointers?

Thanks
Roy

I haven’t tried yet. Going to attempt today, and I’ll let you know.

Note: I have updated this to support 4.10, and added Top/Bottom and Mono export in addition to the SideBySide in the original. See the second post for the new link.

After some trial and error, I got it to work. Here’s the method I used to get YouTube Stereo 360 to work:

  1. Used my tool to export a TopBottom at 4k square (the updated tool now supports that, though to get true 4k wide per eye, you’ll need to hack the source code)
  2. Used the YouTube metaData tool to inject. Settings were “360” and “3d Top/Bottom”
  3. Uploaded to YouTube (while uploading DO NOT check the advanced option for “Make this video 3D” That can invalidate your metadata injection)
  4. Waited about 10 min for my short test to get processed properly. DONE!

I tried with SideBySide, but had no luck, thus I added TopBot support.

Andrew

Thanks, will give it a try

Hi Andrew, thanks for this great tool!

I wanted to give you my feedback after spending 2 days on it.

My PC is a I7-4770k, Geforce gtx980ti, 32gbRAM and I use the minimal default scene (2 seats+table) for my test in 4.10.

Everything works fine if I don’t animate the pano camera.

I just added a straight path to the camera in Matinee.

It starts to render correctly but then after a couple of frames, either it stops or crashes randomly.

-I tried with TopBot or SBS
-I tried to lower the rendering res (512*256 and 256 for the render target cube)
-I tried the same test with your default file PanoSetup.
-I tried with 2 different PC

Still the same issue.

Is there any chance you could improve that?

Do you by any chance have an hmd, like the dk2 on while you are exporting? I’ve found that it needs to be turned off to be stable.