Announcement

Collapse
No announcement yet.

Tutorial: Create a VR cinema with stereoscopic 3D support (UE 4.5 , no C++)

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

    Tutorial: Create a VR cinema with stereoscopic 3D support (UE 4.5 , no C++)

    If you've got an Oculus DK, you've probably already used a VR cinema application. Thanks to UE 4.5's new Media Framework, now you can create your own very easily. Keep in mind that the Media Framework is not finished yet, so make sure your video files are in WMV format to avoid problems.

    - Create a Blank Blueprint Project (Desktop/Console, Maximum Quality, With Starter Content).

    - Create a new Empty Level, save it, and make it the Game Default Map and Editor Startup Map in the Project Settings.

    - In the Content Browser, go to StarterContent/Architecture and place a Wall_400x300 in your level. In the actor's details, set the Z component of its location to 0.0

    - In Windows Explorer, create a folder named Movies inside the Content folder of your project, and copy your WMV files into it.

    - In the Content Browser, right-click on Game and select New Asset -> Miscellaneous -> Media Player.

    - Open your Media Player asset, set Auto Play to true, and choose your video file in the URL field. Save; close the window and open your Media Player asset again. If you press Play you should be able to watch your video file. If not, choose another video file.

    - In the Content Browser, right-click on your Media Player asset and select Create Media Texture. Save your new Media Texture. It should already work correctly.

    - In the Content Browser, right-click on your Media Texture asset and select Create Material.

    - Open the new Material. It will look like this:

    Click image for larger version

Name:	tut01.png
Views:	1
Size:	60.5 KB
ID:	1139682

    - Since the level is in total darkness, you need to make the material emissive:

    Click image for larger version

Name:	tut02.png
Views:	1
Size:	62.5 KB
ID:	1139683

    - You can now save the material and apply it to the wall. You should already see the video playing in the viewport, but not the way you want it to:

    Click image for larger version

Name:	tut03.png
Views:	1
Size:	458.6 KB
ID:	1139684

    - The UVs of the mesh are not right for our purposes. The wall is 400x300 units, and is showing 4x3 videos which are horizontally inverted. We'll fix that in the material, and in the process add support for videos with different aspect ratios, and for SBS (side-by-side) stereoscopic 3D videos. The final material looks like this:

    Click image for larger version

Name:	tut04.png
Views:	1
Size:	118.2 KB
ID:	1139685

    - Let's look at its different parts:

    Click image for larger version

Name:	tut05.png
Views:	1
Size:	178.8 KB
ID:	1139686

    Here we add a Scalar Parameter to control the Aspect Ratio of the video. By default it's 16:9 (1.778), but our wall is 4:3 (1.333), so we divide those values. We multiply the texture coordinates by (-0.25, 0.333) to get rid of the 4x3 horizontally inverted videos problem. This gets any 4:3 video to show correctly, so we multiply by (1, AspectRatio/1.333) in order for any video to show correctly. The last thing we want to do is put the bottom of the video at the bottom of the wall, so we add to the previously computed value (0, AspectRatio/1.333 * -0.333). The constant 300 corresponds to the wall height. We divide it by AspectRatio/1.333 and we'll use that later.

    Click image for larger version

Name:	tut06.png
Views:	1
Size:	174.4 KB
ID:	1139687

    This part adds support for SBS videos. In these videos, the left half corresponds to one eye, and the right half to the other, which is exactly what happens in the frame buffer when rendering to the Oculus DK. We use the ScreenPosition's X component to be able to know which eye the currently rendered pixel corresponds to. [0.0, 0.5) is one eye and [0.5, 1.0] is the other. We have to multiply the computed texture coordinates by (0.5, 1) to make each half of the video fit the entire wall. And then we add (0.5, 0) to the coordinates of one of the eyes, so that each eye gets a different half of the video. Note that different HMDs and different videos might require to switch which coordinates are received by each eye in order for the videos to show correctly. The last thing to do is to add a Scalar Parameter to enable the stereoscopic mode.

    Click image for larger version

Name:	tut07.png
Views:	1
Size:	157.2 KB
ID:	1139688

    Finally, we'll add a black stripe on the top of the wall if the Aspect Ratio is not 4:3. A TV adds two stripes, one on the top and one on the bottom, but we'll take advantage of the fact that the level is in total darkness to make things simpler. The wall's bottom is at Z = 0.0, and its top is at Z = 300.0 . The white line that is connected to the B of the If, is the result of 300/(AspectRatio/1.333), which we computed earlier. This value represents the height the wall should be to fit a video of the current aspect ratio. Instead of scaling the wall, we'll just force the pixels of the wall that are above that height to be rendered black.

    - Great! All that's left to do is give the user the ability to change the values of the Scalar Parameters we created: AspectRatio and Stereo. We'll need to create a Dynamic Material Instance. The proper way to do it would be to create a Blueprint for the wall, but for this simple tutorial we'll just do it in the Level Blueprint. Select the wall in the viewport, open the Level Blueprint, right-click and add a reference to the wall. Then do this:

    Click image for larger version

Name:	tut08.png
Views:	1
Size:	91.4 KB
ID:	1139689

    - Finally, we'll implement some controls to change the parameters. Add two float variables to the Level Blueprint called AspectRatio and Stereo, and then do this:

    Click image for larger version

Name:	tut10.png
Views:	1
Size:	292.4 KB
ID:	1139690

    Now you can quickly select one of the three most common aspect ratios, and enable or disable stereoscopic 3D.

    That's all that can be done without C++ for now. Once playback can be controlled and more video formats are supported, this is going to be great.

    #2
    Awesome - I especially like the SBS 3D support. I was planning to set something like this up for the loading screen videos, but that UV setup looks like it would have been a pain to figure out from scratch - thanks!
    Storyteller - An immersive VR audiobook player

    Dungeon Survival - WIP First person dungeon crawler with a focus on survival and environmental gameplay ala roguelikes

    Comment


      #3
      Goddamnit...

      Been trying to figure the SBS stuff out for ages, can't believe it was so simple. Excuse me whilst I go and smash my face against the wall for a couple of minutes.

      Comment


        #4
        Hi LeonardoBarbero,

        First of all thanks a lot for detailed description on this. This works like a charm on a Quad. However, we have a scenario where we are using different projection methods. e.g> Dome, Spherical, latlong etc. When I tried the above method on the geometry which has custom UV layout for the movie texture playback(stereo SBS), it failed. Simple example would be that of curved screen with tweaked UV layout so that when movie is projected on that screen, renders correctly.

        Coming from an Unity Engine background, I got this done simply by creating two set of geometries at exact same locations. Unity's Oculus camera setup involves two physical cameras present. So the trick is to render one geometry with LE movie renders as texture using one camera ( left camera layer ) & render another geometry with RE movie render using second camera ( right camera layer ).

        Unreal's oculus setup is pretty opaque so not sure what happens under the hood. For your approach to work then perhaps I need to tweak the material itself ?

        Please let me know in case you have any idea on this.

        Regards,
        Rupesh.

        Comment


          #5
          Thanks a lot and now I'm able to set a screen wall though I'm not acquainted with those DMI stuff! One more question is that how should I play the sound at same time? I tried to split out a media sound wave and set it to play but in no avail.

          Comment


            #6
            Feel like a noob. Simply can't find out where to find "Target static mesh component" and the "Target materials" in picture tut08.png What is it that im missing??

            Comment


              #7
              I want to achive an effect of 3d sbs projection but within a game not as a player.. and effect that can be found in oculus rift demo called insurgent. There are 3d videos sbs with actors( actores are keyed out from green screen. I tried to edit the material to achive similar effect but iam nowhere near of findg a solution could sombody point the right direction I am total noobe when comes to ue4 and its materials.
              insurgent effect https://www.youtube.com/watch?v=_TyuuQJUcmk

              and thats what i have so far= means nothing.

              Click image for larger version

Name:	12.jpg
Views:	1
Size:	340.4 KB
ID:	1071618
              Last edited by wizardman; 03-31-2015, 01:48 PM.

              Comment


                #8
                Hi Leonardo, thanks for the walk-through, it was pretty easy to follow!
                It would have been impossible to figure out on my own!

                I didn't need the ability to change aspect ratios and turn on/off stereo,
                so I simply gave the Stereo variable a value of "1" by default.


                Now, I'm trying to figure something out, but I hit a wall.

                Say instead of the wall I have a cube, and instead of the video I have a texture loaded externally,
                or 6 textures for each side (12 for stereo), I guess your approach should work?

                I'm struggling with the cube mapping, and making each set of textures to be visible for one of the eyes only.

                Would you be able to point me in the right direction, please?
                Senior 3D Artist

                Comment


                  #9
                  Hi! There are more noobs out here. Did you ever figure this one out?

                  Comment


                    #10
                    Thx dude

                    Comment


                      #11
                      Hi,


                      tkx for the tutorial! unfortunately i cannot find a way to connect the static mesh to the array material in the level blueprint. Its like the array material before the get cannot get targeted.

                      how did you do that ?

                      Comment


                        #12
                        Hi [MENTION=18033]milko[/MENTION] - did you ever get a reply to this question as I'm keen to learn how to do it also -

                        Cheers!

                        Comment


                          #13
                          Can someone help guide me on how to get this to work for over under 3ds?

                          Comment


                            #14
                            Can anyone help here? I will pay pal them a hundred bucks if they can figure this out for me ASAP! message me please.

                            Comment


                              #15
                              how I can add the Seats and the Cinema Background?

                              Comment

                              Working...
                              X