360 interactive videos

Development Platform & Builds XR Development
1

Hi guys,
We recently engage in a project that requires to have 360 live-action videos display with some minimal interactions. The idea is you are in a 360 environment (which is a 360 live-footage around you, supposedly), and when you turn around to watch the 360 video, it will trigger some hot spot and some pop-up info shows up. So my questions are:

  • What is the final format should we export out from UE4 to work right out of the box with VR headset? An actual package? Or can we output a 360 video with interaction (is it even possible)? HTC Vive seems to be our target platform.
  • What is the best way to display 360 videos in my situation? So far I have tried to map the 360 videos via Media framework, onto an reversed normal sphere. It works but I feel like it can be better.
  • And suppose everything above works, then how exactly can I add hot spot/ pop-up windows that will match the perspective of the live-footage?

This is a more hybrid project that combines live 360 videos and interactions rather than a pure 3D project. So any tip/direction on this will be much appreciated.

Thanks.

2 Likes
2

Hi,
i’m building something similar so i have the same question.
I’m making an app for gearVR so i suppos it will be the same for the Vive. An interactive video it’s an app…
Right now the sphere is my main mesh with the mediaplayer, same as you.
For the pop-up an windows just make a plane with a different mediaplayer and spawn it with your current head rotation.

1 Like
3

Hey Silver3D, thanks for answering the question. I will try that solution to see if it works well.

4

This is a difficult one. We have tried to create a walkthrough app with hotspots just using still 360’s and spheres with dynamic materials in UE4 for use on GearVR and it was quite a nightmare, mainly because of the perspective issues of placing a hotspot. Shoot me for saying this, but we had better luck with Unity in this application. It’s very achievable if you export the 360 video from Unreal Engine and use 3rd party viewers, but why do that if you can build to a stereo Andriod App or IOS straight from the engine using your 3D content. If you are just dealing with 360 video, it is way easier to use a 3rd party application to actually handle the viewing of the 360 videos and the hotspots. Google’s VR View SDK could work with support for Mobile HMD’s, VR View for Android  |  Google VR  |  Google Developers but there are quite a few of them out there. I am not sure on the Vive application’s though, you wouldnt really need a premium headset to view 360 videos and hotspots in any case.

5

Here’s what I did (see link).

It works on GearVR, HTC Vive and Oculus Rift.

6

Hi,

Looking into the very same thing.

What I want is a static panoramic start-screen with buttons, so a user can select one and move to a panoramic video. From there they would have the option to go back to the start-screen.
I build these panoramic tours in Unity before, but in Unreal it’s a much harder task it seems.

@Slayemin: Any chance of a small setup to see how you tackled this? I’m building for Google Cardboard btw…

cheers!

rob

7

@Rob Wu: What I built so far is also very much the same as Slayermin’s work. My approach is just simple: using a simple sphere and map the 360 video on it using media texture. I also heavily use 3D widget as menus and popups in my project. I developed everything in Blueprint as it is easier for artist like me. Unity may be a better choice in this situation but it doesn’t have a robust visual scripting like UE4, at the moment.

8

Have you actually gotten any good results with this? I’m currently running into issues with a game i’m developing for a client. The game plays with the background because a 3840x1920 h.264 360 video playing in VR space.

The quality of the unreal video player is drastically lower then playing on a phone 360 video app, or Unity. And no i don’t want to remake the entire game in unity.

I’m in quite a jam, so anyone that can help out would be greatly appreciated.

9

What do you mean the quality of the video? Is it pixelated, drop fps?

10

It sounds like it might be best to work in unity with this, but I’m definitely interested in this as well. I think there’s a lot of practical applications for VR live streams going forward as a platform as a whole.

11

Bump! Any new features now we’re in 2019 and v 4.21? Been keeping an eye on Ue4 and integrating 360 images and video in this way for a while.

12

Nope, I still use the old way of displaying 360 video.

13

I would advise you to have a look at krpano. We recently did something like that with prerendered 360 content from UE4. Went with the route UE4 content → Unity → Devices / Web and to be honest it was horrible because the Google Daydream SDK is in a very bad shape…

Go the web route and if you have to package it as an app look into something like Cordova etc. for the krpano stuff.