Is it possible to have a VR application supporting pixel streaming and how to do it?

Development Programming & Scripting
1

I want to develop a multiplayer VR application that uses pixel streaming to provide the video and audio to the users over the internet. If it’s possible, how to do it?

2

Hi ZL1202 welcome to the forums,

Cool question, but I don’t think that pixel streaming would be able to provide the sort of latency required for comfortable VR experiences. (I’ve yet to see any VR pixel streaming OR Multiplayer pixel streaming apps.)

My thought is that If you change the direction you are looking in an HMD, needing to wait for your new look direction to be sent to a server, and for that server to compute the frame and send it back to you would feel delayed/sluggish and cause nausea. Also network hiccups might be ok in a 2D app, but might not be acceptable in VR.

I’d love to be proven wrong though! Surely one day in the near or distance future we’ll be feeding our AR/VR glasses with cloud-computed images. :sunglasses: :satellite: :zap: :desktop_computer:

3

Hi ZL1202,
about VR application and pixel streaming, I think it is possible by way:

  • server render full dom (equirectangular ) like 360 video in youtube, It will solve delay when change the direction of HMD
  • client using web-xr (threejs) to display video
  • internet require stream big video(8k)
    both server and client will using more resource in case 2d app
1 Like
4

hey @dragon1999

Do you have any experiences distributing your unreal engine scene to customers through cloud computing services like azure?

5

that’ 360 video BUT not realtime game…

6

still no successful project using Pixel Streaming for XR HMD? latency by network still cannot be solved?

7

Hi gpu,

This video came out a while after I made that post.

I’d call that ‘successful’, but like the video states, it’s still ‘early days’ for this sort of thing.

8

Hi gpu,
if you don’t care about latency you can refer this link
I test Pixel Streaming in UE 5.1 with 2 game instance one for left eye and one for right eye

9

Hi guys,

I know this is an old thread, but it’s worth sharing some information regarding Pixel Streaming with XR.

Pixel Streaming does work with XR, but you are right that the latency introduced by running it on remote instances makes it nigh unplayable. The latency along with head turning/tracking can be uncomfortable for some users.

It is however very usable if running on local networks! If you’re interested in setting up basic VR Pixel Streaming, I’ve made a tutorial here:

Let me know if you had any further questions!

1 Like