Hi community,
I created a VR application last year on Oculus Quest 2 with Unreal Engine 4.27.2
It’s a serious game where you take place as a conductor, it integrate 4 larges 360 videos captured with the musician of the Montreal Symphony Orchestra.
Here a video of the final resuly :
First i dev only for Oculus Quest 2, add a different headset will need to create different build for each one…
And at the time VR was not implemented on UE5, and i can said it’s better to stay on old stable product that go on fancy new engine and get stuck forever.
1- Visual studio 2019
Pption Game / tablet
2- Oculus + Android Studio :
https://developer.oculus.com/documentation/unreal/unreal-quick-start-guide-quest/?device=QUEST
I installed Android API 31
I got save by this video ! At the end he show how rename and rewrite a java file : d8.jar to dx.jar
Here a screeshot of my Android preference :
Yes i installed android version 31 but i write 29
I allow large OBB file because i got large video to feed.
But i you can try to keep you app light and package everything in the .apk, it easier to export and install.
Important is the Build section :
I really try all combinaison, and i try first new vulkan and at the end i have to go back to the calssic OpenGL 3 because Vulkan didn t allow my to use video inside Widget, it get black ! It was a terrible situation for me because all the Ui on the Conductor pupitre was done with Widget and discover that at the end of the projet, i had to delete all the widget and redo it with Blueprint.
Here my Oculus VR setting
Lot to say about colors, i didn t have the same color on the oculus plug in on my computer and since my apk exported, so you can imagine how had it was the back and forth…
Video :
For optimization we cut the 360 video to render only the part where was the musician move as video, the ‘room’ was render as Still image but fade each other because at the beginning and at the end, the lighting of the room change.
So use Color rec 709 in Ae.
In the Oculus doc they say that oculus is a Rec2020, but i used rec 709 too.
We export a lot a time video to match Image and video colors
A lot of coodec was used too.
I try vp8, vp9, av1, mkv, ogg, 3gp, too much to return to only the mp4, here the command line for ffmpeg
ffmpeg -i “input.mov” -c:v libx264 -preset slow -crf 17 -x264-params mvrange=511 -pix_fmt yuv420p -an -movflags faststart “output.mp4”
Audio :
Ochestra mean good audio. We thought ambisonic sound format will help us in this VR spacialisation… and noo
Unreal Engine only supports first-order ambisonic assets = So only 4 channel = Mono + L-R (AKA mid-side stereo)
So after this desilution the Sound Studio had to recreate all the audio source inside of Unreal and animated it depend of the choose of the user instrument selection. You can see the audio source in the screeneshot and the Sphere Mesh Cut.
To blend image and video, we had a nice god ray. When the user select an instrument, the partition appear over them.
What is the most performance consuming in VR mobile app ? TRANSPARENCY.
i just can say avoid it.
Important Foreward rendring for performance.
AntiAliasing : MSAA
I got problem with Scene capture in mobile, we try to avoid video at the begining, for the partition just a camera animated in X axis, could have make the job. but this is an old bug report :
A chance that it was a Unlut scene.
For material quality don t forget to use Full Precision inside Mobile option.
I learned a lot during this projet, but i’m not ready to do it again.
I hope it helps a bit some of you.
