[Gear VR] Engine Changes Requested (for Mobile VR Jam)

Development Platform & Builds XR Development
1

Heya VR Peeps.

I’ve been using UE4’s Gear VR support since it went live, in preparation for the Mobile VR Jam. I like it a lot, and it has been just as easy to use as the Oculus Rift support. However, given that this is the first release there are some rough edges that need fixing. Normally I wouldn’t be worried about these issues and expect them to be fixed sometime down the road. But with the Mobile VR Jam deadline coming fast, I wanted to highlight them here and hopefully hear back that they will be resolved soon. In a perfect world these would be patched into v4.7.x rather than wait for 4.8, as 4.8 carries all of the usual uncertainties of a new release. Personally, I’m fine with a source patch and don’t need to wait for a binary build.

I present to you my UE4 Gear VR issue list:

1. MSAA
There are a lot of jagged edges in UE4 Gear VR as we don’t have the same down sampling we do with the Rift. mentioned releasing a MSAA patch to the community in this post.

To show how important this is, John Carmack has this to say in his tweet (from https://twitter.com/ID_AA_Carmack/status/583648614683172864):

2. Virtual Head Model
I brought this up in this post. This mimics the eye’s positional changes even when all we have is the head’s rotation, based on how the human head moves. Unity has this (see Hero Bound: First Steps) and it really improves the experience by lessening possible simulator sickness.

3. Displaying the Gear VR Platform UI
There are three standard platform UI’s that I can think of that are supplied by the Gear VR SDK: Global menu, ‘Are you sure you want to quit?’ menu, and the volume change pop-up. The code for this appears to be in ThirdParty\Oculus\LibOVRMobile\LibOVRMobile_042\VRLib\jni\App.cpp, although it is my understanding that game engines cannot use this code directly. In UE4 we have the OVRGLOBALMENU command, which I assume should open the Global menu, but it currently crashes your game. I’ve tried out a freshly compiled Unity game (not one on the store) and it supports these platform UI’s. In Unity, the Quit menu takes you to the store, even if you didn’t start there.

4. Support for Touchpad Swipes
We have touchpad presses, but we also need Gear VR touchpad swipes. These are common gestures that nearly every game or app will use. It looks like there is code in ThirdParty\Oculus\LibOVRMobile\LibOVRMobile_042\VRLib\jni\App.cpp within InterpretTouchpad() to support these. I would guess that the same code would need to be moved into UE4’s input system.

Thanks for taking the time to read through these and providing a response. I’m not trying to be overly critical of the Gear VR implementation. I just want to make sure that UE4-based games and apps can stand up against the Unity ones during the Mobile VR Jam, and aren’t disqualified for missing required functionality.

2
  1. MSAA patch below. I’ve had issues with the patch and devices running Lollipop, but that’s been hit or miss. We’re still working on a real solution for this.
  2. Virtual head model is on the short list, and should be simple enough to patch into 4.7. I’ll update this as soon as we get something in.
  3. Yes, there’s a JNI threading issue that cropped up. I’m working on a fix as we speak.
  4. This is a bit further down the list and I need to do a bit of investigation to see how involved gesture support will be. I’ll update once I have a better idea.

For a temporary hacked 2X MSAA, replace FramebufferTexture2D() in OpenGLES2.h with this:



	static FORCEINLINE void FramebufferTexture2D(GLenum Target, GLenum Attachment, GLenum TexTarget, GLuint Texture, GLint Level)
	{
		check(Attachment == GL_COLOR_ATTACHMENT0 || Attachment == GL_DEPTH_ATTACHMENT);
#if 1
		static bool Init = false;
		static GLuint DepthBuffer;

		if (!SupportsMultisampledRenderToTexture())
		{
			glFramebufferTexture2D(Target, Attachment, TexTarget, Texture, Level);
		}
		else
		{
			if (Attachment == GL_DEPTH_ATTACHMENT)
			{
				if (!Init)
				{
					glGenRenderbuffers(1, &DepthBuffer);
					glBindRenderbuffer(GL_RENDERBUFFER, DepthBuffer);
					glRenderbufferStorageMultisampleEXT(GL_RENDERBUFFER, 2, GL_DEPTH_COMPONENT24, 2048, 1024);
					glBindRenderbuffer(GL_RENDERBUFFER, 0);
					Init = true;
				}

				glFramebufferRenderbuffer(GL_FRAMEBUFFER, GL_DEPTH_ATTACHMENT, GL_RENDERBUFFER, DepthBuffer);
			}
			else
			{
				glFramebufferTexture2DMultisampleEXT(Target, Attachment, TexTarget, Texture, Level, 2);
			}
		}
#else
		glFramebufferTexture2D(Target, Attachment, TexTarget, Texture, Level);
#endif
		VERIFY_GL(FramebufferTexture_2D)
	}
3

Thank you, JJ! :D:cool::smiley:

4

Thank-you for the most excellent response, JJ. I’ll give the MSAA patch a go later tonight!

5

You can get basic virtual head camera movement out of the spring arm component (camera boom) in relation to pitch and yaw. It’s not as accurate as a heavily tested mathematical equation but for me it will lean my camera back when I look up and lean my camera forward when I look down almost identical to what my physical eyeballs see when I do the same motion in real life. It could be worth looking into until an actual virtual head is implemented.

Also points1, 3 and 4 are quite important for the VrJam. I am almost certain that the abilities those specific points address were mentioned in the rules and regulations wall of text / judge comments.

6

Hey JJ.

I’ve implemented your MSAA changes and everything looks great. Thanks!

7

Have you noticed any difference in performance?

8

There’s definitely a hit to perf with 2X MSAA, but it’s absolutely worth it. If you were borderline 60fps before, MSAA will take you below 60. It’s worth optimizing content to get back up to 60. I found 4X MSAA to usually be too large a hit to be worth it.

9

I’ve updated the plugin to the 0.5.0 mobile SDK.

https://github.com/EpicGames/UnrealEngine/commit/010b67402085fbef8a0e77c7e37834b7366421de
https://github.com/EpicGames/UnrealEngine/commit/7e8d7ed115ff3c63701cda1fcd0f3d15eaa7263a

And then fixed the back button global menu support in the following commit. Note that the global menu fix is dependent on the 0.5.0 integration.

https://github.com/EpicGames/UnrealEngine/commit/1c0dd94397c9c4a92f2aa2e7c9da8e0126dd5393

10

Also, the 0.5.0 integration isn’t as scary as it looks. The code changes in the plugin are minimal. Most of that commit is just removing the 0.4.2 libs and adding the 0.5.0 ones.

11

Hey JJ:

Excellent, thanks! You’re really pumping these changes out, and it is much appreciated. I’ll see if I can merge these into my 4.7.x branch tomorrow as I don’t want to move my project over to master.

I look forward to your next set of changes (perhaps virtual head model?).

12

It should be an easy merge into 4.7. I’m trying to be careful about that for at least the next few changes. I’d like to have something that folks can just zip up and overlay over a 4.7.x install. There are a few changes coming in the not-too-distant future that may break that, but I want to get as much taken care of before that as is reasonable.

Head model is next on the list as it currently stands.

Just let me know if you have issues getting things merged into 4.7.x.

13

That sounds great! will wait for this zip before playing around with manually overlaying stuff from the master - not something I’ve done before!

PS: Hoping we don’t have to wait too long for point #4 from the top post - pretty important imo

PSS: MSAA patch works great btw! :slight_smile:

14

So if I wanted to get all of these latest updates, could I just do a master zip download and install? I don’t really mess with Github past just downloading a release, no messing around with merging, etc.

16

Hey Peeps!

I’ve merged in the changes that JJ posted above into my 4.7.5 branch and I now have 0.5.0 SDK support and the Gear VR global menu working in my game. A couple of things to note when doing your own code merge:

  1. I had to manually merge all changes to Engine/Build/Commit.gitdeps.xml as there are a bunch of differences between the master and 4.7 branches that might mess things up.
  2. There is a subtle bug introduced in Engine/Plugins/Runtime/GearVR/Source/GearVR/Private/GearVR.cpp that I didn’t initially catch that prevents the GearVR module from compiling. JJ’s code from master uses FMemory::Memzero(VrModeParms); while in 4.7 we need to use FMemory::MemZero(VrModeParms);. See the case difference? Took me a bit to figure it out too (debugging at 2am Yay!).

So rather than put you all through that, I’ve made my changes public:

https://github.com/DavidWyand/UnrealEngine-Fork/tree/VRJam_4.7

You’ll still need to have a GitHub account that is part of the Epic Games organisation to see this, of course. I’ll make sure that this branch remains over the course of the Jam. Here is what it includes beyond the stock 4.7.5:

  • MSAA hack
  • 0.5.0 SDK support
  • Gear VR global menu fix
  • Virtual Head Model hack (added April 16)

Enjoy!

17

Dude, you are awesome. :cool:

Downloaded your VRJam_4.7 fork, yet knowing that I was so close earlier, with the Memzero error on packaging, I want to give it one last attempt. However just knowing that yours is here and ready to go makes me feel so much better and inspires me to want to try again.

Man, that MemZero catch is a total Jedi move. :lightsaber: Thank you, . :smiley:

18

You are the man! Thanks!

19

; thank you very much for this. As a BP-er I was really postponing to figure this all out :smiley:

Thanks again!

JJ of course also thanks… for having this ready for the Jam.

20

THANK YOU ! And thank you JJ!

21

, I wanted to let you know that I’ve decided to run with your version - after reading through your changes and notes in the branch and considering the whole situation, I just feel better about running yours - and am so very grateful. :smiley: