OpenXR VR Template | Feedback and Discussion

I gotchu:

When we’re referring to runtime in this context we’re talking about the software on your computer that is the interface between your devices (HMD/Controllers) and the application (Unreal Engine). In your case it would either be the Windows Mixed Reality OpenXR Runtime (which is built into recent versions of Windows) or SteamVR. I talk a bit about it in the webinar we did recently: Creating AR and VR Projects with Unreal Engine | Webinar - YouTube

The runtime returns device information to Unreal, so if the runtime returns “Windows Mixed Reality” that’s unfortunately all we have to work with. I suggest you make sure you’re using the WMR OpenXR Runtime, you can find details on how to use it on Msft’s documentation: Getting started with OpenXR - Mixed Reality | Microsoft Docs

1 Like

I noticed some issues with the template when using the Valve Index controllers.

The menu input is set up to use the Index controller system buttons, which are reserved for the SteamVR overlay, so it’s not possible to open the menu without rebinding the input first. If you use the SteamVR OpenXR runtime, it will also reserve the menu button on the left Oculus Touch controller and likely the WMR controllers too, so those would fail depending on what runtime the user has active.

The index controller grip position is offset down, so the controller rendermodels are misaligened. I’m guessing this is a regression on SteamVRs side, so I reported the issue to them too.

Here are the rendermodels with the ones rendered by the SteamVR dashboard overlaid:

@VictorLerp It would be nice if you included a basic “debug 3d-widget” in the template. Something that could be easily extended but could also serve as a basic template for setting one up ready to go.

You mention having to build one in the YouTube video “How to debug your game | Live from HQ | Inside Unreal - YouTube

I’ve noticed 2 issues when using the template, not sure if they are bugs or I’m doing something wrong:

Context: Using Oculus Quest 2, both through oculus link and also launching in standalone mode.
Tests made in:
-PIE ‘VRPeview’ Editor button (Oculus Link)
-PIE ‘Standalone Game’ Editor button (Oculus Link)
-Packaged WIndows
-Launch on Android-Quest2 (for Quest2 Standalone mode)

1st Issue:

  • When launching in PIE ‘Standalone Game’ mode or in Packaged Windows (‘start in VR’ on), the point of view of the user goes to ground level (when I have my helment on, my head is in the floor, like if user’s heigh was 0).
  • With VRPreview (PCVR-OculusLink) or Launch to Quest2 (standalone play) the user’s heigh is ok and plays normal.

2nd issue:

  • If I open VRPawn asset blueprint in the editor and play VRPreview and I try to use the teleport function, the teleport trace does not appear and I’m not able to teleport myself (the teleport blue circle appears on the floor just beside me and never moves from there)
  • If I close the VRPawn asset, all goes back to normal and teleport functionallity is ok.
    I know, this last one maybe is not a big deal, I can allways close the asset before making a VRPreview :slight_smile: but it’s a bit annoying, and mainly I post it here in case it’s a smoking gun for a bigger bug.

Thanks!

1 Like

It’s an issue with the models’ origin, not SteamVR. I thought I got all of the ones that were offset updated in 4.27.1 but apparently not. The “fix” is to re-export the Index controller model with a new origin.

Good call on the system menu buttons. We’ve gone ahead and removed them from the list of available input actions since they can’t be used, and I’m updating the template to use thumbstick press instead.

@VictorLerp Does UE4 come with default skeletal hand models for hand tracking in OpenXR on Quest 2 ?

On 4.27.1, using an HP Reverb G2, the VR mode button is grayed out on a newly created template.

I noticed this gets spat out in the console when I start the project:

Blockquote
LogOutputDevice: Error: === Handled ensure: ===
LogOutputDevice: Error: Ensure condition failed: ((Result) >= 0) [File:D:\Build++UE4\Sync\Engine\Plugins\Runtime\OpenXR\Source\OpenXRInput\Private\OpenXRInput.cpp] [Line: 390]
LogOutputDevice: Error: OpenXR call failed with result XR_ERROR_PATH_UNSUPPORTED
LogOutputDevice: Error: Stack:
LogOutputDevice: Error: [Callstack] 0x00007ffbe23626bd UE4Editor-OpenXRInput.dll!UnknownFunction []
LogOutputDevice: Error: [Callstack] 0x00007ffbe2357dee UE4Editor-OpenXRInput.dll!UnknownFunction []
LogOutputDevice: Error: [Callstack] 0x00007ffbe2354b6e UE4Editor-OpenXRInput.dll!UnknownFunction []
LogOutputDevice: Error: [Callstack] 0x00007ffbe235d80f UE4Editor-OpenXRInput.dll!UnknownFunction []
LogOutputDevice: Error: [Callstack] 0x00007ffbf769174d UE4Editor-Core.dll!UnknownFunction []
LogOutputDevice: Error: [Callstack] 0x00007ffbf6a96433 UE4Editor-Projects.dll!UnknownFunction []
LogOutputDevice: Error: [Callstack] 0x00007ffbf6aacaba UE4Editor-Projects.dll!UnknownFunction []
LogOutputDevice: Error: [Callstack] 0x00007ffbf6a961b0 UE4Editor-Projects.dll!UnknownFunction []
LogOutputDevice: Error: [Callstack] 0x00007ff7ac2431c6 UE4Editor.exe!UnknownFunction []
LogOutputDevice: Error: [Callstack] 0x00007ffbea6cb690 UE4Editor-UnrealEd.dll!UnknownFunction []
LogOutputDevice: Error: [Callstack] 0x00007ff7ac260e2b UE4Editor.exe!UnknownFunction []
LogOutputDevice: Error: [Callstack] 0x00007ff7ac2610aa UE4Editor.exe!UnknownFunction []
LogOutputDevice: Error: [Callstack] 0x00007ff7ac2640cd UE4Editor.exe!UnknownFunction []
LogOutputDevice: Error: [Callstack] 0x00007ff7ac275984 UE4Editor.exe!UnknownFunction []
LogOutputDevice: Error: [Callstack] 0x00007ff7ac27853a UE4Editor.exe!UnknownFunction []
LogOutputDevice: Error: [Callstack] 0x00007ffc7b8a7034 KERNEL32.DLL!UnknownFunction []
LogOutputDevice: Error: [Callstack] 0x00007ffc7d102651 ntdll.dll!UnknownFunction []

I can get it to work with SteamVR, though I have to manually add the bindings for the editor mode. I’ve noticed that in the json files with the SteamVRBindings, I can only see the editor mode bindings for the vive_controller.

Mind, once in game it all works fine, it’s the VR Editor mode that seems to have issues.

There is a hand menu example in the template. Doing the “debug”-version is simply having a function that Print String and dynamically adds a Text widget component to the Menu-widget and sets its text :clap:

Quest 2 hand tracking doesn’t work with OpenXR yet - we’re waiting for Oculus to update their plugin to support that.

2 Likes

VR Editor Mode is currently not supported with OpenXR, but it will be in 5.0. VR Preview works with WMR HMD’s, and if you’re using the Reverb G2 enable the HP Controller Input plugin.

1 Like

Should there be any issue with the XR motion controllers triggering OnComponentBeginOverlap Events? I’m spinning my wheels messing with collision settings wondering why I can’t trigger a box collider with my controllers…

@Victor1erp
Is there any roadmap where the controllers will have rigged button response, e.g. Joystick movement, trigger/grip/button depression?

Or, is this going to be developer side expectation?
and if so, does that that mean no longer using built in Motion Controller Visualization and instead rigging custom controllers?

1 Like

AFAIK no, however, there’s a chance that OnBegin and OnEnd will trigger multiple times due to late updates (special function in the render thread for tracked VR devices) - it’s generally better to trace or check for overlaps manually.

Not on any current roadmap, but it’s something that could happen in the future as the OpenXR specification matures. If that’s a feature you want in your project you’ll have to implement it yourself. You can find all of the individual meshes for controllers and their buttons here: webxr-input-profiles/packages/assets/profiles at main · immersive-web/webxr-input-profiles · GitHub

Note that they’re in .glb format so you’ll have to enable the GLTF importer in UE or convert them in a DDC.

3 Likes

Slightly off topic @VictorLerp but… where should I look to get information about that weird unlit material that did black magic with the sun position (or something like that) to fake shadows that you mentioned on the OpenXR template stream?
Just the name of the technique or some forum post on the topic would be greatly appreciated.

I was talking about using Unlit materials that fake self shadowing by applying a bias opposite of a Vector3 (which simulates a sunlight direction). This method doesn’t apply shadows from meshes to other meshes however, and requires most if not all of your content to be designed with it in mind.

Instead I’d recommend MixedReality Graphics Tools from Microsoft! It’s a plugin that gives you a much cheaper way to do lighting, and it works on Quest: GitHub - microsoft/MixedReality-GraphicsTools-Unreal: Graphics tools and components for developing Mixed Reality applications in UE4.

1 Like

This annoying error started popping when I try to Launch to my Quest 2.

LogPlayLevel: Error: ERROR: System.ArgumentException: Attempt to construct staged filesystem reference from absolute path (/T_Grid.ubulk). Staged paths are always relative to the staging root.

It looks like there is something wrong with T_Grid.ubulk? (that’s a texture on the Template afaik)

The “solution” to this problem seems to be to remove the project via adb every time I want to launch again (I took that from a generic android answer to that specific deploying problem)

What can be so special about that texture in particular that breaks the launch? what is an .ubulk?

PS: tried in various clean projects based on the OpenXR template, and the problem persists, it might be something on my quest tho, a corrupted file there or something like that.

@VictorLerp There’s an issue with the controllers snapping to VRPawn 0,0,0 when tracking is lost. Link to more details posted below, can we get any official comment on this?

1 Like

I’m back from winter break and I got Covid right before new years eve so apologize for late response.

I tested this right before break and I wasn’t able to reproduce it. I 100% occluded my controllers and the MotionControllerComponent would stay in the transform it last reported.

Can you provide me with more details: Which HMD, Controllers, and Runtime are you using?

3 Likes

I’m getting it with the Valve Index headset and controllers with the SteamVR OpenXR runtime. Note that I’ve had the same issue with the 4.26 SteamVR plugin as well, and had to patch the engine to fix it.

Edit: The issue with the OpenXR plugin is with version 4.27.2.

Hi Victor, I’m sorry to hear you got sick, hope you’re feeling better.

Repo Steps:

  1. Launcher Build, 4.27.2
  2. New Project, VR, no starter
  3. Active Play Mode, VR Preview
  4. Play w/ no changes to the project / plugins (Quest 2 HMD plugged in)
  5. Everything plays normal, I move my controller below my desk, it jumps to origin.

I’m able to confirm the controllers are actually occluded using the CurrentTrackingStatus node.