UE5.3 Meta Quest - Passthrough with OpenXR

I’m developing an app in Unreal Engine for the Meta Quest headsets. Everything works fine on the headset when I configure my development environment following the proper instructions (this is a good starting point, for those looking for it).

I want to adapt my VR (or fully immersive) app to support AR (or passthrough). The project uses the Meta XR and Meta XR Platform plugins, and relies on the OpenXR plugin to achieve a good performance with movable lights and dynamic shadows.
However, the only way I can get Passthrough working on the headset is by disabling OpenXR… This is not really an option for me because, for some not yet fully understood reason, framerate drops significantly without the OpenXR plugin.

Is there no way to get Passthrough working with MetaXR + OpenXR? Is this an OpenXR bug? Has anyone been able to get this setup working and would it be possible for someone to help me achieve this?

Here’s my setup:

  • Unreal Engine v5.3
  • Meta XR plugin v60.0
  • Meta XR Platform plugin v59.0

does this help?
AgileLens/Passthrough-Demo: Starter project for tethered PCVR passthrough with Meta headsets (github.com)

I’m just replying here for other people searching this issue.
We have a Quest 2, 3 and Pro device in our studio. All have significant performance impact when using MetaXR Plugin as opposed to clean OpenXR.
The performance impact is best seen when using instance meshes. You can essentially have 5-10X more instanced static meshes rendered on the device without the MetaXR plugin. This makes using the MetaXR plugin impossible for anything that’s more complex.

Haven’t tested the latest 62 version of the plugin. Will report back here.

Somehow there isn’t enough noise anywhere about these massive performance issues so I’m thinking it’s perhaps limited to instanced static meshes performance.

And I’m also curios to find any alternative options for managing passthrough on a Quest device without using the MetaXR plugin. Although, for the time being, this seems to be the only way as passthrough is treated as an extension feature by OpenXR :frowning:

3 Likes

in theory you could call passthrough directly through the OpenXR API, but I haven’t seen anyone pull that off yet
https://registry.khronos.org/OpenXR/specs/1.0/html/xrspec.html#XR_FB_passthrough