Hi guys, I’m trying to make an augmented reality application for Quest 3 that can apply post-process to the real world.
I’ve had a couple of issues just making simple passthrough work at first, but I managed to make it work by using the Oculus fork of the engine (5.5). I could not seem to make OpenXR passthrough work with my Meta Quest 3 (probably something missing in the process on my part) with 5.6.
Now I’m trying to apply some post process effects, like distortion (I have a simple distortion post-process material from the marketplace), but the virtual objects are distorted in a way that doesn’t seem to account for VR: I have two simply different images in each eye that do not work together at all. Also, the passthrough parts are completely unaffected by the post-process distortion.
Do I need materials specifically designed for VR? Is that a thing?
And can I affect the passthrough with post-process at all?
There is no access to directly modifying the camera frames in the regular Quest 3 passthrough system. The frames are composited by the runtime compositor, so the application never has access to them.
Meta has experimental support for accessing partial lower resolution frames using the Android camera API. You would have to manually reproject and render them to use them though, and it will only work on native applications.
Thank you for the response, it was very helpful.
Any insight about simple post-process for virtual objects and environments?
I’ve had a look and it seems that post-process for vr is simply not advised, but I wanted to know if there is a way to do it, even at a steep performance cost, and if it only works on pc applications with a vr headset connected or even with a standalone app.
Most effects should be possible with VR. It’s hard to say how easy it would be to fix without knowing how the distortion is implemented though. Many effects are directly mapped to the render target UVs which don’t correspond well to the VR field of view.
There is always some performance cost on post processing effects, but there shouldn’t be any problem running them on PC at least. AFAIK they are heavy on mobile applications though, and if I remember right, require the Mobile HDR feature to be enabled.