Hello everyone, in my project I want my game to be somewhat connected with real life. That’s why I want it to see its way in real life with a setting in the game. Is something like this possible from within Unreal Engine?
e.g: https://developer.oculus.com/blog/mixed-reality-with-passthrough/
That depends a lot on what headsets and platforms you are planning to support. Most platforms that support passthough implement it differently from each other.
Meta seems to only support passthorugh on the Quest using their proprietary OpenXR extensions, and according to their documentation they only seem to have support for UE4.27:
https://developer.oculus.com/documentation/unreal/unreal-passthrough-overview/
Unreal has support for the OpenXR core spec passthrough mode as well. AFAIK only Varjo has implemented it on the runtime-side though:
https://developer.varjo.com/docs/unreal/ue5/mixed-reality-with-unreal5
There is an unofficial WMR API layer for it too:
https://github.com/mbucchia/WMR-Passthrough
SteamVR also provides limited passthough support through OpenVR for the few headsets that actually provide camera data to it. I have written a UE 4 and 5 plugin for it:
https://github.com/Rectus/UE4SteamVRPassthrough