Download

HoloLens 2 Unreal Research Mode - How?

Hello everyone!

I’m new to this forum, so please let me know if I should improve anything. I am running into trouble using Unreal combined with the HoloLens 2.

I would like

To access the special black/white cameras of the HoloLens, for tracking purposes. These are normally not accessible. However, they can be activated by using the “perceptionSensorsExperimental” capability. This should be possible, since it also works with Unity: GitHub - doughtmw/HoloLensForCV-Unity: HoloLens research mode streams for use in Unity.

I tried

To add the capability in the Unreal Project Settings: Config\HoloLens\HoloLensEngine.ini” → “+RescapCapabilityList=perceptionSensorsExperimental”. The project still builds as expected, but I noticed that it doesn’t matter what I add here. Even something random like “+abcd=efgh” doesn’t break the build.

However, if I add “+CapabilityList=perceptionSensorsExperimental”, I get “Packaging (HoloLens): ERROR: The ‘Name’ attribute is invalid - The value ‘perceptionSensorsExperimental’ is invalid according to its datatype ‘Error’ - The Enumeration constraint failed.”. I conclude: 1.) I’m making the changes in the right file. 2.) The right scheme needs to be configured in order for “+RescapCapabilityList=perceptionSensorsExperimental” to work as expected.

My question

How do I add the right schema to my Unreal project? (like in the Unity example referenced above, which uses “http://schemas.microsoft.com/appx/manifest/foundation/windows10/restrictedcapabilities”), I cannot find any example and I cannot find any proper place to put it. Not in the settings, not in the xml/ini files. I am lost here.

Any thoughts are much appreciated!

Greetings Nico