Good thing is you’re not alone. I’m an Emerging Experience MVP (meaning covering , Kinect, Cortana, Ink, and Band on Windows 10). There are quite a few of us pushing for UE support as well. If you don’t build a bridge to UWP and Holographic API’s then looks like me and my colleagues will have to 
I’m very much interested in using UE as opposed to Unity.