VR HUD/menu and selecting objects with sight

Dear All,
I’m quite new to UE dev, so I need some help with VR menus and selecting objects in VR :stuck_out_tongue:

I implemented a 3D widget (Creating 3D Widgets | Unreal Engine Documentation), now I’d like to open it in front of the player when pressing a button. So far, no problems, I created a character blueprint and inserted the 3D widget here :slight_smile:
The problems arise because I’m using an HMD (Oculus DK2, but also GearVR), so I don’t know how to select menu items just by looking at them. In the example above, everything works as long as I use a mouse, but in VR I’m supposed not to use it…

Generally speaking, I need to know the exact position and rotation of the HMD in “Character coordinates”, in order to create a Raycast (LineTrace?) and see what object it intersect. I also do not know if this approach works with 3D Widgets, but if not, I’ll recreate the menu using 3D objects.

I can get this right if using a normal camera, without an HMD attached, but if I use it I still have not found the code to get the “real” orientation of the camera (the HMD “Get Orientation and Position” bp does not work alone, and also getting rotation and location of the camera, it seems that they must be mixed somehow).
In my opinion, the focal point is being able to select actors just by looking at them. I will use this also to put a 3D crosshair in front of the player. Billions of demos are doing this (almost all GearVR apps), but I have not found the blueprint code :rolleyes:

Does anyone has a solution for my problem?
Thank you very much,
Christian

Hey Christian,

There sadly isn’t a lot on the net documenting the process at the moment. You’ve certainly got the right idea about raycasting to the widget. Give Mitch’s series a watch as the last 3 videos talk about using the HMD to interact with objects in the world. https://www.youtube.com/channel/UChvlNUgZKmEd-Gul_Tdv8Uw/videos

The trick that most people use is using the HMD’s trace to interact with a dummy volume and then pass that interaction over to the UMG widget.

That’s the way I’d do it right now.

Thank you, I’ll have a look :slight_smile:
Regards,
Christian