Like many others, I want to control a virtual reality experience using a spectator screen.
I have everything working for controlling a widget in the spectator view with the keyboard, but I’m having a small issue when trying to get the mouse to work as expected.
- A pawn with a
UWidgetInteractionComponentthat is turned to face the widget component.
- A function that gets the “absolute location” for a cursor based on the mouse position by getting the mouse x and y and dividing it by the viewport size x and y divided by the pixel density in virtual reality.
- A function that offsets the absolute location for the cursor such that the widget interaction component’s location is correctly set face the widget component, which needs a pivot of
- A widget with a
UImagebehaving as a virtual cursor that has is visible to the user only (no hit).
- A render target that renders out the real scene and displays it on the UI through a user interface material.
- A behaviour such that when the
Pkey is pressed when running in virtual reality the spectator mode is switched to (and from) texture and the texture is set to the render target of the widget component.
This isn’t a hugely efficient setup, as the widget needs to be rendered in world space, but for now it seems to be the best option.
The only big problem now is that the virtual cursor has correct values while not running in VR but incorrect values in VR such that sometimes the cursor is in the incorrect position
It appears that
GetViewportSize doesn’t actually return the correct values when in virtual reality.
Does anybody know a workaround for this?
If anybody has a moment to take a look at the project to help me figure out why the virtual reality position is incorrect, I’d really appreciate it!
The base code for the solution is MIT licensed in this repository: GitHub - calben/UnrealVrSpectatorControl: Example project for controlling a VR experience through UI as a spectator
Thanks for any help!!