Announcement

Collapse
No announcement yet.

Trouble creating a virtual cursor to control an experience through widget on spectator screen

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

    [OCULUS] Trouble creating a virtual cursor to control an experience through widget on spectator screen

    Hullo everyone,

    context
    ----
    Like many others, I want to control a virtual reality experience using a spectator screen.
    I have everything working for controlling a widget in the spectator view with the keyboard, but I'm having a small issue when trying to get the mouse to work as expected.

    what i've got so far
    ----
    1. A pawn with a `UWidgetComponent` and a `UWidgetInteractionComponent` that is turned to face the widget component.
    1. A function that gets the "absolute location" for a cursor based on the mouse position by getting the mouse x and y and dividing it by the viewport size x and y divided by the pixel density in virtual reality.
    1. A function that offsets the absolute location for the cursor such that the widget interaction component's location is correctly set face the widget component, which needs a pivot of `(.5f,.5f)`.
    1. A widget with a `UImage` behaving as a virtual cursor that has is visible to the user only (no hit).
    1. A render target that renders out the real scene and displays it on the UI through a user interface material.
    1. A behaviour such that when the `P` key is pressed when running in virtual reality the spectator mode is switched to (and from) texture and the texture is set to the render target of the widget component.

    problems
    ----
    This isn't a hugely efficient setup, as the widget needs to be rendered in world space, but for now it seems to be the best option.
    The only big problem now is that the virtual cursor has correct values while not running in VR but incorrect values in VR such that sometimes the cursor is in the incorrect position

    what seems to be the root of all evil
    ----
    It appears that `GetViewportSize` doesn't actually return the correct values when in virtual reality.
    Does anybody know a workaround for this?


    If anybody has a moment to take a look at the project to help me figure out why the virtual reality position is incorrect, I'd really appreciate it!
    The base code for the solution is MIT licensed in this repository: https://github.com/calben/UnrealVrSpectatorControl

    Thanks for any help!!

    #2
    For some reason, I can't access your file.

    If it's rendered in world space, why can't you use a LookAt function to shoot a line from the camera to the mouse, use that rotation to get a forward vector and shoot a widget interaction from the mouse to the widget.

    The position issue when in vr mode is probably caused by incorrectly recentering on the center of the percieved field of view for the hmd and not the center of the left eye.
    Last edited by NotSoAccurateNo1; 11-09-2018, 01:04 PM.

    Comment

    Working...
    X