FSceneView::ProjectWorldToScreen issue in VR

Development Programming & Scripting C++
1

During a VR test session, FSceneView::ProjectWorldToScreen function returns a wrong screen coordinate when attempting to trace a point from World. This is my code:


//Get the current Application Window Size
FVector2D WinSize = LocalPlayer->ViewportClient->GetWindow()->GetSizeInScreen();
FIntRect Rect(0, 0, WinSize.X, WinSize.Y);

//Get ViewProjectionMatrix from Local Player
FSceneViewProjectionData ProjectionData;
if (LocalPlayer->GetProjectionData(LocalPlayer->ViewportClient->Viewport, eSSP_FULL, /*out*/ ProjectionData))
{
    ViewProjectionMatrix = ProjectionData.ComputeViewProjectionMatrix();
}

SceneView->ProjectWorldToScreen(ObjectWorldLocation, Rect, ViewProjectionMatrix, ScreenPosition);

//Test
PlayerController->SetMouseLocation(ScreenPosition.X, ScreenPosition.Y);

This is instead the final result:

Cattura_mod.PNG.jpg
Cattura_mod.PNG.jpg1282×722 37 KB

The orange laser end point is the ObjectWorldLocation coordinate, while the red dot is the ScreenPosition result.

What’s wrong with my code? Any suggestions?

Thanks in advance.

2

Bump, I really need a suggestion for this issue. Please, help me!

3

Call the function from UGameplayStatics:: instead.

4

I have already tried this way, but it failed too (moreover, it was even worse than calling ProjectWorldToScreen from FSceneView, because it didn’t respect the size of the window where I’m running my VR project). Really, I don’t get why the two points are projected in two different locations instead of being placed in the same coordinate on the screen…

Anyway, thanks a lot for your reply. :wink:

5

I’ve made another test using a Sphere actor as final target and then I’ve runned the project either without VR (i.e. playing in New Editor Window (PIE)) or with VR (i.e. playing in VR Preview). In the former case, the target is correctly pointed by the mouse cursor (it stands still even when I move the character around the world with the keyboard) like this:

ezgif191ea5e5d3952-2.gif
Instead, in the latter one, the mouse cursor “flies” around the target when I’m moving the VR controllers and/or the HMD (like in this GIF):

ezgif191ea5e5d3952-2.gif

I’ve also analyzed GetProjectionData throughout several debug sessions in order to realize what kind of values it calculates to generate ViewProjectionMatrix: it seems all regular apparently, but I think that the issue is due to the stereoscopic rendering in VR. Is there a method to compute correctly the matrix (and so the projection point from 3D to 2D) taking into account the stereo view, so that the cursor won’t slide when the character is in movement?

6

Hmm… At this point I would file a bug report and pray :slight_smile: