VR Cursor/Reticle and Raycast from Center of HMD View

Development Platform & Builds XR Development
1

Hi,

I was wondering if anyone has a solid solution for ray-casting from the center of the HMD? I can ray cast from the center of a regular camera and get a pixel-perfect hit on an object and render a decal where the trace hits. The same code fails when rendering in stereo as the center seems to get further and further away from the cursor I’m drawing in the HUD depending on whether I’m looking up, down, left or right. I’ve tried using: GetPositionalTrackingCameraProperties and getting the OutOrigin and using that as the start of my trace, then using a combination of OutOrientation rotating a vector as well as OutOrigin+cameraManager->GetCameraLocation()… it gets me reasonably close to a hit, but as I’m tracing on a surface I can clearly see the drift.


FVector CameraOrigin;
FQuat OutOrientation;
float OutHFOV, OutVFOV, OutCameraDistance, OutNearPlane, OutFarPlane;
hmd->GetPositionalTrackingCameraProperties(CameraOrigin, OutOrientation, OutHFOV, OutVFOV, OutCameraDistance, OutNearPlane, OutFarPlane);

FVector start = CameraOrigin;
FVector end = CameraOrigin + cameraManager->GetCameraLocation() + OutOrientation.RotateVector(FVector(0, 1, 0))*1000000.0;

My HUD cursor is the standard sample code where you get the screen dimensions, divide by two and render a cursor in the center. I don’t know if this is the right solution for VR but that seems to produce a solid cursor in the middle of the screen, but I can’t trace from there. I’ve also tried this to get a hit from the center of the screen to see if that would help but I think it is offset b/c of the VR rendering as well.


const FVector2D ViewportSize = FVector2D(GEngine->GameViewport->Viewport->GetSizeXY());
const FVector2D  ViewportCenter =  FVector2D(ViewportSize.X/2, ViewportSize.Y/2);
playerController->GetHitResultAtScreenPosition(ViewportCenter, UEngineTypes::ConvertToTraceType(ECC_Camera), true, OutHitResult);
2

The crosshair needs to be three dimensional if you want it to be accurate, this is due to the way our eyes perceive depth. Perhaps you should try turning your Cursor/Reticle into a decal that is generated from a separate line trace. That is of course if 100% accuracy is a requirement for what you are doing.

3

Thanks for the tip. This is what I ended up doing but it didn’t really hit me that that made sense until much later. For some reason I figured if I put the cursor at infinity it would somehow be more comfortable, but you’re right, one really does need to line-trace and place the cursor above objects right near the intersection point for it to make any sense at all in VR.

The only issue left now is scale, is there an easy way to keep the crosshair the same size at all times or again is that a comfort thing where you want the cursor to scale a bit with distance? I’m leaving it to scale for now as that’s the default and isn’t uncomfortable.I also had to scale back the position by 95% of the time of where the impact point is on the line trace so that it wouldn’t fight with curved objects in the z-buffer and get clipped, I can’t seem to find a way to disable depth testing on billboards.

4

I’d say it’d feel more natural for a crosshair to scale with distance (bigger nearer, smaller further away). Is that what you ended up going with?

Would be really interested in learning how you did this. It would be great if we could see a code or blueprint example of how you achieved it.

5

I think I’ll leave it to scale with distance both because it is easier and actually does feel more natural when tied to using 3D depth for targetting. It didn’t really hit me that stereo convergence on the reticle does need depth until seeing it for myself :slight_smile:

I’ll post a screenshot of the blueprint when I get a chance but the thing that first hinted at the solution for me was the VR template: GitHub - mitchemmc/UE4FirstPersonVRTemplate: A first person template for unreal engine 4 focused on VR and use with the Oculus Rift and VR Game Template - XR Development - Epic Developer Community Forums

I don’t know if I’m ready to use that as-is, but it is a great reference.

My process in the HUD (or PlayerController blueprint, can’t remember exactly which) was basically



Construction-> add crosshair as a billboard

OnTick->LineTrace
* if impact: draw crosshair at (endPoint-startPoint)*impactTime*.95.  This prevents it from intersecting with the geometry.
* if no impact: draw crosshair at the end point of my trace

The only annoyance is moving the crosshair back by .95 of the impact time as it seems imprecise but I’m intersecting concave surfaces and it looks ugly when half the crosshair is missing.

6

Thanks. This will come in useful I’m sure. Look forward to seeing the blueprint to!