There's something really weird happening with the camera component in VR

Development Platform & Builds XR Development
1

I’m using Unreal Engine 4.13 and the Oculus CV1 for room scale. I still need to test this with the Vive. I’ve decided that I want to manually control the position and orientation of the camera in VR. I setup all of the necessary code to manually control the camera, and it works!!!

However, there’s this really weird thing happening which I can’t quite describe. If you select the camera component and set “Lock to HMD” to true, the world looks crisp and at the correct scale. However, if I untick that checkbox, the world appears to be slightly distorted. I feel like I’m slightly crossing my eyes and as if I had shrunk slightly. I check the Z values for my camera, with the lock enabled and the lock disabled, and they are exactly the same. I am not changing any IPD settings. I’m not changing the world to meters scale at all, and when I check the value with both settings, it’s exactly the same. I am also not changing my screen percentage, that value is still exactly the same. Likewise with the aspect ratio. Yet, toggling “Lock to HMD” seems to change the perspective. I have no idea on why exactly the world looks different.

So. I decided to look into the engine internals for the camera component. If you open up “CameraComponent.cpp” and then find “GetCameraView(float DeltaTime, FMinimalViewInfo& DesiredView)”,
you’ll notice this section of code:



if (bLockToHmd && GEngine->HMDDevice.IsValid() && GEngine->HMDDevice->IsHeadTrackingAllowed())
{
	ResetRelativeTransform();
	const FTransform ParentWorld = GetComponentToWorld();
	GEngine->HMDDevice->SetupLateUpdate(ParentWorld, this);

	FQuat Orientation;
	FVector Position;
	if (GEngine->HMDDevice->UpdatePlayerCamera(Orientation, Position))
	{
		SetRelativeTransform(FTransform(Orientation, Position));
	}
}

This code section is what is causing the VR camera to NOT be distorted. If you don’t run this code section, you get the slightly off, cross eyed & shrunk perspective. I suspect that it has to do with the call to “SetupLateUpdate()”.

Just out of curiosity, I decided to modify this part of the engine source slightly:



if ((bLockToHmd || bLockRotToHmd) && GEngine->HMDDevice.IsValid() && GEngine->HMDDevice->IsHeadTrackingAllowed())
{
	ResetRelativeTransform();
	const FTransform ParentWorld = GetComponentToWorld();
	GEngine->HMDDevice->SetupLateUpdate(ParentWorld, this);

	FQuat Orientation;
	FVector Position;

	if (GEngine->HMDDevice->UpdatePlayerCamera(Orientation, Position))
	{
		if (bLockToHmd)
		{
			SetRelativeTransform(FTransform(Orientation, Position));
		}
		else if (bLockRotToHmd)
		{
			SetRelativeTransform(FTransform(Orientation, FVector::ZeroVector));
		}
	}

}

Sure enough, if I have “bLockRotToHMD” enabled, the perspective looks absolutely perfect. But now, I can manually control the camera position without the HMD movement interfering.

Just as a quick heads up to anyone listening… there are times when you’ll want to manually control the orientation and position of the camera, even if its attached to the HMD. In my game, I want to block the camera from clipping through collision geometry, so to move the camera, players move their head as usual, but the actual camera movement is done with an HMD delta value. IF you have “LockToHMD” enabled, then the manual delta movement gets doubled and the movement ratio becomes 1:2 instead of 1:1. So, to get the 1:1 HMD to camera movement ratio without a second movement update, you have to manually move the camera yourself by the HMD frame position delta.