"LockToHMD" causes HMD tracking to be world rather than parent relative

Development Platform & Builds XR Development
, , , , , ,
1

In 4.11, in the VR camera refactor, a “bLockToHMD” flag was added in order to make tracking the player head in world space easier.

However, the way this was implemented, the head transform is always applied in world space, no matter which way the camera component is actually facing (due to being attached to a parent component which can rotate). So leaning forward always causes the camera to move on the world +x axis, even if the player pawn is facing down the y axis.

The root of the problem is in UCameraComponent::GetCameraView:

if (bLockToHmd && GEngine->HMDDevice.IsValid() && GEngine->HMDDevice->IsHeadTrackingAllowed())
	{
		ResetRelativeTransform();
		const FTransform ParentWorld = GetComponentToWorld();

		GEngine->HMDDevice->SetupLateUpdate(ParentWorld, this);

		FQuat Orientation;
		FVector Position;
		if (GEngine->HMDDevice->UpdatePlayerCamera(Orientation, Position))
		{
			SetRelativeTransform(FTransform(Orientation, Position));
		}
	}

Should be:

if (bLockToHmd && GEngine->HMDDevice.IsValid() && GEngine->HMDDevice->IsHeadTrackingAllowed())
	{
		ResetRelativeTransform();
		
		FTransform ParentWorld;
		if (GetAttachParent())
		{
			ParentWorld = GetAttachParent()->GetComponentToWorld();
		}
		else
		{
			ParentWorld = GetComponentToWorld();
		}

		GEngine->HMDDevice->SetupLateUpdate(ParentWorld, this);

		FQuat Orientation;
		FVector Position;
		if (GEngine->HMDDevice->UpdatePlayerCamera(Orientation, Position))
		{
			SetRelativeTransform(FTransform(Orientation, Position));
		}
	}
2

I think you have to use an empty “Scene” base as a parent to set the position and rotation, but there still seems to be issues with it. These questions are related, and one of them mentions a submitted fix.

3

The empty scene base lets you properly offset the camera position in the world. The problem is that translation from head positional tracking is applied in the parent’s local space, but rather to world space, whereas rotation is applied in the parent’s local space. This means that if you rotate the parent, the camera’s rotation inherits the rotation, but then when you move your head the translation is applied out of alignment with the current head rotation. (i.e. if the parent is rotated 90o right, the camera looks 90o right, but if you lean forward the camera moves 90o left relative to its orientation.)

The post with the fix you linked above was mine, and the fix I was referring to was the one in this bug report. It hasn’t been pushed to the git project, because of lawyers and the fact that I don’t …entirely… understand why it works. (SetupLateUpdate heads into rendererland and I think the root problem is a timing issue. There there be dragons.)

4

By bad. Didn’t realise you were the same person. I hope someone who understands it completely will push a fix to GitHub and soon after release a hotfix.

5

Hello Teiwaz,

I was able to reproduce this issue on our end. I have written up a report (UE-29579) and I have submitted to the developers for further consideration. I will provide updates with any pertinent information as it becomes available. Thank you for your time and information.

Make it a great day

6

I was having the same issue with the late update translating the wrong direction when having the player start not rotated at 0,0,0. Though I don’t see UE-29579 in the 4.12.3 hotfix changelog (4.12.3 Hotfix Released - Announcements - Epic Developer Community Forums), when I merged the 4.12.3 code into my current engine branch, and it seemed to fix the issue. Now whenever I rotate the player start everything works correctly.

I haven’t tried to rotate the player in-game, but translating it (teleporting) doesn’t cause an issue.

Maybe this JIRA# was also a fix for this issue? UE-31787 : Positional Latency w/ Rotation

Here’s the code change inside of SteamVRHMD.cpp from GitHub:

     if (bImplicitHmdPosition)
       {
 +         const FQuat DeltaControlOrientation =  View.ViewRotation.Quaternion() * TrackingFrame.DeviceOrientation[vr::k_unTrackedDeviceIndex_Hmd].Inverse();
           const FVector DeltaPosition = TrackingFrame.DevicePosition[vr::k_unTrackedDeviceIndex_Hmd] - View.BaseHmdLocation;
 -         View.ViewLocation += DeltaPosition;
 +         View.ViewLocation += DeltaControlOrientation.RotateVector(DeltaPosition);
       }

Thanks,
Cameron