Usage of SetupLateUpdate for HMD? (video example)

Development Platform & Builds XR Development
,
1

Hi,

I have a scene capture component that I am updating in my scene. However its position is dependent on the movement of the user wearing the hmd and there seems to be a slight delay (using stale data) at the point of updating the transform.

I have tried peeking into how the camera component is using the late update and have found the following code in CameraComponent.cpp inside the HandleXRCamera().
First the SetupLateUpdate is called, followed by the SetRelativeTransform.

const FTransform ParentWorld = CalcNewComponentToWorld(FTransform());

	XRCamera->SetupLateUpdate(ParentWorld, this, bLockToHmd == 0);

	if (bLockToHmd)
	{
		FQuat Orientation;
		FVector Position;
		if (XRCamera->UpdatePlayerCamera(Orientation, Position))
		{
			SetRelativeTransform(FTransform(Orientation, Position));
		}
		else
		{
			ResetRelativeTransform();
		}
	}

I have tried doing something similar like this:

	IXRTrackingSystem* XRSystem = GEngine->XRSystem.Get();
	auto XRCamera = XRSystem->GetXRCamera();


	XRCamera->SetupLateUpdate(SceneCaptureLeftEye->GetComponentToWorld(), SceneCaptureLeftEye,
	                          Player->Camera->bLockToHmd == 0);
	XRCamera->SetupLateUpdate(SceneCaptureLeftEye->GetComponentToWorld(), SceneCaptureRightEye,
	                          Player->Camera->bLockToHmd == 0);
	SceneCaptureLeftEye->SetWorldTransform(GetLeftEyeTransform());
	SceneCaptureRightEye->SetWorldTransform(GetRightEyeTransform());

I’m not sure if I’m passing in the right parameters, and I’ve tried a couple of different things but it appears that they don’t even make a difference. Is anyone familiar with the late update mechanism and how it should work? It seems as if the documentation for this is nowhere to be found :(.

Example below shows a portal which is showing exactly what is behind it. It’s not visible when the head is not moving, but starting to move your head, the image seems to be delayed.